Add a way to access the current test var in :each fixtures for clojure.test

Question

When looking at (log) output from tests written with clojure.test, I would like to be able to identify the output associated with each test. A mechanism to expose the current test var within an :each fixture would enable this.

One mechanism might be to bind a test-var var with the current test var before calling the each-fixture-fn in clojure.test/test-all-vars.

Proposed: The proposed patch (clj840-20161122.diff) allows 'each' fixtures to access the var associated with the currently executing test by using * }. As a result of this change, 'each' fixtures are able to access the metadata associated with the current test var, including the name.

The patch achieves the above by changing the order in which functions are wrapped when a test and its associated 'each' fixtures are run. Before this patch, 'each' fixtures were combined into a single higher-order function, which was then given a thunk containing an invocation of the {{test-var}} function to execute as its body. After this patch, the {{test-var}} function is now responsible for joining and executing 'each' fixtures but, importantly, it does so within the scope of the binding expression that adds the current test var to * }. {{test-var}} now invokes the joined fixtures function, rather than the joined fixtures function being given a thunk that invokes {{test-var}}.

Patch: clj840-20161122.diff

Comments

Hello, I wanted to weigh in, saying that I would find the addition of this functionality useful in the Clojure test libraries I work on.

For my use-case, Duncan's 4-line patch that adds `*test-var*`and binds it around the `each-fixture-fn` form would suffice.

I would use this feature in a re-implementation of the `state-flow` test lib using base `clojure.test` blocks (`deftest` + fixtures) to allow for better integration with the existing Clojure tooling ecosystem. Without access to the current test var in the fixture I cannot find a way to perform certain test setup in the fixture (https://gist.github.com/philomates/32f8e1f7a2a4e746ac0186964aaaa2d3#file-flow-clj-L74 sketches out the use-case in code)

Answer 1

Comment made by: stuart.sierra

Or just pass the Var directly into the fixture. Vars are invokable.

Answer 2

Comment made by: hugoduncan

I don't think that works, since the the function passed to the fixture is not the test var, but a function calling {{test-var}} on the test var.

Answer 3

Comment made by: hugoduncan

Patch to add test-var

Answer 4

Comment made by: stuart.sierra

                                                                *  } already has this information, but it's not visible to the fixture functions because it gets bound inside {{test-var}}.

Perhaps the {{:each}} fixture functions should be called in {{test-var}} rather than in test-all-vars. (The namespace of a Var is available in its metadata.) But then we have to call {{join-fixtures}} inside {{test-var}} every time.

Answer 5

Comment made by: stuart.sierra

Try this patch: clj840-2.diff.

This makes * } visible to {{:each}} fixture functions, which seems intuitively more correct.

BUT it slightly changes the behavior of {{test-var}}, which I'm less happy about.

Answer 6

Comment made by: hugoduncan

Might it make sense to provide a function on top of * } to return the current test-var?

Answer 7

Comment made by: stuart.sierra

No, that function is {{first}}

Answer 8

Comment made by: hugoduncan

I agree with having the dynamic vars as part of the extension interface, but would have thought that having a function for use when writing tests would have been cleaner. Just my 2c.

Answer 9

Comment made by: jafingerhut

With a commit made on Nov 22, 2013, patch clj840-2.diff no longer applies cleanly to latest master. Updating it appears like it might be straightforward, but best for someone who knows this part of the code well to do so.

Answer 10

Comment made by: joelittlejohn

I'd find it very useful if this one was fixed.

I've added an updated patch that has the same content as clj840-2.diff but applies against the current master (c0326d2), as of November 22nd 2016.

Answer 11

Comment made by: joelittlejohn

I realise I've only translated a patch provided by someone else here, but if there's anything further you think this one needs before it's in a fit state to be considered then please do shout and I'll endeavour to add something further. Thanks.

Answer 12

Comment made by: alexmiller

If you could update the ticket to better describe the approach of the patch that would be helpful.

Answer 13

Comment made by: joelittlejohn

The proposed patch (clj840-20161122.diff) allows 'each' fixtures to access the var associated with the currently executing test by using * }. As a result of this change, 'each' fixtures are able to access the metadata associated with the current test var, including the name.

The patch achieves the above by changing the order in which functions are wrapped when a test and its associated 'each' fixtures are run. Before this patch, 'each' fixtures were combined into a single higher-order function, which was then given a thunk containing an invocation of the {{test-var}} function to execute as its body. After this patch, the {{test-var}} function is now responsible for joining and executing 'each' fixtures but, importantly, it does so within the scope of the binding expression that adds the current test var to * }. {{test-var}} now invokes the joined fixtures function, rather than the joined fixtures function being given a thunk that invokes {{test-var}}.

Hopefully that's clear, ish :)

Answer 14

Comment made by: joelittlejohn

Any chance this one could be considered for inclusion now? The file clj840-20161122.diff is still valid and can be applied to the current HEAD of master.

Answer 15

Comment made by: alexmiller

Can you check that changing the function ordering here hasn't affected the stack trace inspection in do-report? Basically it would be helpful to know in all the various cases (neither each nor once, each but not once, once but not each, and both) that error reporting is unaffected for all those cases. Also, tools like Leiningen monkeypatch parts of this as well. I think all of that's fine, but I'm probably not going to get around to checking all that soon.