Welcome! Please see the About page for a little more info on how this works.

+8 votes
in Test by
retagged by

Set Up

clojure.test is built on the idea that a test will call an assertion function that directly calls a reporting function: is calls report (through do-report). The function report is a multimethod, allowing for adding or overriding keys to allow for implementing alternative reporting styles. It's also dynamic, meant to be rebound if an entirely new reporting system is to be used (TAP, JUnit, etc).

Problem

As discussed in the excellent kaocha source code, these two approaches don't work together, which leads to issues such as kaocha's workarounds (and monkey-patching), leiningen's monkey-patching, Cursive's test runner not always integrating with other tools due to binding report, and others.

Alessandra Sierra wrote about this issue as well when she attempted a follow-up to clojure.test called Lazytest (which I have resurrected over the last year). She ended up not finishing the library, and many other alternatives to Clojure test have not been as successful as one might hope either.

Given that clojure.test is here to stay, it seems prudent to make it easier to use and adapt without changing the fundamental api.

Potential Solution

Separating the "dynamic variable to be rebound" from "the default reporting function" would make it easier for libraries and tools to choose to extend report or replace it entirely, without worrying that they'll clash or run into each other. This can be accomplished by adding a new dynamic variable *reporter* to clojure.test that do-report calls, and defaulting *reporter* to clojure.test/report. (report would stay dynamic so as to not change working code.)

With such a change in place, workarounds such as Leiningen's monkey-patching could simply bind *report* with a new function and not worry about stepping on any other library's toes.

by
It would probably be helpful for clojure.test to be split out as a Contrib library and have a dedicated maintainer so it could evolve independently of Clojure core, but core could still have a dependency on it at each release so that clojure.test would remain a bundled (but overridable) namespace for users.
by
That's the dream, but in the meantime, I think this would be helpful directly.

1 Answer

+1 vote
by

Logged as https://clojure.atlassian.net/browse/CLJ-2882, patch welcome for evaluation

by
Added. Thanks.
...