Welcome! Please see the About page for a little more info on how this works.

0 votes
in Test by

Minimal reproduction:

`
(require 'clojure.test)

(clojure.test/deftest foo-test
(throw (ex-info "I fail" {})))

(clojure.test/deftest bar-test
(.println System/out "bar"))

(clojure.test/test-vars [#'foo-test #'bar-test])
`

Result:

ERROR in (foo-test) (core.clj:4617) Uncaught exception, not in assertion. expected: nil bar actual: clojure.lang.ExceptionInfo: I fail at clojure.core$ex_info.invokeStatic (core.clj:4617) ...

Note "bar" appearing in the output in the middle of the error report for {{foo-test}}.

Analysis:

{{(clojure.test/report {:type :error, :actual some-exception})}} calls {{stack/print-cause-trace}}. Unlike other {{clojure.test/report}} callpaths, this does not flush on newline. Thus, when tests fail with exceptions and there is anything writing directly to Java's {{System.out}}, there can be a large gap between the first part of the error report and the exception trace.

(To explain why this is annoying: we're running Selenium tests via {{clj-webdriver}}, and our system under test is logging with log4j via {{clojure.tools.logging}}. We invariably see dozens or even hundreds of lines between "expected: ..." and the subsequent "actual: ..." exception trace. This makes it very easy to come to completely the wrong conclusion about when failures occurred with respect to the other events that appear interleaved in the log.)

It would be preferable (in my opinion) if {{clojure.test/report}} always constructed the output from each individual invocation into a single string which got written to * } all at once -- that way there could be no way for output to be interleaved from other threads. Absent that, it would at least help a lot if the {{:error}} implementation called {{(flush)}}.

1 Answer

0 votes
by
Reference: https://clojure.atlassian.net/browse/CLJ-1968 (reported by alex+import)
...