Welcome! Please see the About page for a little more info on how this works.

+2 votes
in Test by

If an exception occurs during a clojure.test deftest, the error goes to clojure.test/report with :error key which will (by default) print the entire stack trace. The stack includes the test framework itself and (if invoked via an external test runner like lein test, clojure.main etc), yielding a stack depth of ~60 stack frames which is all incidental noise that is irrelevant.

There is a dyn var clojure.test/stack-trace-depth that defaults to nil (all). This dynvar can be bound around calls to the clojure.test runner to see less stack trace when an exception occurs during a test, but this is both difficult to set for external test runners and error prone.

It would be most useful to filter the framework noise from the bottom of the stack and present only the relevant frames.

Given a test like:

(ns foo.core-test
  (:require [clojure.test :refer :all]))

(deftest a-test
  (testing "FIXME, I fail."
    (throw (ex-info "I suck" {:a 1}))
    (is (= 0 1))))

the reported error (here from lein test, but any external runner is similar), only the top 2 lines are relevant, the rest is incidental noise:

ERROR in (a-test) (core_test.clj:13)
Uncaught exception, not in assertion.
expected: nil
  actual: clojure.lang.ExceptionInfo: I suck
{:a 1}
 at foo.core_test$fn__364.invokeStatic (core_test.clj:13)
    foo.core_test/fn (core_test.clj:11)
    clojure.test$test_var$fn__9737.invoke (test.clj:717)        ;; noise from here down
    clojure.test$test_var.invokeStatic (test.clj:717)
    clojure.test$test_var.invoke (test.clj:708)
    clojure.test$test_vars$fn__9763$fn__9768.invoke (test.clj:735)
    clojure.test$default_fixture.invokeStatic (test.clj:687)
    clojure.test$default_fixture.invoke (test.clj:683)
    clojure.test$test_vars$fn__9763.invoke (test.clj:735)
    clojure.test$default_fixture.invokeStatic (test.clj:687)
    clojure.test$default_fixture.invoke (test.clj:683)
    clojure.test$test_vars.invokeStatic (test.clj:731)
    clojure.test$test_all_vars.invokeStatic (test.clj:737)
    clojure.test$test_ns.invokeStatic (test.clj:758)
    clojure.test$test_ns.invoke (test.clj:743)
    user$eval224$fn__287.invoke (form-init4292042596091073068.clj:1)
    clojure.lang.AFn.applyToHelper (AFn.java:156)
    clojure.lang.AFn.applyTo (AFn.java:144)
    clojure.core$apply.invokeStatic (core.clj:667)
    clojure.core$apply.invoke (core.clj:660)
    leiningen.core.injected$compose_hooks$fn__154.doInvoke (form-init4292042596091073068.clj:1)
    clojure.lang.RestFn.applyTo (RestFn.java:137)
    clojure.core$apply.invokeStatic (core.clj:665)
    clojure.core$apply.invoke (core.clj:660)
    leiningen.core.injected$run_hooks.invokeStatic (form-init4292042596091073068.clj:1)
    leiningen.core.injected$run_hooks.invoke (form-init4292042596091073068.clj:1)
    leiningen.core.injected$prepare_for_hooks$fn__159$fn__160.doInvoke (form-init4292042596091073068.clj:1)
    clojure.lang.RestFn.applyTo (RestFn.java:137)
    clojure.lang.AFunction$1.doInvoke (AFunction.java:31)
    clojure.lang.RestFn.invoke (RestFn.java:408)
    clojure.core$map$fn__5866.invoke (core.clj:2755)
    clojure.lang.LazySeq.sval (LazySeq.java:42)
    clojure.lang.LazySeq.seq (LazySeq.java:51)
    clojure.lang.Cons.next (Cons.java:39)
    clojure.lang.RT.boundedLength (RT.java:1792)
    clojure.lang.RestFn.applyTo (RestFn.java:130)
    clojure.core$apply.invokeStatic (core.clj:667)
    clojure.test$run_tests.invokeStatic (test.clj:768)
    clojure.test$run_tests.doInvoke (test.clj:768)
    clojure.lang.RestFn.applyTo (RestFn.java:137)
    clojure.core$apply.invokeStatic (core.clj:665)
    clojure.core$apply.invoke (core.clj:660)
    user$eval224$fn__299$fn__332.invoke (form-init4292042596091073068.clj:1)
    user$eval224$fn__299$fn__300.invoke (form-init4292042596091073068.clj:1)
    user$eval224$fn__299.invoke (form-init4292042596091073068.clj:1)
    user$eval224.invokeStatic (form-init4292042596091073068.clj:1)
    user$eval224.invoke (form-init4292042596091073068.clj:1)
    clojure.lang.Compiler.eval (Compiler.java:7177)
    clojure.lang.Compiler.eval (Compiler.java:7167)
    clojure.lang.Compiler.load (Compiler.java:7636)
    clojure.lang.Compiler.loadFile (Compiler.java:7574)
    clojure.main$load_script.invokeStatic (main.clj:475)
    clojure.main$init_opt.invokeStatic (main.clj:477)
    clojure.main$init_opt.invoke (main.clj:477)
    clojure.main$initialize.invokeStatic (main.clj:508)
    clojure.main$null_opt.invokeStatic (main.clj:542)
    clojure.main$null_opt.invoke (main.clj:539)
    clojure.main$main.invokeStatic (main.clj:664)
    clojure.main$main.doInvoke (main.clj:616)
    clojure.lang.RestFn.applyTo (RestFn.java:137)
    clojure.lang.Var.applyTo (Var.java:705)
    clojure.main.main (main.java:40)

We are doing things elsewhere to filter frames and I don't think it would be that hard to filter this to improve the default experience (could still be a dyn var to not filter if needed).

2 Answers

0 votes
by
0 votes
by

I think that we need to keep the fixtures frames in some way.

ERROR in (a-test) (core_test.clj:13)
Uncaught exception, not in assertion.
expected: nil
  actual: clojure.lang.ExceptionInfo: I suck
{:a 1}
 at foo.core_test$fn__364.invokeStatic (core_test.clj:13)
    foo.core_test/fn (core_test.clj:11)
    clojure.test$test_var$fn__9737.invoke (test.clj:717)        ;; noise from here down
    clojure.test$test_var.invokeStatic (test.clj:717)
    clojure.test$test_var.invoke (test.clj:708)
    clojure.test$test_vars$fn__9763$fn__9768.invoke (test.clj:735)
    foo.core_test$eval139891$hello_fixture__139892.invoke (core_test.clj:237) ;; this is relevant too!!
    clojure.test$compose_fixtures$fn__9755$fn__9756.invoke (test.clj:694)
    clojure.test$default_fixture.invokeStatic (test.clj:687)
    clojure.test$default_fixture.invoke (test.clj:683)
    
by
It's a good question and needs more analysis. Maybe it would be more helpful to instead say something like:

Active fixtures: hello-fixture

That is, is the line number of the fixture frame relevant?

Probably want something different if the exception comes from the fixture itself as well.
by
Yes, know/list the fixtures seems to be OK.


But we can leave an property to disable it
`-Dclojure.test.cleanup-exception=false`
...