Clojure Q&A - Recent questions and answers in Test

Answered: Separate bindable reporting function from clojure.test/report

Tue, 27 Aug 2024 19:23:45 +0000

Logged as https://clojure.atlassian.net/browse/CLJ-2882, patch welcome for evaluation

Answered: clojure.test/report docstring is stale

Tue, 30 May 2023 14:44:10 +0000

Logged as https://clojure.atlassian.net/browse/CLJ-2781

Answered: Way to set clojure.test/load-tests when starting a REPL

Thu, 02 Feb 2023 16:08:32 +0000

How are you starting a REPL?

Answered: Add option to enable automatically running tests when form is evaluated

Thu, 26 Jan 2023 15:03:44 +0000

Can you start with what problem you're trying to solve? Not sure I get it.

Answered: Add clojure.test api for fixtures on individual deftests

Thu, 18 Aug 2022 21:30:15 +0000

Logged jira at https://clojure.atlassian.net/browse/CLJ-2719

Answered: clojure/data.xml tests fail on JDK 17

Mon, 07 Feb 2022 00:46:28 +0000

This project doesn't actually use Leiningen to run the project tests - the Maven build (mvn clean test) is what's run in the official build.

That said when run that way they have the same problem (well multiple) problems. The logic to only test on Java 8 is wrong for JDK > 8 and will only try to run the tests on Java 8. It depends on a really old ClojureScript version and newer versions have removed a lot of the Nashorn repl stuff the test suite depends on, so using newest ClojureScript also won't work.

What I ideally would want here is not what we have in core.async (which splits the build into a piece that can't be run automatically on the build box during releases) but to fix what we have in clojure.xml to continue working in the maven-based build. There are OpenJDK builds of Nashorn that can be used, so Nashorn is not really a deal breaker either way. I do not follow the CLJS testing stuff enough to know if it's worth doing that and addressing the parts pulled out of cljs repl, or it would be better to use something else and retrofitting the tests into a different framework.

If you're interested in working on it, would be happy to have help on it. We don't use PRs but I've created a jira for this at https://clojure.atlassian.net/browse/DXML-67 and you could sign the contributor agreement and request access to jira to provide patches. https://clojure.org/dev/dev#_becoming_a_contributor

Answered: iteration-seq-equals-reduce build failure with 1.11.0-alpha4 source

Sun, 16 Jan 2022 00:24:26 +0000

Yeah, we are aware of it, will fix in master soon.

Answered: clojure test reporter uses a bound out, not `out`

Thu, 26 Aug 2021 00:18:18 +0000

I believe this is "as designed" and is relied on by some of the utilities that extend clojure.test's behavior. The clojure.test ns docstring has this to say:

SAVING TEST OUTPUT TO A FILE

   All the test reporting functions write to the var *test-out*.  By
   default, this is the same as *out*, but you can rebind it to any
   PrintWriter.  For example, it could be a file opened with
   clojure.java.io/writer.

There's a with-test-out macro that is specifically intended for use by test reporters so they can rely on this behavior:

(defmacro with-test-out
  "Runs body with *out* bound to the value of *test-out*."
  {:added "1.1"}
  [& body]
  `(binding [*out* *test-out*]
     ~@body))

Answered: clojure.test deftest exception reporting lists many useless stack frames

Wed, 30 Jun 2021 13:38:21 +0000

I think that we need to keep the fixtures frames in some way.

ERROR in (a-test) (core_test.clj:13)
Uncaught exception, not in assertion.
expected: nil
  actual: clojure.lang.ExceptionInfo: I suck
{:a 1}
 at foo.core_test$fn__364.invokeStatic (core_test.clj:13)
    foo.core_test/fn (core_test.clj:11)
    clojure.test$test_var$fn__9737.invoke (test.clj:717)        ;; noise from here down
    clojure.test$test_var.invokeStatic (test.clj:717)
    clojure.test$test_var.invoke (test.clj:708)
    clojure.test$test_vars$fn__9763$fn__9768.invoke (test.clj:735)
    foo.core_test$eval139891$hello_fixture__139892.invoke (core_test.clj:237) ;; this is relevant too!!
    clojure.test$compose_fixtures$fn__9755$fn__9756.invoke (test.clj:694)
    clojure.test$default_fixture.invokeStatic (test.clj:687)
    clojure.test$default_fixture.invoke (test.clj:683)

Answered: would clojure.test benefit from `(defmethod assert-expr thrown-with-data? ...)`

Mon, 12 Apr 2021 18:51:27 +0000

Definitely interested if there’s something that makes sense. Would be interested if people have created things like this already if there’s any experience.

Answered: How to I assert that the function I'm testing calls another function 3 times?

Sat, 17 Oct 2020 17:53:29 +0000

You could make retry-count an atom and then use swap! to increment it.

(deftest retries
  (let [retry-count (atom 0)]
    (with-redefs [clj-http.client/get (fn [url & [req & r]]
                                        (swap! retry-count inc)
                                        (throw (UnknownHostException. "Some message")))]
      (is (thrown? ServiceUnavailable (foo/my-function-under-test)))
      (is (= 3 @retry-count)))))

Answered: Running tests in one file

Fri, 16 Oct 2020 04:38:37 +0000

could you provide a sample of what a data file, source file, and test file look like?

Answered: Can't get test to true when the result is a function (Syntax error compiling at … No such var:…)

Fri, 25 Sep 2020 14:27:48 +0000

You can' t compare functions for equality so I think you should investigate alternatives to that strategy in your test. You can test functions for identity, but that is generally a very fragile test.

Answered: is vs is-not in closure.test

Wed, 01 Apr 2020 17:39:42 +0000

Not built in, but the is predicates are an open extensible system (docs at https://clojure.github.io/clojure/#clojure.test):

(defmethod clojure.test/assert-expr 'not [form]
  ...)

Basically you need to determine how to do-report in true / false cases to yield what you like. This probably would take a bit of futzing around, but you'd want to look at something like https://clojure.github.io/clojure/clojure.test-api.html#clojure.test/assert-predicate, which is where that current message is created.

Answered: How to use clojure.spec.test.alpha/check inside the clojure testing framework?

Tue, 05 Nov 2019 17:08:43 +0000

You can also use expound for reporting:

(defn is*
  [results]
  (assert (seq results))
  (test/is
   (if (some :failure results)
     (exp/explain-results results)
     true)))

Usage:

(is* (stest/check `ns/fn))

Note: assert will fail if there is no spec for that function.

Answered: Test output is misleading for `(is false)`

Sat, 12 Oct 2019 22:27:39 +0000

If you use a var or expression that produces false, it becomes a bit more obvious what is going on here:

user=> (def v false)
#'user/v
user=> (t/deftest foo (t/is v))
#'user/foo
user=> (foo)

FAIL in (foo) (NO_SOURCE_FILE:4)
expected: v
  actual: false
nil
user=>

The expected: line is essentially an abbreviation for "I expected this expression to yield truthy" so in your example it is saying " I expected false to yield truthy" (but the actual value was false).

clojure.test could certainly produce better output -- much of the original work done on Jay Fields' Expectations library and Paul Stadig's clojure.test extension Humane Test Output are attempts to produce better error messages when assertions fail.

When I extracted my clojure.test-compatibility work from Expectations and turned it into a standalone library https://github.com/clojure-expectations/clojure-test one of the things I lost was the really nice failure messages that Expectations produced because I switched from Expectations' reporting engine to clojure.test's reporting engine. I've talked to Alex Miller about the possibility of clojure.test being split out of Clojure "core" at some future point so it could be worked on separately and I've expressed an interest in taking over maintenance of it... I don't know whether that will happen or what the timeline would be.

Answered: Testing the -main function who reads from in

Wed, 18 Sep 2019 00:08:46 +0000

Use with-in-str https://clojuredocs.org/clojure.core/with-in-str.

(with-in-str "inputstring"
  (-main))

Answered: ! break clojure.test/is