behavior of deftests after adding assert-expr methods

Question

The following behavior surprised me
1. define a test that calls (is (= on one item
2. override the multimethod clojure.test/assert-expr to define custom testing for = such that it should error if only one item is present
3. run your test and the multimethod is not invoked since it does not error
4. redefine the test
5. the new multimethod is now invoked.

I'm sure there's a smaller repro but this is the real world use case that came up. CIDER changes this reporting in its middleware but the middleware is dynamically loaded after startup only when needed. Thus this behavior only occurs after using CIDER's testing features and if you redefine your tests.

I'm wondering if this is behaving as expected since deftest stuffs the test in meta or if this is an actual bug.

(require '[clojure.test :refer [deftest is assert-expr]])
(deftest foo (is (= (println 1))))

(foo) ;; no error

(defn =-body
  [msg expected more]
  (if (seq more)
    `(let [more# (list ~@more)
           expected# ~expected
           result# (apply = expected# more#)]
       (->> (if result#
              {:type :pass}
              {:type :fail
               :diffs (->> (remove #(= expected# %) more#)
                           (map #(vector % (data/diff expected# %))))})
            (merge {:message ~msg
                    :expected expected#
                    :actual more#})
            test/do-report)
       result#)
    `(throw (Exception. "= expects more than one argument"))))

(defmethod assert-expr '= [msg [_ expected & more]]
  (=-body msg expected more))

(foo) ;; no error
(deftest foo (is (= (println 1))))
(foo) ;; error

Answer 1

Since is is a macro that uses the assert-expr multi-method to expand the code at compile time, changing assert-expr after running deftest will not change the behavior of that test function since it has already been expanded and compiled.

That is why you don't see the new assert-expr functionality until you define new tests (or redefine existing ones).