The bodies of doseq and for seem to be duplicated in their expansions.
$ clojure
user=> (defmacro a [] (prn :expand))
#'user/a
user=> (doseq [_ nil] (a))
:expand
:expand
nil
user=> (for [_ nil] (a))
:expand
:expand
()
I found the same problem in ClojureScript. I could not find an existing discussion about this so I'm not sure if this is by design, apologies if so. I'm aware that these macros have a large code footprint and that doseq cannot use closures (the usual way to prevent exponential expansion), but I missed this detail.
This leads to exponential code blowup if these forms are nested. Artificial demonstration:
$ clojure
Clojure 1.12.0
user=> (def counter (atom -1))
#'user/counter
user=> (defmacro a [] (prn :expand (swap! counter inc)))
#'user/a
user=> #(doseq [_ 1] (doseq [_ 2] (doseq [_ 3] (doseq [_ 4] (a)))))
:expand 0
:expand 1
:expand 2
:expand 3
:expand 4
:expand 5
:expand 6
:expand 7
:expand 8
:expand 9
:expand 10
:expand 11
:expand 12
:expand 13
:expand 14
:expand 15
#object[user$eval926$fn__927 0x76563d26 "user$eval926$fn__927@76563d26"]
This might be especially relevant to core.async, where placing go under doseq is idiomatic, since go is expensive in both expansion time and code size.
user=> (time (eval '(doseq [_ nil] (a/go (doseq [_ nil] (a/go (doseq [_ nil] (a/go))))))))
"Elapsed time: 837.230917 msecs"
nil
It also leads to duplicated reflection warnings from the fully expanded forms, which is how I found this problem.
user=> (doseq [_ nil] (Thread/sleep (identity 1)))
Reflection warning, NO_SOURCE_PATH:1:16 - call to static method sleep on java.lang.Thread can't be resolved (argument types: unknown).
Reflection warning, NO_SOURCE_PATH:1:16 - call to static method sleep on java.lang.Thread can't be resolved (argument types: unknown).
nil
I believe the exponential expansion for doseq was introduced in Clojure 1.1.0 with support for chunked seqs with this commit.
$ clojure -Sdeps '{:deps {org.clojure/clojure {:mvn/version "1.0.0"}}}'
Downloading: org/clojure/clojure/1.0.0/clojure-1.0.0.pom from central
Downloading: org/clojure/clojure/1.0.0/clojure-1.0.0.jar from central
Clojure 1.0.0-
user=> (defmacro a [] (prn :expand))
#'user/a
user=> (doseq [_ nil] (a))
:expand
nil
user=> ^D
$ clojure -Sdeps '{:deps {org.clojure/clojure {:mvn/version "1.1.0"}}}'
Downloading: org/clojure/clojure/1.1.0/clojure-1.1.0.pom from central
Downloading: org/clojure/clojure/1.1.0/clojure-1.1.0.jar from central
Clojure 1.1.0
user=> (defmacro a [] (prn :expand))
#'user/a
user=> (doseq [_ nil] (a))
:expand
:expand
nil
user=>
At first I wasn't sure if this was hopeless, but I think doseq is fixable fusing the chunked and non-chunked cases like so:
(defmacro doseq
"Repeatedly executes body (presumably for side-effects) with
bindings and filtering as provided by \"for\". Does not retain
the head of the sequence. Returns nil.
Unlike clojure.core/doseq, does not cause exponential macro expansion
of expressions in bindings or body."
[seq-exprs & body]
(#'clojure.core/assert-args
(vector? seq-exprs) "a vector for its binding"
(even? (count seq-exprs)) "an even number of forms in binding vector")
(let [step (fn step [recform exprs]
(if-not exprs
[true `(do ~@body)]
(let [k (first exprs)
v (second exprs)]
(if (keyword? k)
(let [steppair (step recform (nnext exprs))
needrec (steppair 0)
subform (steppair 1)]
(cond
(= k :let) [needrec `(let ~v ~subform)]
(= k :while) [false `(when ~v
~subform
~@(when needrec [recform]))]
(= k :when) [false `(if ~v
(do
~subform
~@(when needrec [recform]))
~recform)]))
(let [seq- (gensym "seq_")
chunk- (with-meta (gensym "chunk_")
{:tag 'clojure.lang.IChunk})
count- (gensym "count_")
i- (gensym "i_")
in-chunk- (gensym "in-chunk_")
recform `(if ~in-chunk-
(recur ~seq- ~chunk- ~count- (unchecked-inc ~i-))
(recur (next ~seq-) nil 0 0))
steppair (step recform (nnext exprs))
needrec (steppair 0)
subform (steppair 1)]
[true
`(loop [~seq- (seq ~v), ~chunk- nil,
~count- 0, ~i- 0]
(let [~in-chunk- (< ~i- ~count-)
~seq- (if ~in-chunk- ~seq- (seq ~seq-))]
(when (if ~in-chunk- true ~seq-)
(let [chunked?# (if ~in-chunk- false (chunked-seq? ~seq-))
~k (if ~in-chunk-
(.nth ~chunk- ~i-)
(if chunked?# nil (first ~seq-)))]
(if (if ~in-chunk- false chunked?#)
(let [c# (chunk-first ~seq-)]
(recur (chunk-rest ~seq-) c#
(int (count c#)) (int 0)))
(do ~subform
~@(when needrec [recform])))))))])))))]
(nth (step nil (seq seq-exprs)) 1)))
