Hi Clojure team,
At Liftoff, we are big users of Clojure (400k LOC across 250+ projects). We have built some tooling that automatically generates deps.edn files for projects, based on static analysis of the project's code.
Some of our tools and test code repeatedly calls clojure.tools.deps.alpha/resolve-deps. As of version v0.14.1178, we have noticed the following:
- resolve-deps creates a new java.util.concurrent.ExecutorService on each call, but doesn't shut it down before returning (code). This lead our test code to spawn 20k+ threads.
- clojure.tools.deps.alpha.util.concurrent/submit-task pushes thread bindings onto a fixed size threadpool, but doesn't pop them (code). As threads are re-used, the frame stack keeps growing.
Internally, we are now using a patched version of the tools that unconditionally shuts down the ExecutorService on each call to resolve-deps and pops the thread bindings for each task. This solves our problem, but we don't know if this is right solution.
How is the core Clojure team thinking about concurrency in the context of the CLI tools? Can we expect this behavior to change in future releases of the CLI tools?