Exception using futures with -X on Clojure CLI 1.10.3.929

Question

There seems to be a weird issue related to -X and futures with Clojure CLI 1.10.3.929:

$ clojure --version
Clojure CLI version 1.10.3.929
$ clojure -Srepro -X clojure.core.server/start-server :name '"server"' :port 5555 :accept clojure.core.server/repl :server-daemon false
$ clojure -Srepro -J-Dclojure.server.repl="{:port 6666 :accept clojure.core.server/repl}"
$ nc localhost 5555
user=> (def x (future 1))
Execution error (RejectedExecutionException) at java.util.concurrent.ThreadPoolExecutor$AbortPolicy/rejectedExecution (ThreadPoolExecutor.java:2057).
Task java.util.concurrent.FutureTask@14c75e2b[Not completed, task = clojure.core$binding_conveyor_fn$fn__5772@4f4cba65] rejected from java.util.concurrent.ThreadPoolExecutor@5d5b17a8[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 1]
$ nc localhost 6666
user=> (def x (future 2))
#'user/x

I cannot reproduce the issue with 1.10.3.875.

The -X invocation doesn't work at all on 1.10.3.882, so I can't (easily) narrow it down any further.

Answer 1

I believe with 882, -X called System/exit which is why that version doesn't work.

905 behaves the same way (exits).

912 onward exhibit the "bad" behavior and I'm fairly sure this is because they call shutdown-agents -- which was added to be friendly to exec functions (programs) that would otherwise "hang" at the end.

The root of this change in behavior was probably due to wanting to be able to chain functions -- so the exec functions themselves cannot call shutdown-agents or System/exit because that would "break the chain" and so only the CLI could call those functions once all the exec functions have "completed".

There's an inherent tension between wanting a clean shell exit after calling -X and wanting background threads to stay alive (like the Socket Server example).

Comments

Sean's analysis is mostly correct. Doesn't really have anything to do with chaining, which is not yet a feature, but just in wanting to match expectations on shutdown when executing a -X function. There are multiple cases here:

* Use future/agent, have no background threads, expect full exit. Will wait for 1 minute for background agent pool to shutdown, so MUST have shutdown-agents or System/exit.
* Have background threads (like socket server). Expect to block and NOT exit. CANNOT use System/exit (or you kill server).
* Use future/agent, and have background threads (this case). Expect to block and NOT exit. CANNOT use System/exit (would kill server) or shutdown-agents (would kill futures/agents run from server).

I don't think there is any behavior that works for all cases by default (you can always wrap a function and do whatever you know is right).

Logged as https://clojure.atlassian.net/browse/TDEPS-198

---

We've been partially solving this at work by wrapping stuff with yet another exec function but that is kind of tedious and doesn't generalize, so I'm pleased to see this addressed in 1.10.3.933 -- thank you!