core.async thread pool vs willy-nilly Java threads?

Question

core.async has a thread pool, limited by Java system property "clojure.core.async.pool-size". But the following little REPL interaction (briefly) creates thousands of Java threads.

(require '[clojure.core.async :as a])
(def p (a/promise-chan))
(def a (atom 0))
(def b (atom 0))
(def cc (into [] (repeatedly 10000 (fn []
                                     (a/go
                                       (swap! a inc)
                                       (a/<! p)
                                       (swap! b inc))))))

Immediately upon the (def cc...), I noticed, among other things, that jstack (in another terminal) showed a brief increase of thousands of Java threads.

Question: When is the fixed-size thread pool used, vs creating a Java thread for every "go"?

Answer 1

That shouldn't be the case unless you set it explicitly to some huge number.
It's 8 by default: https://github.com/clojure/core.async/blob/master/src/main/clojure/clojure/core/async/impl/exec/threadpool.clj#L20

How did you "count" your threads?

Comments

from visual vm running the above: live threads 8081 and daemon threads 8080


and dumped into the repl:

> Exception in thread "async-dispatch-133" Exception in thread "async-dispatch-132" Exception in thread "async-dispatch-131" Exception in thread "async-dispatch-130" Exception in thread "async-dispatch-128" Exception in thread "async-dispatch-127" Exception in thread "async-dispatch-126" java.lang.AssertionError: Assert failed: No more than 1024 pending takes are allowed on a single channel.
(< (.size takes) impl/MAX-QUEUE-SIZE)
    at clojure.core.async.impl.channels.ManyToManyChannel.take_BANG_(channels.clj:235)
    at clojure.core.async.impl.ioc_macros$take_BANG_.invokeStatic(ioc_macros.clj:988)
    at clojure.core.async.impl.ioc_macros$take_BANG_.invoke(ioc_macros.clj:987)
    at investigate$fn__9405$fn__9414$state_machine__6606__auto____9415$fn__9417.invoke(NO_SOURCE_FILE:1)
    at investigate$fn__9405$fn__9414$state_machine__6606__auto____9415.invoke(NO_SOURCE_FILE:1)
    at clojure.core.async.impl.ioc_macros$run_state_machine.invokeStatic(ioc_macros.clj:978)
    at clojure.core.async.impl.ioc_macros$run_state_machine.invoke(ioc_macros.clj:977)
    at clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invokeStatic(ioc_macros.clj:982)
    at clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invoke(ioc_macros.clj:980)
    at investigate$fn__9405$fn__9414.invoke(NO_SOURCE_FILE:1)
    at clojure.lang.AFn.run(AFn.java:22)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at clojure.core.async.impl.concurrent$counted_thread_factory$reify__479$fn__480.invoke(concurrent.clj:29)
    at clojure.lang.AFn.run(AFn.java:22)
    at java.base/java.lang.Thread.run(Thread.java:830)


... couple hundred liens that look like the above...

Exception in thread "async-dispatch-99" [37.890s][warning][os,thread] Failed to start thread - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
Exception in thread "async-dispatch-8177" java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached

 Its true that you are creating 8000 threads but you only have 8 alive at any time. You're piling up tons of pending takes which are unsatisfied and running into core.async's 1024 limit.  https://github.com/clojure/core.async/blob/master/src/main/clojure/clojure/core/async/impl/protocols.clj#L13

So what you're doing is creating thousands of threads and killing them immediately, with many more jumping into the breach based on the following FixedThreadPool

```
(defn thread-pool-executor
  ([]
    (thread-pool-executor nil))
  ([init-fn]
   (let [executor-svc (Executors/newFixedThreadPool
                        @pool-size
                        (conc/counted-thread-factory "async-dispatch-%d" true
                          {:init-fn init-fn}))]
     (reify impl/Executor
       (impl/exec [this r]
         (.execute executor-svc ^Runnable r))))))
```

---

@dpsutton I expected my code snippet to put 10000 things onto a queue, which is serviced by a thread pool of fixed size 8; not to create 10000 threads.  Your comment led me to the javadoc of Executors, which says, "If any thread terminates due to a failure during execution prior to shutdown, a new one will take its place if needed to execute subsequent tasks."  That would explain about 9000 Thread objects to garbage-collect from the heap, but it does not address why so many actual threads+stacks linger in the VM after having been replaced by the Executor with a new thread.  Maybe the OS thread sticks around until garbage collection finalizes the Thread object?