Do stream interop functions close the stream?

Question

Docs for the handy new stream interop functions (stream-seq!, stream-into! etc) say they "are terminal stream operations (they consume the stream)."

Just to be totally clear, does this mean they close (.close) the streams?

I ask because I use these functions with the stream returned from java.nio.file.Files/list and when I'm benchmarking the app I sometimes get "too many open files" Java errors. I can't think where else I might conceivably be leaving a file handle open, beyond that I'm almost entirely just using slurp (for now at least).

(Naively, I would expect everything except stream-seq! to close out the stream by the time it returns, and stream-seq! I would expect to close once the seq is consumed and/or garbage collected)

Comments

Looks like no (https://github.com/clojure/clojure/blob/fb22fd778a272b034684a4ee94509552b46ee8a9/src/jvm/clojure/lang/RT.java#L536), but I could be missing something. Depending on how you got the stream, `with-open` might help you manage closing it reliably.

Answer 1

Do they? no

Should they? maybe, will discuss but for now you should close the stream yourself (with-open is a good use here).

Comments

Not all streams require releasing resources, and in fact very few do. In the cases that do it's advised to use something that will close it on your behalf (as Alex mentioned). Additionally, the stream functions don't create those streams, so it shouldn't be their responsibility to close them as they're not resource management operations.

---

Thank you for the info! I will use `with-open`.

(I do not know enough about this problem space to have an opinion about the stream interop functions. I will provide my naive thought process after reading the docstrings here in case it is helpful: "Hmmm, it says these are "terminal operation"s, so I'm not sure I'm even supposed to run any other operations on them, and technically `.close` is an operation, so maybe I am not allowed to run `.close`. Also, if these are terminal operations they will probably call `.close` themselves because why wouldn't they?")

(All that said I have now, in Clojure core fashion,  gone to the dictionary definition of "terminal" and I see it does not always mean the actual end, it can just be related to the end, so presumably near the end, so I would not say the word is incorrect. Maybe drawing from the wording on the interop page with something like"this function consumes the stream" would be somewhat clearer, I don't really know.)

(I am overall just glad these functions exist!)

---

Here's one that will be tricky to rewrite. It returns a lazy seq and so the consumer has no way (right?) of closing the stream once this lazy seq leaves the function. I suppose I need to return a map of like {:seq seq :stream stream} and pass that around so it can be closed when I'm done. In theory (?) garbage collection should take care of this but I'm not sure.

(defn visible-subdirs-recursive
  [^Path dir]
  (->> dir
       Files/list
       stream-seq!
       (sequence (comp
                  (filter (every-pred finally-dir?
                                      (complement Files/isHidden)))
                  (mapcat (fn [dir]
                            (cons dir (visible-subdirs-recursive dir))))))))