pr-str captures stdout from printing side-effects of lazily evaluated expressions.

Question

Because clojure.core/pr-str uses with-out-str to capture the output of pr (and pr cannot be parsed a writable thing - just uses out).

If you pr-str the result of something lazy you can get side-effects written to stdout with println interspersed with the output. For example in my case I was extracting benchmarks from the library criterium and trying to print the data structure to the file. The solution would be to provide an overload of pr/pr-str that takes a writer. I note that pr-on provides some of the functionality but it is private.

This is an ugly bug when you're trying to persist program output in EDN, because the randomly interspersed stdout messages make it invalid for read-string. We shouldn't need our functions to be pure for pr-str to work as expected.

I've omitted a patch because although I think a fix is straight-forward I'm not sure quite where it should go (e.g. make pr-on public, change pr, change pr-str)

Answer 1

Comment made by: stu

as a workound for this, use print-dup or print-method

Answer 2

Comment made by: olical

So I think a work around for this would be to change (link: https://github.com/clojure/clojure/blob/841fa60b41bc74367fb16ec65d025ea5bde7a617/src/clj/clojure/core.clj#L4702-L4709 text: pr-str) to use (link: https://github.com/clojure/clojure/blob/841fa60b41bc74367fb16ec65d025ea5bde7a617/src/clj/clojure/core.clj#L3660-L3667 text: pr-on) with it's own string writer. This way things intended for out will still get there, the user can then wrap the call in with-out-str if they really want to.

I can't think of why anyone would've relied on this functionality in the past, but there is the risk that this would (link: https://xkcd.com/1172/ text: break someone's workflow). I propose (and will submit a patch for if there's no good reason not to) that we change pr-str to use pr-on directly with a string writer instead of pr which defaults to out. This does mean that pr-str will have to reimplement the var args processing but that's okay I think.

The main motivator for this is that prepl uses pr-str to encode data to send down the socket. This means my (link: https://github.com/Olical/conjure text: prepl tooling) will capture out from all lazy-seq results since pr-str is used upstream. The other fix for this would be to configure my prepl to not use pr-str but my own function, but I think that's side stepping the issue.

I think the right thing to do here is to fix pr-str, I don't think the behaviour is correct or expected and will only catch more people out in the future. Again, I'd love to have a go at fixing this but I wanted to run my thoughts past you first.

Answer 3

Comment made by: olical

I've now realised all p...-str functions have this same issue, currently having a think about how I could solve it for all of them. So any printing lazy-seq wouldn't have it's output sent to out. Maybe I could split this so that it doesn't get sent to out but something else that can be rebound separately without using with-out-str.

Answer 4

Comment made by: olical

How about doall on any lazy-seq passed to a p...-str function? They need to be fully resolved regardless, this way we can resolve them before they're wrapped in with-out-str. I think that means the inner printing functions can remain as they are and the -str variants can simply ensure the sequences are resolved before they get printed. I don't think this'd change the characteristics of the functions too much but would fix all of these issues.

At the very least, consumers can use (pr-str (doall my-seq)) to ensure that all laziness is resolved outside of the with-out-str call within pr-str. This could be used as a temporary workaround.

Thoughts?

Answer 5

Reference: https://clojure.atlassian.net/browse/CLJ-1532 (reported by alex+import)