lazy seq that encounters an exception has differing behavior on repeated use

Question

It seems the below does not compile with 1.8.0 and 1.9.0-alpha14, the same errors appear in both versions.

`
user=> (def fibonacci-1
((fn fib [a b]

(lazy-seq  (cons a  (fib b  (+ a b)))))
0 1))

user=> (filter #(< % 100) fibonacci-1)

ArithmeticException integer overflow clojure.lang.Numbers.throwIntOverflow (Numbers.java:1501)

user=> (filter #(< % 100) fibonacci-1)

NullPointerException clojure.lang.Numbers.ops (Numbers.java:1013)

user=> (def fibonacci-2

     (lazy-cat [0 1] (map + (rest fibonacci-2) fibonacci-2)))

user=> (filter #(< % 100) fibonacci-2)

ArithmeticException integer overflow clojure.lang.Numbers.throwIntOverflow (Numbers.java:1501)

user=> (filter #(< % 100) fibonacci-2)
(0 1 1 2 3 5 8 13 21 34 55 89)
`

Patch: 0001-CLJ-2069-cache-exceptions-thrown-during-lazy-seq-rea.patch

Proposal: Cache exceptions thrown during lazy-seq realization, to avoid re-running bodyfn which is declared as ^:once

Prescreened by: Alex Miller

Answer 1

_Comment made by: find_myway

Maybe I should use take-while instead of filter.

However, can anyone explain why I get ArithmeticException while running

(filter #(< % 100) fibonacci-2)

for the first time and get the right result at the second time?

Answer 2

Comment made by: bronsa

The NPE is caused by the interaction between:
- lazy-seq throwing an exception while realizing part of the sequence
- lazy-seq internally using ^:once for locals clearing

lazy-seq expects the bodyfn to be run exactly once and then the result to be cached, but if an exception gets thrown during the execution of bodyfn, the function will get run again when the sequence tries to be realized a second time. However if locals clearing has already happened (even partially) this means some locals in bodyfn will now be nil rather than holding their actual value.

Answer 3

Comment made by: bronsa

Attached patch that fixes this issue

Answer 4

Reference: https://clojure.atlassian.net/browse/CLJ-2069 (reported by alex+import)