Use transducers on clojure.core lazy sequence transformations

Tue, 11 Apr 2023 06:55:00 +0000

Hi,

I heard that if transducers were included before in the language, they would have been used as the building blocks of all sequence lazy operations. Since they were added later, you need to adapt your code a bit to use transducers. And I was wondering why using eduction is not an option? I'm trying to understand the scenarios in which it's not a performance advantage to define the sequence lazy operations like this (of course assuming the same behaviour):

(defn map
  ([f] ;; the standard transducer definition
   ,,,)
  ([f coll]
   (eduction (map f) coll))

I'm trying to understand given the assumption (which might be an erroneous assumption) that

(->> (range 5000000)
     (eduction (map inc))
     (eduction (filter odd?))
     (eduction (map dec))
     (eduction (filter even?))
     (eduction (map (fn [n] (+ 3 n))))
     (eduction (filter odd?))
     (eduction (map inc))
     (eduction (filter odd?))
     (eduction (map dec))
     (eduction (filter even?))
     (eduction (map (fn [n] (+ 3 n))))
     (eduction (filter odd?))
     (into []))

is the same as

(->> (range 5000000)
     (map inc)
     (filter odd?)
     (map dec)
     (filter even?)
     (map (fn [n] (+ 3 n)))
     (filter odd?)
     (map inc)
     (filter odd?)
     (map dec)
     (filter even?)
     (map (fn [n] (+ 3 n)))
     (filter odd?)
     (into []))

Is lazy sequence chunking deterministic?

Mon, 28 Mar 2022 16:11:27 +0000

I know that lazy sequences can be processed in a chunked manner. That is, it may be that if my program needs n elements from a lazy sequence, more than n might be realized.

Can I assume that if I run the same code a second time in the same version of Clojure, that exactly the same elements will get realized? I would think so, but I don't want to assume that without knowing.

(I'm including additional info here in case anyone is interested in why this matters, but I don't think it's necessary to know my use case in order to answer the question. So feel free to skip this paragraph. I'm running simulations that make use of a pseudorandom number generator. These involve random walks that are truncated if the walk encounters a "target". It's convenient to generate a walk in advance as a lazy sequence, and then stop realizing its elements when a target is found. Normally, I throw away intermediate results such as the original random walk. They exist in a data structure at one time, but then I don't use them in the end. However, I want to be able to go back and rerun a simulation, sometimes, in order to examine the intermediate results. It's easy enough to store the seed that I used for a simulation run, and use it to set the random number generator to start in the same state. However, chunking can cause the random number generator to be called more times than are strictly necessary. That's OK, but if chunking can vary depending on something other than my code and the Clojure version, that could mean that rerunning a simulation with the same seed would not be guaranteed to produce the same results. This is not at present a big deal. I can make the intermediate data non-lazy without much of a performance hit. However, I'm still wondering whether that's necessary.)

Clojure Q&A - Recent questions tagged lazy chunking

Use transducers on clojure.core lazy sequence transformations

Is lazy sequence chunking deterministic?