Welcome! Please see the About page for a little more info on how this works.

+1 vote
in Collections by

References to elements {{pop}}'ed from PersistentQueue are not immediately removed from the queue. This may lead to situations where the queue holds onto many more objects than it currently contains.

This happens because the PersistentQueue is implemented using two PersistentVectors, one for head, and one for tail. Once the head is empty, the tail vector is transformed into ChunkedSeq (by calling seq on it) and put into the head. However, ChunkedSeq always retains the reference to the whole starting vector, even as it is shrinked with {{next}}.

Example, with clj-memory-meter:

(require '[clj-memory-meter.core :as mm])

(def q clojure.lang.PersistentQueue/EMPTY)
(def strs (repeatedly 100000 #(String. (.toCharArray "Hello"))))

;; Queue with 100k strings "Hello"
(mm/measure (into q strs))
=> "5.9 MB"

;; Shrink such queue to just 1 element
(nth (iterate pop (into q strs)) 99999)
=> ;; a queue of one element

;; But it still occupies 6 mb
(mm/measure (nth (iterate pop (into q strs)) 99999))
=> "5.9 MB"

Perhaps, if this behavior is expected, it can be at least reflected in the docs somewhere? Given that PQ is already hidden and underdocumented.

2 Answers

0 votes

Comment made by: alexyakushev

EDIT: Changed priority to Minor as this is not a classical memory leak (queue will not grow indefinitely, the biggest leak has the size of the longest streak of {{conj}} before {{pop}}).

0 votes
Reference: https://clojure.atlassian.net/browse/CLJ-2397 (reported by alexyakushev)