Clojure Q&A - Recent questions in core.rrb-vector

rrb-vector Unhandled java.lang.ArrayIndexOutOfBoundsException

Tue, 17 Nov 2020 02:02:59 +0000

I faced this error during I'm playing with aoc2018 d9. It seems this error pops up when the vector has items around over 2M.

Execution error (ArrayIndexOutOfBoundsException) at clojure.core.rrb-vector.rrbt/slice-left$fn (rrbt.clj:393).
Index 33 out of bounds for length 33

      rrbt.clj:  393  clojure.core.rrb-vector.rrbt/slice-left/fn
      rrbt.clj:  392  clojure.core.rrb-vector.rrbt/slice-left
      rrbt.clj:  375  clojure.core.rrb-vector.rrbt/slice-left
      rrbt.clj: 1098  clojure.core.rrb-vector.rrbt.Vector/slicev
rrb_vector.clj:   83  clojure.core.rrb-vector/subvec
rrb_vector.clj:   73  clojure.core.rrb-vector/subvec

I can't find the issue page in github, so I'd like to leave a question here to get a help.

reduce-kv throws IndexOutOfBoundsException on empty rrb-vector

Fri, 18 Sep 2020 13:14:40 +0000

expected:

(reduce-kv vector nil [])
=> nil
(reduce-kv vector nil [1])
=> [nil 0 1]

actual in CLJS:

(reduce-kv vector nil (clojure.core.rrb-vector/vec []))
Error: No item 0 in vector of length 0
    at Object.cljs$core$vector_index_out_of_bounds [as vector_index_out_of_bounds] (http://localhost:8080/cljs-runtime/cljs.core.js:19788:8)
    at Object.clojure$core$rrb_vector$rrbt$IVecImpl$_array_for$arity$2 (http://localhost:8080/cljs-runtime/clojure.core.rrb_vector.rrbt.js:1028:18)
    at Object.cljs$core$IKVReduce$_kv_reduce$arity$3 (http://localhost:8080/cljs-runtime/clojure.core.rrb_vector.rrbt.js:613:21)
    at Object.cljs$core$_kv_reduce [as _kv_reduce] (http://localhost:8080/cljs-runtime/cljs.core.js:2359:13)
(reduce-kv vector nil (clojure.core.rrb-vector/vec [1]))
=> [nil 0 1]

actual in CLJ:

(reduce-kv vector nil (clojure.core.rrb-vector/vec [])) 
Execution error (IndexOutOfBoundsException) at clojure.core.rrb_vector.rrbt.Vector/arrayFor (rrbt.clj:806).
(reduce-kv vector nil (clojure.core.rrb-vector/vec [1])) 
=> [nil 0 1]

*"vector nil" part is not important here, it is just to justify calling reduce-kv.

Friday night, forgot to add any versions info:

 :deps  {org.clojure/clojure              {:mvn/version "1.10.1"}
         org.clojure/clojurescript        {:mvn/version "1.10.597"}
         org.clojure/core.rrb-vector      {:mvn/version "0.1.1"}

Probably a core.rrb-vector bug, exercised via fipp library

Fri, 02 Nov 2018 17:11:14 +0000

See the description of the test case involved on this Github issue: https://github.com/brandonbloom/fipp/issues/53

core.rrb-vector fails on JDK 11 EA builds

Thu, 05 Jul 2018 17:39:53 +0000

I tried to test boot on JDK 11 EA builds and there seems to be fipp somewhere in the dependency chain that has core.rrb-vector 0.0.11 as dependency. It fails with the below error :

 java.lang.IllegalArgumentException: Must hint overloaded method: toArray

clojure.lang.Compiler$CompilerException: java.lang.IllegalArgumentException: Must hint overloaded method: toArray, compiling:(clojure/core/rrb_vector/rrbt.clj:282:1)

         clojure.lang.ExceptionInfo: java.lang.IllegalArgumentException: Must hint overloaded method: toArray, compiling:(clojure/core/rrb_vector/rrbt.clj:282:1)

Reference Travis build : https://travis-ci.org/tirkarthi/boot/jobs/398611988#L803

This was passing a few days back but fails as of now. Feel free to close this if it's fixed in master and it's not released yet.

Successful build : https://travis-ci.org/tirkarthi/boot/jobs/398610834

StackOverflowError when building rrb-vector by concatenation at the beginning

Sun, 19 Feb 2017 23:18:11 +0000

I am trying to build rrb-vectors by concatenating singleton vectors at the beginning. I am doing to deliberatelly create relaxed structures, as part of a research project I am conducting on RRB-Trees. However it seems that this causes a stack overflow (I guess, due to some bug in the RRB-Vector structure?):

(def benchmark-size 100000)

(defn vector-push-f [v]
  (loop [v v
         i 0]
    (if (< i benchmark-size)
      (recur (fv/catvec (fv/vector i) v)
             (inc i))
      v)))

(def benchmark-rrb-vector-f (vector-push-f (fv/vector)))

Support bootstrapped ClojureScript

Mon, 23 Jan 2017 16:38:29 +0000

Revise code to support bootstrapped ClojureScript.

(This can be done with a few minor changes to the {{clojure.core.rrb-vector.macros}} namespace.)

nth fails with ArrayIndexOutOfBoundsException 33 clojure.core.rrb-vector.rrbt.Vector/fn--19277 (rrbt.clj:373)

Tue, 20 Dec 2016 09:00:35 +0000

Was playing with advent of code and puzzle 19 second part.

I have a program that mostly does catvec + subvec at https://github.com/mattiasw2/adventofcode1/blob/master/src/adventofcode1/nineteen_b.clj

The smallest sample that fails with

ArrayIndexOutOfBoundsException 33 clojure.core.rrb-vector.rrbt.Vector/fn--19277 (rrbt.clj:373)

(puzzle-b 978)

(I was using (stest/check `puzzle-b) )

I reimplemented the program using plain vectors, and everything works fine (except that I haven't been able to run the big sample)

https://github.com/mattiasw2/adventofcode1/blob/master/src/adventofcode1/nineteen_c.clj

which works fine.

Throws ClassCastException on assoc

Sat, 01 Oct 2016 22:22:14 +0000

(let [coll (into (fv/vector-of :byte) (repeat (* 32 n) 0)]
(assoc coll i 1))

throws ClassCastException [B cannot be cast to [Ljava.lang.Object; clojure.core.rrb-vector.nodes/reify--1261 (nodes.clj:69)

for every i < n * 32 - 32.

So you can assoc to elements in the last internal primitive array, and even extend it to create the next internal primitive array.
But you can't assoc to elements in any previous internal primitive array.

I wonder why nobody discovered this bug before...

ClassCastException: clojure.lang.PersistentVector$Node cannot be cast to [I

Sun, 30 Aug 2015 16:34:50 +0000

I'm seeing some strange behaviour which I've been unable to find a small failing case for.

(defn quicksort [v]
  (if (<= (count v) 1)
    v
    (let [[x & xs] v]
      (rrb/catvec (quicksort (filterv #(<= % x) xs))
                  [x]
                  (quicksort (filterv #(> % x) xs))))))

We can check this implementation with test.check

(defn ascending? [coll]
  (every? (fn [[a b]] (<= a b))
          (partition 2 1 coll)))

(def property
  (prop/for-all [v (gen/vector gen/int)]
    (let [s (quicksort v)]
      (and (= (count v) (count s))
           (ascending? s)))))

(tc/quick-check 10000 property)

;; => {:result true, :num-tests 10000, :seed 1440948212354}

(I've bumped the number of checks to 1e6 but test.check was not able to find a failing case)

Still, for some reason I get a "ClassCastException: clojure.lang.PersistentVector$Node cannot be cast to [I" for the attached 1193 element vector.

{}
(quicksort (read-string (slurp "failing-1193.edn")))
{}

Issue with Clojure 1.8.0-alpha2 Tuples

Wed, 29 Jul 2015 11:06:15 +0000

When trying our core.rrb-vector with the latest Clojure 1.8.0-alpha2 I see the following error:

Exception in thread "main" java.lang.IllegalArgumentException: No implementation of method: :slicev of protocol: #'clojure.core.rrb-vector.protocols/PSliceableVector found for class: clojure.lang.Tuple$T0, compiling:(mikera/timeline/examples.clj:12:1)

at clojure.lang.Compiler.load(Compiler.java:7307)
at clojure.lang.RT.loadResourceScript(RT.java:372)
at clojure.lang.RT.loadResourceScript(RT.java:363)
at clojure.lang.RT.load(RT.java:453)
at clojure.lang.RT.load(RT.java:419)
at clojure.core$load$fn__5448.invoke(core.clj:5866)
at clojure.core$load.doInvoke(core.clj:5865)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core$load_one.invoke(core.clj:5671)
at clojure.core$load_lib$fn__5397.invoke(core.clj:5711)
at clojure.core$load_lib.doInvoke(core.clj:5710)
at clojure.lang.RestFn.applyTo(RestFn.java:142)
at clojure.core$apply.invoke(core.clj:632)
at clojure.core$load_libs.doInvoke(core.clj:5749)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invoke(core.clj:632)
at clojure.core$require.doInvoke(core.clj:5832)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at com.theoryinpractise.clojure.testrunner$eval41$iter__42__46$fn__47$fn__48.invoke(run-test181131806783576232.clj:22)
at com.theoryinpractise.clojure.testrunner$eval41$iter__42__46$fn__47.invoke(run-test181131806783576232.clj:21)
at clojure.lang.LazySeq.sval(LazySeq.java:40)
at clojure.lang.LazySeq.seq(LazySeq.java:49)
at clojure.lang.RT.seq(RT.java:514)
at clojure.core$seq__4128.invoke(core.clj:137)
at clojure.core$dorun.invoke(core.clj:3009)
at com.theoryinpractise.clojure.testrunner$eval41.invoke(run-test181131806783576232.clj:21)
at clojure.lang.Compiler.eval(Compiler.java:6850)
at clojure.lang.Compiler.load(Compiler.java:7295)
at clojure.lang.Compiler.loadFile(Compiler.java:7233)
at clojure.main$load_script.invoke(main.clj:275)
at clojure.main$script_opt.invoke(main.clj:337)
at clojure.main$main.doInvoke(main.clj:421)
at clojure.lang.RestFn.invoke(RestFn.java:421)
at clojure.lang.Var.invoke(Var.java:383)
at clojure.lang.AFn.applyToHelper(AFn.java:156)
at clojure.lang.Var.applyTo(Var.java:700)
at clojure.main.main(main.java:37)

Caused by: java.lang.IllegalArgumentException: No implementation of method: :slicev of protocol: #'clojure.core.rrb-vector.protocols/PSliceableVector found for class: clojure.lang.Tuple$T0

at clojure.core$_cache_protocol_fn.invoke(core_deftype.clj:556)
at clojure.core.rrb_vector.protocols$eval965$fn__966$G__956__975.invoke(protocols.clj:6)
at clojure.core.rrb_vector$subvec.invoke(rrb_vector.clj:71)
at mikera.timeline.impl.Timeline.add_event(impl.clj:82)
at mikera.timeline$log.invoke(timeline.clj:64)
at mikera.timeline$log.invoke(timeline.clj:61)
at clojure.lang.Atom.swap(Atom.java:51)
at clojure.core$swap_BANG_.invoke(core.clj:2239)
at mikera.timeline.examples$eval1620.invoke(examples.clj:15)
at clojure.lang.Compiler.eval(Compiler.java:6850)
at clojure.lang.Compiler.load(Compiler.java:7295)
... 36 more

I believe the cause of this is the new Tuple types. core-rrb-vector should ideally be able to handle this like any other IPersistentVector instance, but it does not currently seem to be able to do so.

Calling empty in CLJS results in a PersistentVector

Wed, 01 Jul 2015 10:18:53 +0000

I discovered that (pop some-rrb-vec) where some-rrb-vec is of length 1 returns a PersistentVector rather than a clojure.core.rrbt.Vector in ClojureScript.

This isn't true for the Clojure side, so I assume it's a problem. Patch attached with tests.

Repeated subvec and catvec fails

Tue, 17 Mar 2015 12:32:17 +0000

The attached patch contains a test which fails for both the Clojure and ClojureScript implementations. It simply repeatedly calls subvec and catvec to assemble an interleaved vector. The patch attached contains fixes for Clojure only. I will investigate the fixes for CLJS, but thought it best to raise an issue with my findings so far.

2371 is the minimum length that exhibits the behaviour (triggers an ArrayIndexOutOfBoundsException).

Efficient reverse resulting in reversed rrb-vector

Fri, 20 Dec 2013 09:21:09 +0000

Is it possible to implement an efficient reverse on a rrb-vector resulting in an rrb-vector? This would be pretty useful in scenarios like the 2-opt neighborhood for local search for the Traveling Salesman Problem. The complete operation is as follows.

Given: a = (link: a_0, a1, ..., a{n-1}, a{n}, a{n+1}, ..., a{n+k-2}, a{n+k-1}, a{n+k}, ..., a{n+k+m-1})
Goal: (link: a_0, a1, ..., a{n-1}, a{n+k-1}, a{n+k-2}, ..., a{n+1}, a{n}, a{n+k}, ..., a{n+k+m-1})

Possible Clojure implementation of the described operation:

(let [x (subvec a 0 n), y (subvec a n (+ n k)), z (subvec a (+ n k))] (catvec x, (reverse-vec y), z))