ThreadLocalRandom instead of Math/random

Question

The standard Math.random() is thread-safe through being declared as a synchronized static method.

The patch uses java.util.concurrent.ThreadLocalRandom which actually seems to be two times faster than the ordinary Math.random() in a simple single threaded criterium.core/bench:

The reason I investigated the function at all was to be sure random-number generation was not a bottleneck when performance testing multithreaded load generation.

If necessary, one could of course make a conditional declaration (like in fj-reducers) based on the existence of the class java.util.concurrent.ThreadLocalRandom, if Clojure 1.7 is to be compatible with Java versions < 1.7

Answer 1

_Comment made by: claj_

Benchmark on current rand (clojure 1.6.0), $ java -version
java version "1.7.0_51"
OpenJDK Runtime Environment (IcedTea 2.4.4) (7u51-2.4.4-0ubuntu0.13.10.1)
OpenJDK 64-Bit Server VM (build 24.45-b08, mixed mode)

:jvm-opts ^:replace [] (ie no arguments to the JVM)


(bench (rand 10))                                                                                                                                                                                                            
Evaluation count : 1281673680 in 60 samples of 21361228 calls.                                                                                                                                                                     
             Execution time mean : 43.630075 ns                                                                                                                                                                                    
    Execution time std-deviation : 0.420801 ns                                                                                                                                                                                     
   Execution time lower quantile : 42.823363 ns ( 2.5%)                                                                                                                                                                            
   Execution time upper quantile : 44.456267 ns (97.5%)                                                                                                                                                                            
                   Overhead used : 3.194591 ns                                                                                                                                                                                     
                                                                                                                                                                                                                                   
Found 1 outliers in 60 samples (1.6667 %)                                                                                                                                                                                          
        low-severe       1 (1.6667 %)                                                                                                                                                                                              
 Variance from outliers : 1.6389 % Variance is slightly inflated by outliers

Clojure 1.7.0-master-SNAPSHOT.

(bench (rand 10))                                                                                       
Evaluation count : 2622694860 in 60 samples of 43711581 calls.                                                
             Execution time mean : 20.474605 ns                                                               
    Execution time std-deviation : 0.248034 ns                                                                
   Execution time lower quantile : 20.129894 ns ( 2.5%)                                                       
   Execution time upper quantile : 21.009303 ns (97.5%)                                                       
                   Overhead used : 2.827337 ns                                                                
                                                                                                              
Found 2 outliers in 60 samples (3.3333 %)                                                                     
        low-severe       2 (3.3333 %)                                                                         
 Variance from outliers : 1.6389 % Variance is slightly inflated by outliers

I had similar results on Clojure 1.6.0, and ran several different tests with similar results. java.util.Random.nextInt is suprisingly bad. The ThreadLocalRandom version of .nextInt is better, but rand-int can take negative integers, which would lead to some argument conversion for (.nextInt (ThreadLocalRandom/current) n) since it need upper and lower bounds instead of a simple multiplication of a random number [0,1).

CHANGE:

The (.nextDouble (ThreadLocalRandom/current) argument) is very quick, but cannot handle negative arguments. The speed given a plain multiplication is about 30 ns.

Answer 2

Comment made by: claj

Added some simplistic tests to be sure that rand and rand-int accepts ratios, doubles and negative numbers of various kinds. A real test would likely include repeated generative testing, these tests are mostly for knowing that various arguments works etc.

Answer 3

Comment made by: claj

0001-rand-using-ThreadLocalRandom-and-tests-for-random.patch contains the changed (rand) AND the test cases.

Answer 4

Comment made by: alexmiller

Clojure requires Java 1.6.0 so this will need to be reconsidered at a later date. We do not currently have any plans to bump the minimum required JDK in Clojure 1.7 although that could change of course.

Answer 5

Comment made by: gfredericks

I've always thought that the randomness features in general are of limited utility due to the inability to seed the PRNG, and that a * } dynamic var would be a reasonable way to do that.

Maybe both of these problems could be partially solved with a standard library? I started one at https://github.com/fredericksgary/four, but presumably a contrib library would be easier for everybody to standardize on.

Answer 6

Comment made by: claj

Gary, I'm all for creating some well-thought out random-library, which could be a candidate for some library clojure.core.random if that would be useful.

Please have a look at http://code.google.com/p/javarng/ since that seems to do what you library four does (and more). Probably we could salvage either APIs, algorithms or both from this library.

I'll contact you via mail!

Answer 7

Comment made by: gfredericks

Come to think of it, a rand var in clojure.core shouldn't be a breaking change, so I'll just make a ticket for that to see how it goes. That should at the very least allow solving the concurrency issue with {{binding}}. The only objection I can think of is perf issues with dynamic vars?

Answer 8

Comment made by: gfredericks

New issue is at CLJ-1452.

Answer 9

Comment made by: jafingerhut

Patch 0001-rand-using-ThreadLocalRandom-and-tests-for-random.patch dated May 11 2014 no longer applied cleanly to latest master after some commits were made to Clojure on Aug 29, 2014. It did apply cleanly before that day.

I have not checked how easy or difficult it might be to update this patch. See section "Updating Stale Patches" on this wiki page for some tips on updating patches: http://dev.clojure.org/display/community/Developing Patches

Answer 10

Reference: https://clojure.atlassian.net/browse/CLJ-1420 (reported by claj)