Incorrect `rem` behavior between clojure.lang.BigInt, Double

Question

This is related to issue 1555, but the types at play are different, so the underlying problem might be different too.

rem seems to be returning very incorrect results between bigint and float. Note the large difference in using 10000 as the divisor vs 1e4:

user> (rem 9037601485536036300227801N 10000)
7801N
user> (long (rem 9037601485536036300227801N 1e4))
1073741824

test.check provides a tighter example of clearly incorrect results:

user> (rem 9007199254740993N 2)
1N
user> (rem 9007199254740993N 2.0)
0.0

Here's a generative test to reproduce this result:

(clojure.test.check/quick-check
 100 (prop/for-all [l gen/size-bounded-bigint
                    r (gen/fmap inc gen/nat)]
                   (== (rem l (double r))
                       (rem l (long r)))))

Answer 1

I believe that the current Clojure/JVM implementation of (rem 9007199254740993N 2.0) is the same as for (rem (double 9007199254740993N) 2.0), and so for large enough BigInt values will do some kind of rounding in the least significant bits, and so for small divisors will definitely give the wrong numerical answer.

Handling nearly arbitrary levels of precision for arbitrary pairs of numerical arguments of all supported types seems like a tough thing to deliver on, in terms of thinking, development, testing, fixing unanticipated corner cases, etc.

I am not a decision maker here, but I would not be surprised if the Clojure core dev team would lump this kind of behavior under "don't use float/double if you want exact answers, go read 'What every computer scientist should know about floating point arithmetic'"

https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html

Comments

A fair response and probably a correct guess about the core team's response! Generative testing just pushes this stuff in your face :) that in itself is a good lesson and a reason to use test.check everywhere.