`(s/gen ratio?)` throws exceptions

Question

(s/gen ratio?) is implemented as (gen/such-that ratio? gen/ratio)

The problem is that about 10% of the time gen/ratio returns a whole number, which fails the ratio? predicate. If that happens 10 times in a row, gen/such-that throws an exception.

With a large enough sample size, (s/gen ratio?) reliably fails:

(dorun (gen/sample (s/gen ratio?) 10000000))

Execution error (ExceptionInfo) at clojure.test.check.generators/fn (generators.cljc:435).
Couldn't satisfy such-that predicate after 10 tries.

A quick fix is to increase the max-tries param for such-that:

(dorun (gen/sample (gen/such-that ratio? gen/ratio 100) 10000000))

=> nil

I do however also find it odd that (s/gen ratio?) always generates Ratios, but gen/ratio generates a mixture of ratios and integers. So perhaps a better fix would be to update gen/ratio to not generate integers?

Answer 1

Yes, it seems weird that gen/ratio sometimes makes integers - comes from test.check. But also seems weird to have a spec that only accepts ratios but not integers - does that actually make sense for your use case?

Comments

We don't have specs that accept ratios and not integers, rather we have specs that accept most if not all number types and need to be able to generate all those different types (including ratios) to properly test them.

Unfortunately the built in generator `(s/gen number?)` only generates longs and doubles (that's a whole separate topic), so we intentionally generate bigints, bigdecs, ratios, etc. in our tests. The way I was generating ratios was with `(s/gen ratio?)`, which caused test flakes due to the such-that exception (which if you've ever experienced such flakes, you'll know they're the hardest ones to debug! But that's also a separate topic).

Switching to `(gen/ratio)` solves that and works for my use case because our specs also happen to accept integers, but I lost quite a lot of time getting to that point.

Answer 2

Logged as https://clojure.atlassian.net/browse/CLJ-2855