Welcome! Please see the About page for a little more info on how this works.

0 votes
in core.logic by
`lein benchmark` (or some other non-lein-based incantation) prints a report listing the name of each benchmark and its timing.

Example:

$ lein benchmark
membero 2839
zebra 152738
$ lein benchmark comparison-report {:baseline "benchmark-5.9.2013-1"
                                    :runs 5
                                    :diffs-only true
                                    :threshold 25} ; only consider different if the delta is > 25 ms.                                     
membero +68ms
zebra -122ms
$ lein benchmark {:pretty true}
Thu May  9 11:21:41 PDT 2013

Linux mars 2.6.32-5-amd64 #1 SMP Fri Feb 15 15:39:52 UTC 2013 x86_64 GNU/Linux

java version "1.7.0_21"
Java(TM) SE Runtime Environment (build 1.7.0_21-b11)
Java HotSpot(TM) 64-Bit Server VM (build 23.21-b01, mixed mode)

membero     2839 ms
zebra     152738 ms


I haven't looked for any Clojure benchmarking libs, but ideally this would be a trivial script that automates the repetitive manual task of running benchmark tests. Unlike the test suite, we aren't looking for binary success or failure. Every run will generate unique results, so the script should accommodate a fuzzier comparison.

A "pretty" output that includes system info would be great for bug reports.

2 Answers

0 votes
by

Comment made by: dnolen

Sounds like an excellent enhancement to me. Patch welcome for this.

0 votes
by
Reference: https://clojure.atlassian.net/browse/LOGIC-136 (reported by austinhaas)
...