`lein benchmark` (or some other non-lein-based incantation) prints a report listing the name of each benchmark and its timing.
Example:
$ lein benchmark
membero 2839
zebra 152738
$ lein benchmark comparison-report {:baseline "benchmark-5.9.2013-1"
:runs 5
:diffs-only true
:threshold 25} ; only consider different if the delta is > 25 ms.
membero +68ms
zebra -122ms
$ lein benchmark {:pretty true}
Thu May 9 11:21:41 PDT 2013
Linux mars 2.6.32-5-amd64 #1 SMP Fri Feb 15 15:39:52 UTC 2013 x86_64 GNU/Linux
java version "1.7.0_21"
Java(TM) SE Runtime Environment (build 1.7.0_21-b11)
Java HotSpot(TM) 64-Bit Server VM (build 23.21-b01, mixed mode)
membero 2839 ms
zebra 152738 ms
I haven't looked for any Clojure benchmarking libs, but ideally this would be a trivial script that automates the repetitive manual task of running benchmark tests. Unlike the test suite, we aren't looking for binary success or failure. Every run will generate unique results, so the script should accommodate a fuzzier comparison.
A "pretty" output that includes system info would be great for bug reports.
