Compile parser ahead of time

Mon, 27 Feb 2023 19:57:36 +0000

tools.cli/parse-opts accepts the args, the option-spec, and the additional options. It calls compile-option-specs and required-arguments to build the specs and req. Then it performs the validation with those specs and req on the provided args (along with the options). For a given application, the first two steps aren't going to change across calls.

I propose a new make-parse-opts-fn function that performs the compile-option-specs and required-arguments up front and returns a function that relies on the compiled specs and req. It could be used like this: (def compiled-parser (make-parse-opts-fn cli-options)).

Microbenchmarking with criterium shows a more than double increase in speed (tools.cli/parse-opts first, pre-compiled parser second):

; user=> (def cli-options
  [["-h" "--help" "This message"]
   [nil "--extra" "Output in extra format"
    :default false]
   ["-q" "--quiet" "Print no suggestions, only return exit code"
    :default false]])
#'user/cli-options

; user=> (bench (cli/parse-opts ["--quiet" "src"] cli-options :in-order true))
Evaluation count : 10962 in 6 samples of 1827 calls.
             Execution time mean : 65.021329 µs
    Execution time std-deviation : 5.276033 µs
   Execution time lower quantile : 58.658638 µs ( 2.5%)
   Execution time upper quantile : 69.724228 µs (97.5%)
                   Overhead used : 9.607933 ns
nil

; user=> (bench (compiled-parser ["--quiet" "src"] :in-order true))
Evaluation count : 24660 in 6 samples of 4110 calls.
             Execution time mean : 25.090846 µs
    Execution time std-deviation : 253.361821 ns
   Execution time lower quantile : 24.769079 µs ( 2.5%)
   Execution time upper quantile : 25.413286 µs (97.5%)
                   Overhead used : 9.607933 ns
nil

If the additional options are lifted into the make-parse-opts-fn as well (trading flexibility for speed), the difference is even more dramatic, providing roughly 10x speed increase over the existing tools.cli/parse-opts function:

; user=> (bench (compiled-parser-2 ["--quiet" "src"]))
Evaluation count : 73116 in 6 samples of 12186 calls.
             Execution time mean : 8.349923 µs
    Execution time std-deviation : 51.419864 ns
   Execution time lower quantile : 8.282266 µs ( 2.5%)
   Execution time upper quantile : 8.407998 µs (97.5%)
                   Overhead used : 9.607933 ns

Found 2 outliers in 6 samples (33.3333 %)
	low-severe	 1 (16.6667 %)
	low-mild	 1 (16.6667 %)
 Variance from outliers : 13.8889 % Variance is moderately inflated by outliers
nil

I can provide a patch for this if there's interest.

Clojure Q&A - Recent questions tagged reuqest

Compile parser ahead of time