Why does the Clojure compiler give different output for the same input?

Question

I have observed an oddity with the Clojure compiler: that it can produce different output for the same input. i.e. the generated .class files contain different bytes, though otherwise have identical runtime behavior. Why is this? I have created a Docker container and script which demonstrates the problem.

Though otherwise harmless in terms of the functionality of the resulting Java program, this is an issue for any attempts to cache the compiled classes based on simple byte equality, such as with Docker layer caching. Ideally, it would be possible to compile third-party Clojure libraries and store these in a separate Docker layer in containers for Clojure apps. If such a layer resulted in the same byte output given a particular fixed set of dependencies, then the layer could be recognized as already existing when pushing to a repository. This would avoid the storage & network costs that result from creating a whole new uberjar for every single source change within your application code.

Such caching is possible when not using AOT compilation, but this comes at the cost of slower application startup.

Answer 1

There are a few different reasons this can happen, but two of the main reasons are state being held in unordered Clojure collections (sets or maps) during compilation, or relying on the (arbitrary) result order returned from Java reflection APIs.

We do not consider it a priority to make Clojure builds repeatable, but we are happy to evaluate such issues and fix when it makes sense. (Example from 1.11: https://clojure.atlassian.net/browse/CLJ-1973).

Working with a whole Docker thing is a lot - if you can narrow this down to a specific problem, I'm happy to file a jira with it.

Comments

Thanks for such a quick answer Alex! It's a shame that repeatable builds aren't a priority, but I guess that's a reasonable tradeoff in prioritising Clojure's dynamism.

Perhaps the lack of guarantees about build reproducibility could be worth mentioning on the AOT compilation docs page? (https://clojure.org/reference/compilation)