Welcome! Please see the About page for a little more info on how this works.

0 votes
in tools.deps.alpha by

It is impossible to use different versions of the same transitive dependency.
Will it be possible?
Can Java 9 (jigsaw) help?

1 Answer

+2 votes
selected by
Best answer

It's not possible by default, and jigsaw doesn't help. However, there is hope.

In the JVM the default classloader setup uses the Java classpath. The classpath is a list of locations to search (in order) for the location of a class. As such, if there are two versions of the same dependency on the classpath, whichever one happens to be first in the classpath will be loaded.

The module system introduced in Java 9 introduces a new concept of "modules" and a new "module path" that specifies the base set of modules to load. Modules provide new levels of encapsulation, preventing external use of non-public parts of the implementation, which can be useful. However, the module system has no concept of version or version selection, so the same problem of including multiple versions of the same module exists. The module loader will detect and complain about this problem, rather than the silent choice you get with the classpath loading. But it does nothing to "fix" it.

When you use Maven (or Maven-based resolution systems like Leiningen or Boot), you are asking Maven to examine the tree of transitive dependencies and pick one version to use. With Maven, the choice is essentially based on first version encountered from a breadth-first walk of the transitive tree. tools.deps uses its own selection algorithm that prefers the newest version of a dependency instead (while still building logically sensible dep trees). If you follow the advice in Rich's Spec-ulation keynote and make only additive, not breaking changes in your versions, then this will greatly reduce breaking conflicts from unexpected version selection.

If you literally want to load multiple versions of a library at the same time, the Java classloader architecture does allow you to do that, but it requires a custom classloader and some architectural discipline. In this scenario you create an interface (or protocol) that is loaded from your base classloader, and then use a post-delegation classloader and load implementations of the classes only in that classloader. The post-delegation allows each independent "plugin" to load whatever it needs but then communicate back to the app through base-level interfaces. As I said, this works but requires a significant amount of setup and discipline and has some limitations on how it works, so is usually only worth doing if you are building a product that you strongly expect to have contributed "plugin" type architecture with a high likelihood of dependency conflicts. OSGi is another approach to this problem. Neither of these is particularly fun to implement in Clojure.

The JVM team seems really averse to the idea of letting people load multiple versions of the same classes. They went as far as detecting renames where versions would be appended to prevent people from embedding versions in modules. No idea why.

Question I had was, is there a mode where tools.deps could just fail on conflict and use explicit declarations to specify the version in such case?
In tools.deps, top-level dep versions are always used regardless of any other version specified in transitive deps. So, you can always explicitly declare the dep at the top level to "pin" a particular version.

Having some option to influence version selection is possible but it's then much harder to convey to others what to do when using your lib, so I'm quite hesitant to make that something that surfaces in clj.

I'm not even sure what "conflict" should mean - if you are talking just about multiple versions for the same lib occurring in the deps  tree, this happens on essentially every non-trivial set of deps (even if jus resolving the version of Clojure to include).