Transitive dependency resolution skipping some libs

Question

Hello.

I am using Clojure 1.11.1 (and org.clojure/tools.deps "0.18.1354") but I am experiencing problems with the transitive dependency resolution: some dependencies are not being included in the classpath.

The simplified scenario that is causing the issue is as follows:

"dependency tree""

my-company/my-lib :mvn/version 5
  |_ my-company/base-lib :mvn/version 1
       |_ general/some-lib :mvn/version 1.0
            |_ general/some-other-lib :mvn/version 1.0

my-company/another-lib :mvn/version 3
  |_ my-company/base-lib :mvn/version 2
       |_ general/some-lib :mvn/version 1.0
            |_ general/some-other-lib :mvn/version 1.0

The issue that is happening is

transitive dependency expansion/resolution logic

add: [my-company/my-lib [5] -> my-company/base-lib [1] -> general/some-lib [1.0]]
add: [my-company/my-lib [5] -> my-company/base-lib [1] -> general/some-lib [1.0] -> general/some-other-lib [1.0]]
add: [my-company/my-lib [5] -> my-company/base-lib [2] -> general/some-lib [1.0]]  <== when this arrives the "children" [general/some-other-lib] is not added to the queue because the [general/some-lib [1.0]] has already been "seen" and previously added

However, the dependency "[my-company/my-lib [5] -> my-company/base-lib [1] -> general/some-lib [1.0] -> general/some-other-lib [1.0]]" will be "rejected" because of :missing-parent, as the "[my-company/my-lib [5] -> my-company/base-lib [1] -> general/some-lib [1.0]]" was replaced by the "[my-company/my-lib [5] -> my-company/base-lib [2] -> general/some-lib [1.0]]" (newer version of the my-company/base-lib). Unfortunately there is no entry in the queue for the general/some-lib [1.0], as it was previously skipped from being added, as its direct parent general/some-lib [1.0] had already been added (although from a different "absolute" path).
And then my project fails because the classes/namespaces from general/some-other-lib [1.0] could not be found, as this lib was not included in the classpath.

Debugging a bit I was able to identify that the problem seems to be with this part of the code:
https://github.com/clojure/tools.deps/blob/master/src/main/clojure/clojure/tools/deps.clj#L230-L240

Answer 1

This description doesn't make sense to me - in both branches "general/some-lib :mvn/version 1.0" exists and should be included with its own transitive deps, so it feels like something has been lost here in the process of redacting. In particular, I'm also wondering if there are :exclusions at any point in the tree.

It would be helpful if you could narrow the deps tree to whatever the equivalent is above, capture a trace with -Strace and send a gist of the trace.edn (redacted if necessary). I understand if redaction is necessary, but the closer I can get to the original, the easier it will be for me to understand what's happening. You can send dm on Clojurians Slack or email too if that's better.

You might also find this page helpful, particularly the orphans section:
https://clojure.org/reference/dep_expansion

Comments

Thank you for the reference, Alex. It helped me to pinpoint what was causing my issues.
(the redaction was to simplify because most of the libs were internal to my company, so exposing their names or not would be the same, as they are not publicly available)

My issue was caused by an `:exclusion` on the third dependency level:
my-bundle -> my-component -> my-lib
and "my-lib" contain this below in the "deps.edn":
     amazonica/amazonica                   {:mvn/version "0.3.165" :exclusions [com.taoensso/nippy]}
     com.taoensso/faraday                  {:mvn/version "1.12.0"  :exclusions [com.taoensso/nippy]}
     com.taoensso/nippy                    ^:antq/exclude {:mvn/version "3.1.1"}

Removing the `:exclusions`, as a version of nippy was being set at the same level, made the weird behavior with other libs disappear (although now Cursive in IntelliJ shows two nippy versions, 3.1.1 and 3.2.0, but deps' tree does not).

The inconsistent behavior was happening with the libs `tick/tick [0.7.5]` and `com.stuartsierra/dependency [1.0.0]` (from `com.stuartsierra/components [1.1.0]`) not being included in the classpath