Support preparing :deps/:extra-deps in aliases

Question

Right now there's no way to prepare dependencies only included as as :deps or :extra-deps in aliases with clj -X:deps prep. Adding support for an :aliases option that would prepare these be extremely helpful, e.g.

clj -X:deps prep :aliases '[:my-alias]'

Real-world use case: at Metabase we have a :drivers alias that adds several :local/root data warehouse driver subprojects to our classpath. For our Spark SQL driver we need to AOT a few :gen-class namespaces that we use as proxy JDBC drivers. Currently we have people run clj -X:deps prep in multiple directories; a way to pass aliases to prep would let us do this in a single command.

JIRA Issue: https://clojure.atlassian.net/browse/CLJ-2652

Answer 1

Added a patch for this to the JIRA issue

Answer 2

There are good reasons why we don't currently support this (all projects on the machine share the same cached lib paths, for example). Why can't these drivers be in separate projects that are included as needed? Or provided as artifacts?

Comments

Thanks for your quick reply, Alex. Let me try to explain the use case in a little more detail:

We have our various drivers up as separate subprojects, and we include them as needed as `:local/root` coordinates. We have a `:drivers` alias that includes all of our drivers that we use for various local dev tasks; we don't include them as top-level `:deps` because we build them with a separate process.

Sometimes we do `clj -M:run` (so as to not include our driver subprojects on the classpath), and sometimes we do `clj -M:drivers:run` (so as to include our driver subprojects)... the problem we have is that there's no easy way to prep the subprojects added by the `:drivers` alias short of manually `cd`ing into each of their respective directories and prepping them  individually.

The workaround I came up with was to create a dummy `deps.edn` file in a separate directory that contains the same `:local/root` `:extra-deps` as the `:drivers` alias, but under the top level `:deps`. People can `cd` to this directory and run `clj -X:deps prep` there instead of having to do it for each of the driver directories individually. So right now our first-time setup instructions are basically

```
clj -X:deps prep && cd modules/drivers && clj -X:deps prep
```

Doing `clj -X:deps` prep twice in two different folders isn't so bad, but doing `clj -X:deps prep :aliases '[:drivers]'` once (and not creating/maintaining the dummy `deps.edn` file) would be a lot nicer.


Arguably we _could_ bundle the files that have to be AOT'ed as separate artifacts, but pushing those to Clojars, or setting up an S3 bucket to serve as a Maven repo or something like that (and setting CI up to update it automatically on change) is not something I really want to do if I don't have to. Plus, it would make tweaking those files locally and testing changes more difficult.

RE affecting other projects on the same machine:

I'm not sure I'm following the reasons why it would make any difference. My understanding is that it ultimately wouldn't be any different from someone manually copying and pasting all the `:deps`/`:extra-deps` from the aliases in question into the top-level `:deps` of the project `deps.edn` and then running `clj -X:deps prep`. (The only difference is that we're potentially prepping more libs than we would have done otherwise; but we're not prepping stuff any differently.)

---

I think I misread your request as wanting to specify aliases in a dep, rather than aliases from the original project.

---

That's right, sorry if I wasn't clear enough. We just want to be able to prep our project with or without some of its various aliases in effect.

---

Adding another scenario: if you have a Polylith-based project, there are usually no :deps in the top-level EDN file -- everything for development is brought in under a :dev alias.

There are subprojects with their own deps.edn that would effectively let you prep a dependency via that path but, while you are initially developing, you only have the :dev alias because you haven't gotten as far as declaring how to build an artifact (which is what you would do in a "project").

I would say that, as a matter of general course, any and all tools in a deps.edn environment should allow for :aliases to be specified when calculating the basis needed for the tool to do its work (I don't mean on the initial CLI invocation via -T or -X but that the tools themselves should always accept an :aliases exec arg and use it to calculate the basis for the project).

Answer 3

Added support for basis modifiers, including :aliases, in Clojure CLI prerelease 1.10.3.1082.