Specify an alias that is a set of other aliases?

Question

It would be useful to have an alias that enables other aliases.

One use case is sometimes doing development against just one project, but other times doing development against a collection of projects.

Here's an example:

{:aliases 
  {:dev/project1 {:override-deps {:local/root "/some/path/project1"}}
   :dev/project2 {:override-deps {:local/root "/some/path/project2"}}
   :dev/project3 {:override-deps {:local/root "/some/path/project3"}}
   ;; something like the following:
   :dev/all-projects {:aliases #{:dev/project1 :dev/project2 :dev/project3}}}

Answer 1

(following a Slack discussion asking about the status https://clojure.atlassian.net/browse/CLJ-2638)

The idea of an alias requiring other aliases might not seem that useful at first but I believe it is a powerful construct for organizing Clojure monorepos in a simple way. Of course, there are quite a few ways for tackling the question of Clojure monorepos but a straightforward approach is:

• Maintain a single deps.edn at the root of the repo
• 1 alias per external dependency
• 1 alias per internal "module" and concern (:foo/main, :foo/test, :bar/main, :bar/test, ...)

This gives us:

• Everything in one clear deps.edn
• No need to build deps.edn files dynamically, like seen in some projects (defeats the purpose of having a declarative tool)
• External dependencies are pinned ; e.g. all modules requiring an external library use the same version without duplication by using a dedicated alias for that library
• Very easy to build the exact classpath that you need ; simply concat aliases
• No situations like transitive local deps not being updated
• No unholy practises like using ../../local-lib/src paths

This is a very precise approach and just works. The only downside is that you now have to juggle with many aliases and it is not always obvious what is really needed or not:

clj -M:module-1/dev:module-1:main:module-2/main:ext/lib-1:ext/lib-2:ext/lib-3...
You can of course script invokations requiring a gazillion aliases but it is unnecessarily complex and error-prone. An alias being able to require a vector of aliases would solve this, case dismissed.

Comments

Another benefit is that it plays nice with CI and ops since it prevents having to update a gazillion scripts when adding an alias for instance (i.e. you can always use the same "main" alias everywhere and add/remove aliases it requires, chance is invisible outside `deps.edn`).

---

We have a monorepo and this was definitely one of the pain points until we landed on a structure that was better suited to deps.edn -- see my series of blog posts on corfield.org about that journey -- and now we're migrating to Polylith, and that only requires a couple of aliases. That said, my own development setup relies on an invocation like this:

SOCKET_REPL_PORT=5000 clojure -J-Dlog4j2.configurationFile=log4j2-sean.properties -M:rebel:portal:everything:dev:test:runner:build:add-libs:dev/repl

and it would certainly be nice to be able to wrap that all up in a single :dev/sean alias (and I often add a couple _more_ aliases to that list for certain tasks).

Answer 2

HT to John Stevenson for pointing out that there's the potential for conflicts when inheriting from multiple aliases. For this reason, referring to a vector of aliases instead of a set is probably preferable. There may be other design challenges to work out here as a result of this concern, but I still think it's useful feature to have.

Comments

yeah, ordering is definitely important

Answer 3

My concern would be introducing the complexities of inheritance (and even multiple inheritance) into an otherwise simple configuration. One of the key things I learned from years of Java was to avoid these approaches and use composition, which provided much greater flexibility.

Would using tools.build be a simpler approach to addressing this issue with mono-repos?

In the example posed in the original question, would there be a mechanism to guard against or at least warn of a situation of:dev/project1 included :dev/all-projects or another alias that grouped aliases? How many levels down could this grouping go ?

Would there be well defined precedence rules?
Would there be tools to diagnose where the conflicts are - perhaps -Stree is enough, but I am not sure

Would aliases grouping be limited to aliases defined in the project deps.edn or would the user level deps.edn be also included (I assume it would add more complexity to exclude the user level config)

Perhaps a way to avoid this would be to have something like a :meta-aliases that could only include alases and not other :meta-aliases, but then that still seems to be adding complexity.

I would hope that the issue specific to mono-repls could be solved without adding complexity to the deps.edn configuration (especially avoiding inheritance or even multiple inheritance).

Answer 4

I think a syntax that might work would be one sort of like what Leiningen does:

{:aliases
 {:with-tests {:extra-paths ["test"]}
  :with-dev-deps {:extra-deps [...]}
  :dev [:with-test :with-dev-deps {:exec-fn some/fn}]}}

an alias inside :aliases could be either a normal map like we already support, or a vector of aliases named, optionally followed by a map. The vector of aliases could just be the equivalent of including those aliases as well, e.g. the dev alias in the example above would basically be the equivalent of with-tests:with-dev-deps:dev (using the map part of dev)

Answer 5

Logged as https://clojure.atlassian.net/browse/CLJ-2638

Answer 6

This would be mighty useful for linting.

clj -M:eastwood:kibit:antq

could be

clj -M:lint

Comments

When you combine aliases that all contain :main-opts, only the last one is used (because main opts are positional, passed to clojure.main's -main function), so you cannot run multiple lint tools via a single command-line like this, even with multiple aliases.

You need some other process to run each of the three lint tool's -main functions one after the other.

---

Yeah, this is a well known issue, but still one should be able to overcome this by providing a handcrafted :main-opts that combines all the tool's main fns calls into one within the :lint alias itself e.g. as per Cam Saul's Lein-alike syntax example. This way it will be the last one and shall override the rest.

---

Mark, you mean have :main-opts that contains a "-e" and an expression that programmatical calls each of the three -main functions? Like "(do (eastwood/-main) (kibit/-main) (antq/-main))"
(whatever the main namespaces are).

Given that a lot of tools call (shutdown-agents) at the end, you likely won't be able to run -main functions after that...