Clojure Q&A - Recent questions in tools.build

tools.build compile-clj capture out/err

Mon, 28 Jul 2025 19:41:29 +0000

When calling compile-clj, a user can pass in :out and :err to capture the output of the compilation call. However, if for some reason the sub-process exits with a non-0, compile-clj throws without returning that output, making it hard to know what went wrong. (see here: https://github.com/clojure/tools.build/blob/2d2f1b05fd84c3cc756221c17fcfb98034b4d65c/src/main/clojure/clojure/tools/build/tasks/compile_clj.clj#L120)

I would find it helpful if the ex-data of the thrown exception included the :out (variable ps-out and :err (variable ps-err) from the process/process call. Then I could have further insight into what went wrong.

tools.build fails with http_proxy/https_proxy set

Wed, 02 Jul 2025 11:56:53 +0000

The autopkgtest of the tools.build package fails with:

Running tests in #{"src/test/clojure"}

Testing clojure.tools.build.tasks.test-basis
Warning: failed to load the S3TransporterFactory class
Jul 01, 2025 8:15:38 AM org.apache.http.impl.execchain.RetryExec execute
INFO: I/O exception (java.net.SocketException) caught when processing request to {s}->https://repo1.maven.org:443: Network is unreachable
[...]

The autopkgtest script debian/tests/source:

#!/bin/bash

set -ueo pipefail

cp=($(< debian/libtools-build-clojure.classpath))
test "${cp[0]}" = usr/share/java/tools.build.jar
unset cp[0] # drop it

test_cp=(src/main/clojure
         src/test/clojure
         /usr/share/java/test-runner.jar
         /usr/share/java/maven-resolver-named-locks.jar
         /usr/share/java/maven-repository-metadata-3.x.jar
         "${cp[@]}")

test_cp="$(IFS=:; echo "${test_cp[*]}")"

# Derived from ./deps.edn, and avoids a circular dep on clojure-cli
java -XX:-OmitStackTraceInFastThrow -cp "$test_cp" clojure.main \
     -m cognitect.test-runner cognitect.test-runner.api/test \
     --dir src/test/clojure --namespace-regex '.*'

The Ubuntu autopkgtest infrastructure restrict the Internet access and sets http_proxy/https_proxy.

Please support those proxy settings.

Bug-Ubuntu: https://launchpad.net/bugs/2115740

Please consider making tools.build pom.properties reproducible (e.g. the "Generated by" date)

Sun, 16 Feb 2025 18:43:21 +0000

At the moment (in 1.2.24) the pom.properties includes the current date in a comment, making the resulting jars differ. For now, in Debian we're deferring to SOURCE_DATE_EPOCH whenever it's set: https://salsa.debian.org/clojure-team/tools-build-clojure/-/blob/5739183ec76c39453ff267c6ba19c0605eb406b7/debian/patches/0002-write-pom-make-pom.properties-generated-date-reproducible.patch

How to ignore :src-pom default for clojure.tools.build.api/write-pom if ./pom.xml exists?

Sun, 31 Mar 2024 20:50:22 +0000

I have a project that generates more than one jar artifact. For project A, I have ./pom.xml that is used as a template and everything works fine.

For project B, I want to generate a pom.xml that's completely separate from project A which does not use ./pom.xml as a template. I generate the pom as follows:

(require '[clojure.tools.build.api :as b])
(b/write-pom {:class-dir class-dir
                  :pom-data pom-data
                  :lib coord
                  :version version
                  :basis basis})

However, since ./pom.xml exists, it is used as a template. Explicitly passing nil for :src-pom doesn't help. The only workaround I could find was to pass in a bogus path for :src-pom.

Warn about user.clj during build compilation

Wed, 28 Feb 2024 13:20:46 +0000

We’ve had a few cases where people encountered issues due to the existence of user.clj during uberjar building with tools.build.

Because user loads early, anything it loads is not (typically) reloaded during compilation, which can lead to some namespaces not being compiled.

Should maybe warn if user is on classpath or even actively recompile with something like

(binding [*compile-files* true] (require 'user :reload-all))

switch tools.build to pull from jars rather than exploding onto disk

Tue, 29 Aug 2023 20:49:38 +0000

Following on from https://ask.clojure.org/index.php/10854/getting-an-error-with-tools-build-uberjar?show=10854#q10854

Jars are case-sensitive. This means that an uberjar can have a file named license and LICENSE/epl.txt simultaneously inside them. But if you explode these to disk when building an uberjar in a file system that is case-sensitive (in particular, OSX), you can an error along the lines of

Execution error (ExceptionInfo) at clojure.tools.build.tasks.uber/explode1 (uber.clj:166).
Cannot write META-INF/license/LICENSE.aix-netbsd.txt from io.grpc/grpc-netty-shaded as parent dir is a file from another lib. One of them must be excluded.

The solution to this is to track down the conflicting files and add them to exclusions. But often this can be tedious if the single file you want ot exclude license exists and the error message is about one of several files that conflicts LICENSE/epl.txt.

The error is annoying also because these files can sit together just fine in an uberjar, but not in their temporary location on disk. It would be nice if the uberjar could just pull files from each jar rather than copy them onto disk in an intermediate step.

Can 'write-pom' in tools.build support providing additional dependencies in the pom.xml with scope 'provided'?

Wed, 29 Mar 2023 16:55:52 +0000

To work appropriately in the Java ecosystem, optional dependencies for libraries can be included with scope 'provided' within a pom.xml file.

deps.edn explicitly does not support including dependencies with a scope of 'provided', in favour of using aliases.

In order to work well as a library within a wider Java ecosystem, it would be very helpful to be able to write out a library jar file with a pom.xml file with additional dependencies that are not included in deps.edn, but with a scope of 'provided' for Java build tools.

An example is cljdoc, which cannot build documentation when a library is built and all dependencies are not included appropriately.

'write-pom' already has an option to include a template 'source' pom.xml, but the dependencies are overwritten with information from the deps.edn file.

There are workarounds - e.g. see thread https://clojurians.slack.com/archives/C8V0BQ0M6/p1680090647156119 which patch the process to include deps with a 'provided' scope - but they are not ideal.

Options.

do nothing - we monkey patch tools.build as per Slack thread example, or otherwise alter the generated pom.xml to include additional 'provided' scope deps.
merge existing dependencies from the source (template) pom.xml instead of overwriting [but perhaps a breaking change]
provide an additional parameter to 'write-pom' to include arbitrary deps, a sequence of maps that are converted to XML including :scope which would be :provided for these optional dependencies.
work with cljdoc to support aliases so that documentation builds can be customised by library authors so builds such as this do not fail.

tools.build.api/uber uses EDN for data_readers.cljc -- it should support reader conditionals

Tue, 07 Mar 2023 19:36:03 +0000

The conflict handler for data_readers.clj* treats the files as EDN when it reads them to merge them. This fails for .cljc files containing reader conditionals:

RuntimeException No dispatch macro for: ?
	clojure.lang.Util.runtimeException (Util.java:221)
	clojure.lang.EdnReader$DispatchReader.invoke (EdnReader.java:552)
	clojure.lang.EdnReader.readDelimitedList (EdnReader.java:757)
	clojure.lang.EdnReader$MapReader.invoke (EdnReader.java:680)
	clojure.lang.EdnReader.read (EdnReader.java:145)
	clojure.lang.EdnReader.read (EdnReader.java:111)
	clojure.lang.EdnReader.readString (EdnReader.java:67)
	clojure.edn/read-string (edn.clj:46)
	clojure.edn/read-string (edn.clj:37)
	clojure.edn/read-string (edn.clj:37)
	clojure.tools.build.tasks.uber/conflict-data-readers (uber.clj:89)
	clojure.tools.build.tasks.uber/conflict-data-readers (uber.clj:86)

Ability to customise java-cmd in tools.build

Sat, 27 Aug 2022 20:54:32 +0000

Moving the original question from Clojurians Slack:

When moving one of my projects to JDK 19 I was pointing clojure to the jdk using the JAVA_CMD env var. Everything was fine til I hit the uber jar step in the CI, where it simply refused to recognise the jdk 19 functions.

Took me a while and including reading the tools.build source to realise why it wasn't propagating the JAVA_CMD var and turns out I had to read env and send the :java_cmd in the b/compile-clj call.

I can see why the behaviour is like this, but would :java_cmd defaulting to the value of JAVA_CMD something that could be considered? Feels a bit more intuitive for me, as mostly the invocations of tools.build fns would be from the shell.

Exception in EdnReader$DispatchReader

Sat, 25 Jun 2022 16:32:02 +0000

(from https://clojurians.slack.com/archives/C02B5GHQWP4/p1656165951371179)
When I include tick/tick {:mvn/version "0.5.0-RC6"} as a dep, I get the exception below. I cannot reproduce in an deps.edn that only contains this library so I believe there is some kind of conflict with another library.

{:clojure.main/message
"Execution error at clojure.tools.build.tasks.uber/conflict-data-readers (uber.clj:90).\nNo dispatch macro for: ?\n",
:clojure.main/triage
{:clojure.error/class java.lang.RuntimeException,
:clojure.error/line 90,
:clojure.error/cause "No dispatch macro for: ?",
:clojure.error/symbol
clojure.tools.build.tasks.uber/conflict-data-readers,
:clojure.error/source "uber.clj",
:clojure.error/phase :execution},
:clojure.main/trace
{:via
[{:type java.lang.RuntimeException,

:message "No dispatch macro for: ?",
:at [clojure.lang.Util runtimeException "Util.java" 221]}],

:trace
[[clojure.lang.Util runtimeException "Util.java" 221]
[clojure.lang.EdnReader$DispatchReader invoke "EdnReader.java" 552]
[clojure.lang.EdnReader readDelimitedList "EdnReader.java" 757]
[clojure.lang.EdnReader$MapReader invoke "EdnReader.java" 680]
[clojure.lang.EdnReader read "EdnReader.java" 145]
[clojure.lang.EdnReader read "EdnReader.java" 111]
[clojure.lang.EdnReader readString "EdnReader.java" 67]
[clojure.edn$read_string invokeStatic "edn.clj" 46]
[clojure.edn$read_string invokeStatic "edn.clj" 37]
[clojure.edn$read_string invoke "edn.clj" 37]
[clojure.tools.build.tasks.uber$conflict_data_readers

invokeStatic
"uber.clj"
90]

[clojure.tools.build.tasks.uber$conflict_data_readers

invoke
"uber.clj"
86]

[clojure.tools.build.tasks.uber$handle_conflict

invokeStatic
"uber.clj"
120]

[clojure.tools.build.tasks.uber$handle_conflict

invoke
"uber.clj"
111]

[clojure.tools.build.tasks.uber$explode invokeStatic "uber.clj" 157]
[clojure.tools.build.tasks.uber$explode invoke "uber.clj" 137]
[clojure.tools.build.tasks.uber$uber$fn3537$fn3541

invoke
"uber.clj"
253]

[clojure.lang.PersistentVector reduce "PersistentVector.java" 343]
[clojure.core$reduce invokeStatic "core.clj" 6885]
[clojure.core$reduce invoke "core.clj" 6868]
[clojure.tools.build.tasks.uber$uber$fn__3537 invoke "uber.clj" 252]
[clojure.core.protocols$iter_reduce invokeStatic "protocols.clj" 49]
[clojure.core.protocols$fn__8230 invokeStatic "protocols.clj" 75]
[clojure.core.protocols$fn__8230 invoke "protocols.clj" 75]
[clojure.core.protocols$fn8178$G8173__8191

invoke
"protocols.clj"
13]

[clojure.core$reduce invokeStatic "core.clj" 6886]
[clojure.core$reduce invoke "core.clj" 6868]
[clojure.tools.build.tasks.uber$uber invokeStatic "uber.clj" 250]
[clojure.tools.build.tasks.uber$uber invoke "uber.clj" 237]
[clojure.lang.Var invoke "Var.java" 384]
[clojure.tools.build.api$uber invokeStatic "api.clj" 480]
[clojure.tools.build.api$uber invoke "api.clj" 408]
[org.corfield.build$uber invokeStatic "build.clj" 250]
[org.corfield.build$uber invoke "build.clj" 208]
[build$uber invokeStatic "build.clj" 14]
[build$uber invoke "build.clj" 12]
[clojure.lang.AFn applyToHelper "AFn.java" 154]
[clojure.lang.AFn applyTo "AFn.java" 144]
[clojure.lang.Var applyTo "Var.java" 705]
[clojure.core$apply invokeStatic "core.clj" 667]
[clojure.core$apply invoke "core.clj" 662]
[clojure.run.exec$exec invokeStatic "exec.clj" 48]
[clojure.run.exec$exec doInvoke "exec.clj" 39]
[clojure.lang.RestFn invoke "RestFn.java" 423]
[clojure.run.exec$_main$fn__205 invoke "exec.clj" 180]
[clojure.run.exec$_main invokeStatic "exec.clj" 176]
[clojure.run.exec$_main doInvoke "exec.clj" 139]
[clojure.lang.RestFn applyTo "RestFn.java" 137]
[clojure.lang.Var applyTo "Var.java" 705]
[clojure.core$apply invokeStatic "core.clj" 667]
[clojure.main$main_opt invokeStatic "main.clj" 514]
[clojure.main$main_opt invoke "main.clj" 510]
[clojure.main$main invokeStatic "main.clj" 664]
[clojure.main$main doInvoke "main.clj" 616]
[clojure.lang.RestFn applyTo "RestFn.java" 137]
[clojure.lang.Var applyTo "Var.java" 705]
[clojure.main main "main.java" 40]],
:cause "No dispatch macro for: ?"}}

relative directories in classpath-roots

Sun, 19 Jun 2022 16:50:53 +0000

In the context of a monorepo containing several modules I have noticed an issue with clojure.tools.build.api/compile-clj which is making difficult to compile files when the basis project root is not in the current directory.

Here is a minimal scenario demonstrating the problem.

I have a top-level monorepo deps.edn using tools.build to prepare/package its modules

 {:deps {monorepo/module-a {:local/root "modules/a"}}
 :aliases {:build {:deps {io.github.clojure/tools.build {:git/tag "v0.8.2"
                                                         :git/sha "ba1a2bf"}}
                   :ns-default build}}}

Module A contains some clojure source files. Here is modules/a/deps.edn

{:paths ["src"]}

Those files require a compilation step. Here is the top-level build.clj file expected to proccess those files.

(ns build
  (:require [clojure.tools.build.api :as b]))

(def module-a (binding [b/*project-root* "modules/a"]
                (b/create-basis {:project "deps.edn"})))

(defn compile-a [_]
  (b/compile-clj
   {:basis module-a
    :src-dirs ["modules/a/src"]
    :class-dir "target/module-a/classes"}))

However when invoking clojure -T:build compile-a from top-level directory I encounter the following error

Execution error (FileNotFoundException) at user/eval136$fn (compile.clj:5).
Could not locate monorepo/module/a__init.class, monorepo/module/a.clj or monorepo/module/a.cljc on classpath.

The problem is that the "src" relative directory is not expanded in the :classpath-roots which lead compile-clj to search source files in the wrong location. It is possible to work around the issue by binding the *project-root* again and adapting :src-dirs and class-dir relative to module A root directory like this

(defn compile-a [_]
  (binding [b/*project-root* "modules/a"]
    (b/compile-clj
     {:basis module-a
      :src-dirs ["src"]
      :class-dir "../../target/module-a/classes"})))

However IMO it would be better if the computed classpath in the basis was fully expanded by transforming relative paths into absolute ones to avoid this extra ceremony.

What do you think?
Is there a reason why relative source path directories are currently kept relative ?

Files in deps :local/root :git/url will directly copy into uberjar without go through conflict-handlers

Fri, 22 Apr 2022 10:58:50 +0000

https://github.com/clojure/tools.build/blob/1309f935b098123eb807c972a053eeab77f6f4cd/src/main/clojure/clojure/tools/build/tasks/uber.clj#L172

The handling between jar and lib folders like git deps or local root are completely different. In my case, I have some config files in edn format (duct's duct_hierarchy.edn) exists in both jar deps and git deps. My additional conflict handler is never receiving the edn file that exists in git deps.

Is it the expected behaviour? I can imagine that even LICENSE or data_readers.clj would suffer the same issue. e.g. being overrided by files in git deps.

Is it possible to set a var during compilation in tools.build?

Mon, 28 Feb 2022 20:50:43 +0000

Specifically I'd like to set clojure.test/*load-tests* to false, so that test code is excluded by the clojure.test/deftest,with-test etc. macro's. Lein provided injections as an escape hatch, tools.build doesn't seem to have an equivalent.

Looking at the code, tools.build seems to compile code from a separate generated script in a temp file, which seems to make injections or wrappers impossible. Also couldn't find options to pass other than compiler options.

A more in-depth description of this use case can be found on clojureverse: https://clojureverse.org/t/require-import-inside-deftest/8644/3

Is there a solution or suggested workaround available?

clojure.tools.build.api/process + ignore hangs when process has large output

Tue, 25 Jan 2022 09:56:37 +0000

tools.build.api/process hangs when ignoring output and the process has large output. Here is curl downloading and printing 1mb of text:

$ clj -Sdeps '{:deps {io.github.clojure/tools.build {:git/tag "v0.7.5" :git/sha "34727f7"}}}'
Clojure 1.11.0-alpha3
user=> (require '[clojure.tools.build.api :as tb])

user=> (tb/process {:command-args ["curl" "https://gist.githubusercontent.com/khaykov/a6105154becce4c0530da38e723c2330/raw/41ab415ac41c93a198f7da5b47d604956157c5c3/gistfile1.txt"] :out :ignore})
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0

I think instead of using PIPE, the ignore option should be using DISCARD:

https://docs.oracle.com/en/java/javase/16/docs/api/java.base/java/lang/ProcessBuilder.Redirect.html#DISCARD

One caveat: DISCARD is only available on Java 9 and newer. We could do resolve in a try/catch on the top level to see if DISCARD is available and then use that, if available and fall back on PIPE if it isn't. Alternatively, use a thread to consume output while the process is running or redirect to a temporary file.

I'll provide a patch, if desired, and when agreed upon the direction of the solution.

Will tools.build be released as a jar?

Wed, 05 Jan 2022 15:02:43 +0000

I am developing a library with a dependency on tools.build and I would like to publish it as a jar. Whilst tools.build is only available as a git dep, I cannot include it as a dependency in my pom file. A wider discussion on this topic was held on this slack thread, where @alexmiller requested that I raise this topic here.

Allow specifying the output pom path

Tue, 07 Dec 2021 21:25:13 +0000

My initial use for tools.build is to update the pom.xml in the project with the appropriate version. It would be great if I could specify that write-pom should be output to pom.xml, rather than the folder layout that is imposed on me by the current function.

tools.build/jar doesn't support "sections" in manifest

Sun, 05 Dec 2021 14:52:52 +0000

Sections are groups which start with Name: and have section-specific entries in them.

Capsule uses this for having platform-specific information about how to start.

Here's an example of using the API to get section information:

Main-Class: clojure.main

Name: Frank
Main-Class: frank.clojure.main

and

❯ clj       
Clojure 1.10.3
user=> (require '[clojure.java.io :as io])
nil
user=> (io/input-stream "/tmp/MANIFEST.MF")
#object[java.io.BufferedInputStream 0x69ce2f62 "java.io.BufferedInputStream@69ce2f62"]
user=> (def mf (java.util.jar.Manifest. (io/input-stream "/tmp/MANIFEST.MF")))
#'user/mf
user=> (.getAttributes mf "Frank")
{#object[java.util.jar.Attributes$Name 0x2e3a5237 "Main-Class"] "frank.clojure.main"}
user=> (.getMainAttributes mf)
{#object[java.util.jar.Attributes$Name 0x2e3a5237 "Main-Class"] "clojure.main"}
user=> (.getEntries mf)
{"Frank" {#object[java.util.jar.Attributes$Name 0x2e3a5237 "Main-Class"] "frank.clojure.main"}}

tools.build copy-dir with replace loses file permissions

Fri, 03 Dec 2021 06:29:30 +0000

When you use clojure.tools.build.api/copy-dir with :replace it reads the source file and then writes a new target file -- it doesn't "copy" the file so the target file is created with default permissions.

This is a problem if you are copying executable shell scripts and trying to replace text in them because the target file created won't have executable permissions.

This cropped up with deps-new in this issue: https://github.com/seancorfield/deps-new/issues/22

I think that if you source file is executable, the target file should also be executable -- and that works fine without :replace.

Is it possible to build an uberwar using deps.edn using tools.build or other tool?

Thu, 02 Dec 2021 13:22:01 +0000

We have a small Clojure & ClojureScript web app running in Tomcat. The project managed to end up with a deps.edn for use with Shadow-cljs & lein for the backend to essentially build an uberwar. Is there a way to build an uberwar with tools-build or in some other way using deps.edn?

Thanks for any pointers

Cheers
Daniel Sunnerek

tools.build uber's default conflict handlers don't append to META-INF/services properly

Fri, 26 Nov 2021 17:29:12 +0000

Our project has some dependencies that register parsers for various file formats via the java services framework.

We’re using b/uber and are finding some META-INF/services/ definitions are not correctly being appended.

I’ve chased this down to the regex in the default-handlers being ^META-INF/services/ instead of something more like ^META-INF/services/.*

The current regex would match under re-find, but not under re-matches which the implementation uses.

tools.build uber task fails when the dependency tree contains a dep which has no jar

Tue, 12 Oct 2021 14:00:42 +0000

The clojure.tools.build.api/uber task fails when the dependency tree contains a dep which has no jar.

A relatively minimal failing test case is the following tools.build task:

(ns build 
  (:require [clojure.tools.build.api :as b]))

(defn failing-uber [_]
    (b/uber {:class-dir "classes"
     :uber-file "foo.jar"
     :basis (b/create-basis {:project {:deps {'org.apache.jena/apache-jena-libs {:mvn/version "4.2.0" :extension "pom"} }}})})

(comment 
   (failing-uber {})
 )

Evaluating the above function results in the following exception:

1. Unhandled clojure.lang.ExceptionInfo
   Unexpected lib file:
   /Users/rick/.m2/repository/org/apache/jena/apache-jena-libs/4.2.0/apache-jena-libs-4.2.0.pom
   {}
                  uber.clj:  182  clojure.tools.build.tasks.uber/explode
                  uber.clj:  141  clojure.tools.build.tasks.uber/explode
                  uber.clj:  257  clojure.tools.build.tasks.uber/uber/fn/fn
     PersistentVector.java:  343  clojure.lang.PersistentVector/reduce
                  core.clj: 6829  clojure.core/reduce
                  core.clj: 6812  clojure.core/reduce
                  uber.clj:  256  clojure.tools.build.tasks.uber/uber/fn
             protocols.clj:   49  clojure.core.protocols/iter-reduce
             protocols.clj:   75  clojure.core.protocols/fn
             protocols.clj:   75  clojure.core.protocols/fn
             protocols.clj:   13  clojure.core.protocols/fn/G
                  core.clj: 6830  clojure.core/reduce
                  core.clj: 6812  clojure.core/reduce
                  uber.clj:  254  clojure.tools.build.tasks.uber/uber
                  uber.clj:  241  clojure.tools.build.tasks.uber/uber
                  Var.java:  384  clojure.lang.Var/invoke
                   api.clj:  394  clojure.tools.build.api/uber
                   api.clj:  339  clojure.tools.build.api/uber
                      REPL:   21  build/eval25043
                      REPL:   21  build/eval25043
             Compiler.java: 7181  clojure.lang.Compiler/eval
             Compiler.java: 7136  clojure.lang.Compiler/eval
                  core.clj: 3202  clojure.core/eval
                  core.clj: 3198  clojure.core/eval
    interruptible_eval.clj:   87  nrepl.middleware.interruptible-eval/evaluate/fn/fn
                  AFn.java:  152  clojure.lang.AFn/applyToHelper
                  AFn.java:  144  clojure.lang.AFn/applyTo
                  core.clj:  667  clojure.core/apply
                  core.clj: 1977  clojure.core/with-bindings*
                  core.clj: 1977  clojure.core/with-bindings*
               RestFn.java:  425  clojure.lang.RestFn/invoke
    interruptible_eval.clj:   87  nrepl.middleware.interruptible-eval/evaluate/fn
                  main.clj:  437  clojure.main/repl/read-eval-print/fn
                  main.clj:  437  clojure.main/repl/read-eval-print
                  main.clj:  458  clojure.main/repl/fn
                  main.clj:  458  clojure.main/repl
                  main.clj:  368  clojure.main/repl
               RestFn.java:  137  clojure.lang.RestFn/applyTo
                  core.clj:  667  clojure.core/apply
                  core.clj:  662  clojure.core/apply
                regrow.clj:   20  refactor-nrepl.ns.slam.hound.regrow/wrap-clojure-repl/fn
               RestFn.java: 1523  clojure.lang.RestFn/invoke
    interruptible_eval.clj:   84  nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:   56  nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:  152  nrepl.middleware.interruptible-eval/interruptible-eval/fn/fn
                  AFn.java:   22  clojure.lang.AFn/run
               session.clj:  202  nrepl.middleware.session/session-exec/main-loop/fn
               session.clj:  201  nrepl.middleware.session/session-exec/main-loop
                  AFn.java:   22  clojure.lang.AFn/run
               Thread.java:  825  java.lang.Thread/run

I believe this is because the above dependency has no corresponding jar file, and only contains a pom.xml which groups up a set of dependencies.

tools.build.api/copy-dir corrupts binary files if :replace is used

Sun, 10 Oct 2021 01:32:33 +0000

Without a :replace parameter, b/copy-dir faithfully copies over any binary (and text) files in from the source directories.

However, if a :replace parameter is provided, then it corrupts any binary files when copying. It does the replacement properly for any text files in the directories.

Repro:

Create a source dir mysrc. In that dir, place the file
cryogen.png from
https://raw.githubusercontent.com/cryogen-project/cryogen/master/src/leiningen/new/cryogen/img/cryogen.png
Then run (b/copy-dir :src-dirs ["mysrc"] :target-dir "newdir" :replace {"<<name>>" "me"})
The cryogen.png in newdir will be corrupted.

Add git-hash function

Thu, 23 Sep 2021 14:30:27 +0000

I want api to get git hash to find the exact version.
Thank you.

(defn git-hash
  "Shells out to git and returns hash of commits on this branch:
    git rev-parse HEAD

  Options:
    :dir - dir to invoke this command from, by default current directory
    :short? - shorten hash to 7 length characters"
  [{:keys [dir short?] :or {dir "."} :as params}]
  (assert-specs "git-hash" params
                :dir ::specs/path
                :short? ::specs/boolean)
  (-> {:command-args (cond-> ["git" "rev-parse"]
                       short? (conj "--short")
                       :default (conj "HEAD"))
       :dir (.getPath (resolve-path dir))
       :out :capture}
      process
      :out
      str/trim))

tools.build's compile-clj does not allow setting jvm options

Wed, 22 Sep 2021 14:03:32 +0000

I've a project that uses Malli with a custom default registry, a feature which requires the code be compiled with a certain jvm opt set as is detailed in its documentation.

It was suggested to me in this issue that this is currently not supported in tools.build's compile-clj function. I'm posting here in the hopes this can be filed as a feature request so that I can eventually move this project to tools.build.

Thanks.

Add git-branch function

Mon, 20 Sep 2021 22:31:14 +0000

This is useful for deciding whether to issue a SNAPSHOT release or not.

Here's a working implementation if anyone wants it for their project now:

(ns build
  (:require [clojure.string :as str]
            [clojure.tools.build.api :as b]
            [clojure.tools.build.api.specs :as specs]))

(defn git-branch
  "Shells out to git and returns the current branch
    git branch --show-current
  Options:
    :dir - dir to invoke this command from, by default current directory"
  [{:keys [dir] :or {dir "."} :as params}]
  (@#'b/assert-specs "git-branch" params :dir ::specs/path)
  (-> {:command-args ["git" "branch" "--show-current"]
       :dir (.getPath (b/resolve-path dir))
       :out :capture}
    b/process
    :out
    str/trim))

(def lib 'my-group/my-lib)

(def version
  (if (= "master" (git-branch nil))
    (format "1.0.%s" (b/git-count-revs nil))
    (format "1.0.%s-SNAPSHOT" (b/git-count-revs nil))))

Should tools.build functions respect with-dir (from tools.deps.alpha)?

Wed, 08 Sep 2021 22:02:10 +0000

I ran across this issue with depstar but observed that tools.build behaves the same, possibly incorrect, way.

See https://github.com/seancorfield/with-dir-repro

If you wrap (b/create-basis ..) in with-dir from tools.deps.alpha, it calculates the basis from the deps.edn file in the context of the with-dir location.

However, if you use that basis with tools.build functions, they do not behave correctly -- even if they are also wrapped in the same with-dir context.

The dependencies come from the project deps.edn (Clojure 1.9.0 in this example) but the src used is from the root of the repo, not from the project folder.

This is counter-intuitive (and took me a while to track down with depstar -- where I definitely consider it to be a bug).

How to control the transient deps that get compiled/assembled into an uberjar?

Thu, 05 Aug 2021 16:36:31 +0000

I am having trouble understanding how to control the contents of an uberjar using tools.build. In this scenario, the runtime environment (a Spark cluster) provides some deps and thus those deps should not be in the uberjar. Let's say I have 2 classpaths (represented as aliases):

:compile The classpath with all deps/classes that are required for compilation.
:uber The classpath with only the deps/classes that should be in the final jar.

In theory, if you take the :compile classpath and subtract the :uber classpath, we should get the set of classes that are provided by the runtime environment.

Consider the following simplified dependency graph:

We only need the blue deps to be in the jar. The white "provided" deps need to be used during compilation but should not be in the final uberjar.

I looked into using the :filter-nses option of compile-clj but it creates problems with transient deps.

If we filter to only my.project we will lose the important deps of some.other/dep and some.transient/dep.
If we filter to exclude org.apache (not possible with :filter-nses, but in theory it could be done) we would still have org.scala-lang/scala-library hanging around.

I also experimented using different calls to create-basis during compile and jar building. I'm not sure if that makes any sense.

Is it possible to leverage the dependency graph to control either 1) what gets compiled or 2) which classes get put into an uberjar? Any advice would be much appreciated.

getting an error with tools.build uberjar

Tue, 27 Jul 2021 15:48:45 +0000

I'm creating an uberjar with

build=> (b/uber {:class-dir class-dir
                 :uber-file "uberjar.jar"
                 :basis     basis})

This throws an error

build=> (b/uber {:class-dir class-dir
                 :uber-file "uberjar.jar"
                 :basis     basis})
Execution error (FileNotFoundException) at java.io.FileOutputStream/open0 (FileOutputStream.java:-2).
/var/folders/_h/3fpgc8pj4fb2_w3mfqfl59fr0000gn/T/uber8421760143696109740/license/LICENSE (Not a directory)
build=> *e
#error {
 :cause "/var/folders/_h/3fpgc8pj4fb2_w3mfqfl59fr0000gn/T/uber8421760143696109740/license/LICENSE (Not a directory)"
 :via
 [{:type java.io.FileNotFoundException
   :message "/var/folders/_h/3fpgc8pj4fb2_w3mfqfl59fr0000gn/T/uber8421760143696109740/license/LICENSE (Not a directory)"
   :at [java.io.FileOutputStream open0 "FileOutputStream.java" -2]}]
 :trace
 [[java.io.FileOutputStream open0 "FileOutputStream.java" -2]
  [java.io.FileOutputStream open "FileOutputStream.java" 298]
  [java.io.FileOutputStream <init> "FileOutputStream.java" 237]
  [java.io.FileOutputStream <init> "FileOutputStream.java" 187]
  [clojure.tools.build.tasks.uber$explode invokeStatic "uber.clj" 66]
  [clojure.tools.build.tasks.uber$explode invoke "uber.clj" 53]
  [clojure.tools.build.tasks.uber$uber$fn__16290 invoke "uber.clj" 114]
  [clojure.core$run_BANG_$fn__8813 invoke "core.clj" 7717]
  [clojure.core.protocols$naive_seq_reduce invokeStatic "protocols.clj" 62]
  [clojure.core.protocols$interface_or_naive_reduce invokeStatic "protocols.clj" 72]
  [clojure.core.protocols$fn__8181 invokeStatic "protocols.clj" 169]
  [clojure.core.protocols$fn__8181 invoke "protocols.clj" 124]
  [clojure.core.protocols$fn__8136$G__8131__8145 invoke "protocols.clj" 19]
  [clojure.core.protocols$seq_reduce invokeStatic "protocols.clj" 31]
  [clojure.core.protocols$fn__8166 invokeStatic "protocols.clj" 75]
  [clojure.core.protocols$fn__8166 invoke "protocols.clj" 75]
  [clojure.core.protocols$fn__8110$G__8105__8123 invoke "protocols.clj" 13]
  [clojure.core$reduce invokeStatic "core.clj" 6830]
  [clojure.core$run_BANG_ invokeStatic "core.clj" 7712]
  [clojure.core$run_BANG_ invoke "core.clj" 7712]
  [clojure.tools.build.tasks.uber$uber invokeStatic "uber.clj" 114]
  [clojure.tools.build.tasks.uber$uber invoke "uber.clj" 104]
  [clojure.lang.Var invoke "Var.java" 384]
  [clojure.tools.build.api$uber invokeStatic "api.clj" 277]
  [clojure.tools.build.api$uber invoke "api.clj" 266]
  [build$eval16234 invokeStatic "NO_SOURCE_FILE" 11]
  [build$eval16234 invoke "NO_SOURCE_FILE" 11]
  [clojure.lang.Compiler eval "Compiler.java" 7181]
  [clojure.lang.Compiler eval "Compiler.java" 7136]
  [clojure.core$eval invokeStatic "core.clj" 3202]
  [clojure.core$eval invoke "core.clj" 3198]
  [clojure.main$repl$read_eval_print__9110$fn__9113 invoke "main.clj" 437]
  [clojure.main$repl$read_eval_print__9110 invoke "main.clj" 437]
  [clojure.main$repl$fn__9119 invoke "main.clj" 458]
  [clojure.main$repl invokeStatic "main.clj" 458]
  [clojure.core.server$repl invokeStatic "server.clj" 180]
  [clojure.core.server$repl invoke "server.clj" 180]
  [clojure.lang.AFn applyToHelper "AFn.java" 152]
  [clojure.lang.AFn applyTo "AFn.java" 144]
  [clojure.lang.Var applyTo "Var.java" 705]
  [clojure.core$apply invokeStatic "core.clj" 667]
  [clojure.core.server$accept_connection invokeStatic "server.clj" 73]
  [clojure.core.server$start_server$fn__8902$fn__8903$fn__8905 invoke "server.clj" 117]
  [clojure.lang.AFn run "AFn.java" 22]
  [java.lang.Thread run "Thread.java" 829]]}
build=>

This error is actually thrown from two libs on our classpath:
- .m2/repository/xml-apis/xml-apis/1.4.01/xml-apis-1.4.01.jar
- .m2/repository/xml-apis/xml-apis-ext/1.3.04/xml-apis-ext-1.3.04.jar

Each of these has an entry license/LICENSE. The problem seems to arise that I am on osx which uses a case insensitive but case-preserving filesystem and there is already a top-level LICENSE file from another jar. So it cannot create a license/LICENSE entry in addition to a LICENSE file that already exists.

I repod this with the following code that creates a few jars:

(ns build
  (:require [clojure.tools.build.api :as b]))

;; create a jar with just a single namespace and a license/LICENSE file
(b/write-file {:path "target/liba/a.clj"
               :content (symbol "(ns a)")})
(b/write-file {:path "target/liba/license/LICENSE"
               :content (symbol "this is the license from a")})
(b/write-file {:path "target/liba/deps.edn"
               :content '{:paths ["."]}})
(b/write-pom {:basis (b/create-basis {:project "target/liba/deps.edn"})
              :class-dir "target/liba"
              :lib 'liba/liba
              :version "1.0.0"
              :src-dirs ["target/liba"]})
(b/jar {:class-dir "target/liba"
        :jar-file "target/jars/liba.jar"})

;; create another jar libb with just a single ns and a license file
(b/write-file {:path "target/libb/b.clj"
               :content (symbol "(ns b)")})
(b/write-file {:path "target/libb/license"
               :content (symbol "this is the license from b")})
(b/write-file {:path "target/libb/deps.edn"
               :content '{:paths ["."]}})
(b/write-pom {:basis (b/create-basis {:project "target/libb/deps.edn"})
              :class-dir "target/libb"
              :lib 'libb/libb
              :version "1.0.0"
              :src-dirs ["target/libb"]})
(b/jar {:class-dir "target/libb"
        :jar-file "target/jars/libb.jar"})

;; create a consumer of these
(b/write-file {:path "target/libc/license"
               :content (symbol "this is the license from c")})
(b/write-file {:path "target/libc/deps.edn"
               :content '{:deps {liba/liba {:local/root "../jars/liba.jar"}
                                 libb/libb {:local/root "../jars/libb.jar"}}}})

(def old-project-root b/*project-root*)
(b/set-project-root! "target/libc")
#_(b/set-project-root! old-project-root)

(b/uber {:class-dir "."
         :uber-file "uberjar.jar"
         :basis (b/create-basis {:project "deps.edn"})
         :main 'clojure.main})

But the basic error happens when trying to explode the jars and creating a folder license when a file LICENSE already exists.

EDIT
Sean mentioned that depstar has an api-compatible uber function and that has worked for me. Since the contents of the jars never hit the filesystem this constraint is avoided.

tools.build create-basis cannot use alias that includes tools.deps.alpha

Mon, 26 Jul 2021 23:38:47 +0000

If you try to call create-basis with an alias that involves tools.deps.alpha, you get the following exception:

Execution error (ExceptionInfo) at clojure.tools.deps.alpha.extensions.maven/get-artifact (maven.clj:133).
Could not find artifact org.clojure:tools.deps.alpha:jar:${project.version} in central (https://repo1.maven.org/maven2/)

This is because tools.build includes a source version of deps.edn with that string, instead of a version number:

./tools.build/src/main/resources/clojure/tools/deps/deps.edn
    :deps {:extra-deps {org.clojure/tools.deps.alpha {:mvn/version "${project.version}"}}}

Here's the minimal repro: in an otherwise empty directory:

;; deps.edn 
{:aliases
 {:build
  {:deps {io.github.clojure/tools.build {:git/tag "v0.1.6" :git/sha "5636e61"}}
   :ns-default build}}}

;; build.clj 
(ns build
  (:require [clojure.tools.build.api :as b]))

(defn bug [_]
  (b/create-basis {:aliases [:deps]}))

Then run:

clojure -T:build bug

Given that tools.build already explicitly depends on t.d.a as follows, I don't understand why it also needs a copy of t.d.a's deps.edn source in in the resources tree?

org.clojure/tools.deps.alpha {:mvn/version "0.12.1003"}

Removing that file from the checked out tools.build (under ~/.gitlibs/libs) seems to solve the problem and doesn't appear to break anything else...

What does "Manifest type :jar not loaded when finding deps ..." mean?

Sun, 18 Jul 2021 06:16:17 +0000

I have a jar that I created from one of my projects using tools.build. I have included it has a :local/root dep in a different project but get the following error whenever I invoke clj with the jar on the classpath.

Error building classpath. Manifest type :jar not loaded when finding deps for io.github.erp12/clark-scala in coordinate {:local/root "/Users/eddie/Projects/Software/clark/clark-scala/target/clark-scala-2.12.13.1.jar", :deps/manifest :jar, :deps/root "/Users/eddie/Projects/Software/clark/clark-scala/target/clark-scala-2.12.13.1.jar", :parents #{[]}, :paths ["/Users/eddie/Projects/Software/clark/clark-scala/target/clark-scala-2.12.13.1.jar"]}

This error message helps identify which dep (and in this case, which exact jar file) is causing the problem, but I don't know if there is any information in there that could help identify the cause problem. Am I missing the way to interpret this exception? Are there any common causes of this?

Add ability to customize JAR manifest created by clojure.tools.build.api/uber

Fri, 16 Jul 2021 19:53:16 +0000

There are several reasons we need to customize the META-INF/MANIFEST.MF file created by clojure.tools.build.api/uber:

A Java library we use, Liquibase, requires a Liquibase-Package entry in the manifest file to be used inside uberjars
We'd like to exclude Multi-Release: true from the built uberjar even tho some of our libs our multi-release. Log4j is an order of magnitude slower in a multi-release JAR and until we figure out how to fix that issue we've just been creating single-release JARs

With Leiningen you can specify the contents of the manifest by including a :manifest key in project.clj... adding an option like that to tools.build would be extremely helpful.

Exclude existing META-INF/MANIFEST.MF when invoking clojure.tools.build.api/uber

Sat, 10 Jul 2021 19:24:16 +0000

Hello,

I have been trying to build an uberjar using tools.deps and I encountered the following exception:

Execution error (ZipException) at java.util.zip.ZipOutputStream/putNextEntry (ZipOutputStream.java:233).
duplicate entry: META-INF/MANIFEST.MF

This is because one library appears to provides a META-INF/MANIFEST.MF that conflicts with the one clojure.tools.build.api/uber is trying to add in the uberjar.

I guess such file should be ignored by default for example by defining clojure.tools.build.tasks.uber/uber-exclusions like this:

(def ^:private uber-exclusions
  [#"project.clj"
   #"META-INF/.*\.(?:SF|RSA|DSA|MF)"])

I don't have access to Clojure JIRA but If somebody give me access I am willing to provide a trivial patch containing this fix and a test case for the specific MANIFEST.MF exclusion.