Improve error message for tools.deps when m2 permissions do not allow write

Question

I've encountered this error and it took a while to figure out.

For some reason downloading eclipse artifacts was failing.
It was very frustrating.
I found the issue - to be caused by wrong permissions for directories under $HOME/.m2/repository.

Downloading deps failed to write any files to local Maven repo and that triggered a fail in deps resolving for my project.
The error was related to FileNotFound - and not "I can't write to this directory" .
It was more confusing since I had some dependencies in that directory (probably from an previous download).

I suspect the directory permissions where changed by mounting the local maven repo inside a docker container.

I reported this on slack and created this question as suggested by Alex Miller.
He mentioned this might be caused by maven libraries and out of control of tools.deps.

IMO I find it strange that maven libs don't error out when trying to create the local file and we get an exception when trying to read the file by tools.deps.

Steps to reproduce:

rm -rf ~/.m2/repository/org/eclipse 
mkdir -p ~/.m2/repository/org/eclipse/
chown root:root ~/.m2/repository/org/eclipse
clojure -Sdeps '{:deps {info.sunng/ring-jetty9-adapter {:mvn/version "0.22.0"}}}'

Exception

clojure -Sdeps '{:deps {info.sunng/ring-jetty9-adapter {:mvn/version "0.22.0"}}}'
Warning: failed to load the S3TransporterFactory class
Warning: failed to load the S3TransporterFactory class
Warning: failed to load the S3TransporterFactory class
Warning: failed to load the S3TransporterFactory class
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Warning: failed to load the S3TransporterFactory class
Warning: failed to load the S3TransporterFactory class
Warning: failed to load the S3TransporterFactory class
Warning: failed to load the S3TransporterFactory class
Error building classpath. Failed to read artifact descriptor for org.eclipse.jetty.websocket:websocket-servlet:jar:11.0.15
org.eclipse.aether.resolution.ArtifactDescriptorException: Failed to read artifact descriptor for org.eclipse.jetty.websocket:websocket-servlet:jar:11.0.15

    at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(Unknown Source)
    at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor(Unknown Source)
    at org.eclipse.aether.internal.impl.DefaultRepositorySystem.readArtifactDescriptor(Unknown Source)
    at clojure.tools.deps.extensions.maven$read_descriptor.invokeStatic(maven.clj:115)
    at clojure.tools.deps.extensions.maven$read_descriptor.invoke(maven.clj:106)
    at clojure.tools.deps.extensions.maven$fn__825.invokeStatic(maven.clj:146)
    at clojure.tools.deps.extensions.maven$fn__825.invoke(maven.clj:143)
    at clojure.lang.MultiFn.invoke(MultiFn.java:244)
    at clojure.tools.deps$expand_deps$children_task__463$fn__465$fn__466.invoke(deps.clj:407)
    at clojure.lang.AFn.applyToHelper(AFn.java:152)
    at clojure.lang.AFn.applyTo(AFn.java:144)
    at clojure.core$apply.invokeStatic(core.clj:667)
    at clojure.core$with_bindings_STAR_.invokeStatic(core.clj:1990)
    at clojure.core$with_bindings_STAR_.doInvoke(core.clj:1990)
    at clojure.lang.RestFn.invoke(RestFn.java:425)
    at clojure.lang.AFn.applyToHelper(AFn.java:156)
    at clojure.lang.RestFn.applyTo(RestFn.java:132)
    at clojure.core$apply.invokeStatic(core.clj:671)
    at clojure.core$bound_fn_STAR_$fn__7194.doInvoke(core.clj:2020)
    at clojure.lang.RestFn.invoke(RestFn.java:397)
    at clojure.lang.AFn.call(AFn.java:18)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
    at java.base/java.lang.Thread.run(Thread.java:1583)

Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact org.eclipse.jetty.websocket:websocket-servlet:pom:11.0.15 from/to central (https://repo1.maven.org/maven2/): /home/ieugen/.m2/repository/org/eclipse/jetty/websocket/websocket-servlet/11.0.15/websocket-servlet-11.0.15.pom.part.lock (No such file or directory)

    at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(Unknown Source)
    at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(Unknown Source)
    at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifact(Unknown Source)
    ... 25 more

Caused by: org.eclipse.aether.transfer.ArtifactTransferException: Could not transfer artifact org.eclipse.jetty.websocket:websocket-servlet:pom:11.0.15 from/to central (https://repo1.maven.org/maven2/): /home/ieugen/.m2/repository/org/eclipse/jetty/websocket/websocket-servlet/11.0.15/websocket-servlet-11.0.15.pom.part.lock (No such file or directory)

    at org.eclipse.aether.connector.basic.ArtifactTransportListener.transferFailed(Unknown Source)
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run(Unknown Source)
    at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run(Unknown Source)
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector$DirectExecutor.execute(Unknown Source)
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector.get(Unknown Source)
    at org.eclipse.aether.internal.impl.DefaultArtifactResolver.performDownloads(Unknown Source)
    ... 28 more

Caused by: java.io.FileNotFoundException: /home/ieugen/.m2/repository/org/eclipse/jetty/websocket/websocket-servlet/11.0.15/websocket-servlet-11.0.15.pom.part.lock (No such file or directory)

Answer 1

This is a common issue when you mount ~/.m2 and friends into Docker using a volume in order to avoid downloading artifacts all over again. If something gets downloaded inside docker anyhow, that file might be written as root into the users home directory. I also run into this at least a couple of times over the years. Last time last week. :slightly_smiling_face: Not a Docker expert either, and I may be doing something wrong with it. When this happens it is not obvious what happened, and you are luck when you remember that you have been there.

Comments

We also had this problem in the past. What we've been doing for years now is to have a "run-as-user.sh" script with the following content:

```
#!/usr/bin/env bash

set -eu

NEW_UID=$(stat -c '%u' /app)
NEW_GID=$(stat -c '%g' /app)

groupmod -g "$NEW_GID" -o hop >/dev/null 2>&1
usermod -u "$NEW_UID" -o hop >/dev/null 2>&1

exec chpst -u hop:hop -U hop:hop env HOME="/home/hop" "$@"
```

that we use as the entry point for the container. Then in the Docker file for the container we also add the following lines (in additon to whatever is needed for that container), and the command required to install the `runit` package (which is the one that provides the `chpst` command) in the distribution used by your Docker image:

```
RUN useradd --home-dir /home/hop --create-home --shell /bin/bash --user-group hop
COPY run-as-user.sh /usr/local/bin
WORKDIR /app
ENTRYPOINT ["run-as-user.sh"]
```

And when running the container, we mount whatever directories are needed (~/.m2, the Clojure app directory, etc) using volumes inside the container. We mount the Clojure application directory under `/app` inside the container (which is where the `run-as-user.sh` script will look). Then the `run-as-user.sh` script takes care of changing the UID and GID of the account used inside the container, to match those of the user owning the `/app` directory (which is the outside user).

Hope that helps.