(re-)building without internet access (with nix)

Question

When building something that depends on tools.build, I expect to be able to build once, which fetches all the required dependencies, and then build again without internet access.

In particular, I'd like to be able to fetch all those dependencies separately and then build from inside a nix build.

This works in clojure 1.10.3.855, but breaks in 1.10.3.933, and I'm not sure why.

You can find a reproducible repo at https://github.com/radvendii/clojure-tools.build-test, with instructions in the readme.

Answer 1

With Clojure CLI 1.10.3.1087 (latest), I can't reproduce this. If I build with the instructions with the repo, then cut net access, I can keep rebuilding the uberjar successfully.

Comments

I note that 933 changed the installer to deal with installing the built-in tools.edn tool installation, maybe the nix build needs to be changed to account for that? I don't think there have been any other installer changes since then.

---

We are not yet sure why the reproduction seems to fails sometimes, but we have played around with the reproducing case a bunch and noticed the following:
When we remove the :tag item from the one dependency in deps.edn, we can successfully build offline.

Our reading of the code is:
- If that item is present, t.d.a calls gitlibs/tags. (https://github.com/clojure/tools.deps.alpha/blob/master/src/main/clojure/clojure/tools/deps/alpha/extensions/git.clj#L64)
- The impl of this calls git fetch unconditionally. (https://github.com/clojure/tools.gitlibs/blob/master/src/main/clojure/clojure/tools/gitlibs/impl.clj#L179)
- This in turn always hits the network.

For the time being, we can work around this by leaving the tag out. But it's still a bit mystifying that the issue fails to reproduce on MacOS. Could it be related to the git installation of the OS?

---

For reference, how exactly did you try to reproduce this? MacOS doesn't have unshare AFAIK, so I suppose you have some other way to cut network access.

---

I just turned off my wifi to test. The reason it didn't fail is because the classpath info was cached in .cpcache. Using -Sforce was sufficient to cause it to repro. I've filed a ticket at https://clojure.atlassian.net/browse/TDEPS-223

---

FYI: things are now much better for us since we removed those tags from our direct deps. Luckily, we don't seem to have such issues in our current transitive deps. But it could very much happen so an offline mode would be very welcome indeed :)