Welcome! Please see the About page for a little more info on how this works.

Ask a Question

Cannot resolve Git deps through proxy

+2 votes
in tools.deps by

In environments where the internet is accessed through a proxy, {{clj}} fails to resolve Git dependencies.

`
Cloning: https://github.com/cognitect-labs/test-runner.git
Error building classpath. https://github.com/cognitect-labs/test-runner.git: cannot open git-upload-pack
org.eclipse.jgit.api.errors.TransportException: https://github.com/cognitect-labs/test-runner.git: cannot open git-upload-pack

    at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:248)
    at org.eclipse.jgit.api.CloneCommand.fetch(CloneCommand.java:306)
    at org.eclipse.jgit.api.CloneCommand.call(CloneCommand.java:200)
    at org.eclipse.jgit.api.CloneCommand.call(CloneCommand.java:89)
    at clojure.tools.gitlibs.impl$call_with_auth.invokeStatic(impl.clj:50)
    at clojure.tools.gitlibs.impl$call_with_auth.invoke(impl.clj:41)
    at clojure.tools.gitlibs.impl$git_clone_bare.invokeStatic(impl.clj:71)
    at clojure.tools.gitlibs.impl$git_clone_bare.invoke(impl.clj:68)
    at clojure.tools.gitlibs.impl$ensure_git_dir.invokeStatic(impl.clj:110)
    at clojure.tools.gitlibs.impl$ensure_git_dir.invoke(impl.clj:100)
    at clojure.tools.gitlibs$resolve.invokeStatic(gitlibs.clj:33)
    at clojure.tools.gitlibs$resolve.invoke(gitlibs.clj:29)
    at clojure.tools.gitlibs$procure.invokeStatic(gitlibs.clj:47)
    at clojure.tools.gitlibs$procure.invoke(gitlibs.clj:41)
    at clojure.tools.deps.alpha.extensions.git$eval925$fn__927.invoke(git.clj:41)
    at clojure.lang.MultiFn.invoke(MultiFn.java:239)
    at clojure.tools.deps.alpha$expand_deps.invokeStatic(alpha.clj:189)
    at clojure.tools.deps.alpha$expand_deps.invoke(alpha.clj:172)
    at clojure.tools.deps.alpha$resolve_deps.invokeStatic(alpha.clj:239)
    at clojure.tools.deps.alpha$resolve_deps.invoke(alpha.clj:221)
    at clojure.tools.deps.alpha.script.make_classpath$create_classpath.invokeStatic(make_classpath.clj:59)
    at clojure.tools.deps.alpha.script.make_classpath$create_classpath.invoke(make_classpath.clj:52)
    at clojure.tools.deps.alpha.script.make_classpath$run.invokeStatic(make_classpath.clj:70)
    at clojure.tools.deps.alpha.script.make_classpath$run.invoke(make_classpath.clj:64)
    at clojure.tools.deps.alpha.script.make_classpath$_main.invokeStatic(make_classpath.clj:109)
    at clojure.tools.deps.alpha.script.make_classpath$_main.doInvoke(make_classpath.clj:84)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at clojure.lang.Var.applyTo(Var.java:705)
    at clojure.core$apply.invokeStatic(core.clj:665)
    at clojure.main$main_opt.invokeStatic(main.clj:491)
    at clojure.main$main_opt.invoke(main.clj:487)
    at clojure.main$main.invokeStatic(main.clj:598)
    at clojure.main$main.doInvoke(main.clj:561)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at clojure.lang.Var.applyTo(Var.java:705)
    at clojure.main.main(main.java:37)

Caused by: org.eclipse.jgit.errors.TransportException: https://github.com/cognitect-labs/test-runner.git: cannot open git-upload-pack

    at org.eclipse.jgit.transport.TransportHttp.connect(TransportHttp.java:592)
    at org.eclipse.jgit.transport.TransportHttp.openFetch(TransportHttp.java:345)
    at org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:137)
    at org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:123)
    at org.eclipse.jgit.transport.Transport.fetch(Transport.java:1269)
    at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:237)
    ... 35 more

Caused by: java.net.ConnectException: Connection refused (Connection refused) github.com

    at org.eclipse.jgit.util.HttpSupport.response(HttpSupport.java:213)
    at org.eclipse.jgit.transport.TransportHttp.connect(TransportHttp.java:506)
    ... 40 more

`

In principle, tools.gitlibs already supports (via JGit) configuring a proxy server using the standard JVM (link: https://docs.oracle.com/javase/8/docs/api/java/net/doc-files/net-properties.html#Proxies text: system properties). Just passing {{-Dhttps.proxyHost=proxy.my.org -Dhttps.proxyPort=3128}} would fix this for most users. However, there is currently no way of passing these flags to the {{make-classpath}} invocation in the {{clojure}} script. {{-J}} flags are only passed to the final {{java}} invocation, not to the {{make-classpath}} invocation where dependencies are actually resolved.

So one straightforward solution would be to allow passing additional flags to the {{make-classpath}} invocation in the {{clojure}} script.

4 Answers

+1 vote
by
_Comment made by: glts_

Hiya [~sundbp], if you modify [this line|https://github.com/clojure/brew-install/blob/1.10.0.442/src/main/resources/clojure#L301] in the {{clojure}} script to include the JVM flags {{-Dhttps.proxyHost=proxy.my.org -Dhttps.proxyPort=3128}}, Git resolution will succeed. I have verified this in my proxied work environment.

A fourth option was discussed in TDEPS-20, namely relying on the somewhat conventional environment variables {{http_proxy}} and friends. That was my preference. I’m not going to work on this, so anyone feel free to take it on.
0 votes
by

Comment made by: alexmiller

I was just wondering about this yesterday... :)

I would really like to trigger from the same Maven system properties so there's one place to set things.

by
Would be great to fix this. I just started work at a new place, and stumbled on this issue by accident. I upgraded the clj tools, and unwittingly broke the build. I didn't realise that the clj tools had been modified to work with a proxy as described in one of the other answers, and I overwrote it.
0 votes
by

Comment made by: sundbp

I've got an interest in this issue sitting behind a proxy.

As I see it we have 3 potential sources to pickup proxy config from:
1. JVM system properties, fitting in with how the JVM "typically" config proxies
2. Maven system properties, fitting in with a proxy setting a user is likely to need/have set for using clj
3. The git CLI tools' proxy settings managed via git config.

2 has the benefit of being a setting already mention in the docs and so forth. Having one instead of 2 separate settings makes for a slightly smoother ride. The drawback is that there is no obvious logic tying JGit settings to mvn.

1 and 3 are both relatively logical settings in relation to JGit, the first focusing on the JVM way of doing things (although it isn't like the JVM is 100% standardized). The 2nd focusing on the git aspect (which is clear and well defined). Both options are also settings that the user may already be using.

Overall I'm sort of leaning towards option 1, especially if David is correct that this is already supported and set up in JGit. However, it's not obvious to me it is from a few quick searches of their git repo. There is a CLI tool based on JGit part of the repo, which does have code to setup proxies from the JVM system properties - but I don't see it part of the straight library and we'd need to add the code to gitlibs. If you have pointers as to why this is wrong I'm super interested @David!

Opinions around the options?

0 votes
by
Maintained by Cognitect on behalf of the Clojure community
...