Welcome! Please see the About page for a little more info on how this works.

+1 vote
in tools.deps by

There are many issues with using JGit+JSch currently:

https with authentication is not supported yet which is a very common transport used by private repos. Git (the mainline binary) uses a "credential helper" as an authentication oracle.
ssh authentication via JSch has low usability, some modern KEX methods unsupported
Esoteric .ssh/config options can break cloning because the underlying java ssh library (JSch) misreads the .ssh/config file
ed25519 identity keys supported via ssh-agent, but not via ~/.ssh/config entries. JSch gets confused when there is an agent & ssh/config
The terminal is not interactive in clojure, so you cannot accept unseen public host keys - this is probably ok, but the user might not know what to do when it fails.

Porting gitlibs to shell out to git would help all of these issues, if remains compatible with Windows support. (This is what python and Go do, shell out.) It would also alleviate many transient issues that users report that do not have associated tickets.


Proposed Patch (API Compatible with tools.gitlibs)

6 Answers

+1 vote
_Comment made by: mfikes_

Another reason to ditch JGit+JSch, if not yet already captured: JSch can't read the new OpenSSH key formats, which are now written by default on macOS.

Here is an example:

$ ssh-keygen -f /tmp/foo

This creates a key with a header:


If instead you do

$ ssh-keygen -m PEM -f /tmp/bar

you will get a key with the usual header (and key format):


To illustrate that the latest JSch cannot read this new key type, note the {{invalid privatekey}} error when attempting to read {{"/tmp/foo"}}:

$ clj -Sdeps '{:deps {com.jcraft/jsch {:mvn/version "0.1.55"}}}'
Clojure 1.10.0
user=> (import (com.jcraft.jsch JSch KeyPair))
user=> (KeyPair/load (JSch.) "/tmp/foo")
Execution error (JSchException) at com.jcraft.jsch.KeyPair/load (KeyPair.java:664).
invalid privatekey: [B@485e36bc
user=> (KeyPair/load (JSch.) "/tmp/bar")
#object[com.jcraft.jsch.KeyPairRSA 0x38f116f6 "com.jcraft.jsch.KeyPairRSA@38f116f6"]
+1 vote
_Comment made by: ryan_

There may be a related issue with macOS 10.15 as well. I'm running 10.15 beta 2, and tools.deps can't access my private `:git/url` libs.

I have an `id_rsa` generated under macOS 10.12 that can successfully pull private git deps under macOS 10.14 / of the clojure cli.

When I use that one with macOS 10.15b2, any `clojure` or `clj` command will die with an error building the classpath:

`org.eclipse.jgit.api.errors.TransportException: git@github.com:opengb/spork.git: USERAUTH fail`

If I use an `id_rsa` generated under 10.15, the error is:

`org.eclipse.jgit.api.errors.TransportException: git@github.com:opengb/spork.git: invalid privatekey: [B@3d3a1903`

I'm guessing this is going to start biting more of us Mac types come September and 10.15 release. My workaround is to just manually arrange `~/.gitlibs` to match my old 10.14 machine and pull / package in there with a bash script.
+1 vote

Recent versions of JGit no longer use JSch and have moved to Apache MINA sshd. The new key formats etc should now be supported.


0 votes

Comment made by: colinfleming

One thing that I would like to be taken into account if we go down this route is the need for a progress monitor for tools such as Cursive (see TDEPS-81). I have no idea if this is possible when shelling out to git - I believe JGit does provide something that could be hooked into.

0 votes

Comment made by: ryan

Oh, also the keys have passphrases, so I'm using an ssh-agent. ssh-add -l looks good and SSH_AUTH_SOCK is getting set in env.

0 votes
Reference: https://clojure.atlassian.net/browse/TDEPS-91 (reported by gshayban)