Welcome! Please see the About page for a little more info on how this works.

+1 vote
in Clojure CLI by

Hi,

I'm getting a PKIX certificate exception when trying to download project dependnecies with the clojure tool behind a strict firewall on MS-Windows:

Could not transfer artifact ... from/to central
(https://repo1.maven.org/maven2/):
sun.security.validator.ValidatorException: PKIX path building failed:
sun.security.provider.certpath.SunCertPathBuilderException: unable
to find valid certification path to requested target

I believe this is due to self-signed root certificate utilized by the Firewall to control & monitor the traffic.

Analysis follows and apologies for the long read.


It is a standard practice in big organisations for workstations to sit behind a firewall that controls and monitors all traffic to and from the Internet. It is also common for the company to create their own self-signed root certificate and replace with it the certificates found in the https traffic so as to decrypt and analyze the data flows, utilizing in effect the man-in-the-middle attack to their advantage.

For this to work, the user workstation where the connection originates from needs to have the self-signed certificates installed in their trusted certificate store. On Windows, this will be the trusted root authority certification keystore.

Java comes with its own certificate store, the java KeyStore, which is separate from the general keystore available on the workstation. Thus, any java https connections to the Internet through the firewall is most likely to fail with an unknown certificate error, since the self-signed certs are not installed on the java keystore.

This adversely affects the clojure/clj command line tools when downloading libs because it can't find the self-signed cert in the java keystore. For example, an exception is thrown when trying to access the Internet from such computers:

 > clojure -P
 
 Error building classpath ... 
 org.eclipse.aether.resolution.ArtifactDescriptorException:   Failed to
 read artifact descriptor for ....  ...  Caused by:
 org.eclipse.aether.resolution.ArtifactResolutionException:   Could not
 transfer artifact ... from/to central
 (https://repo1.maven.org/maven2/):   
 sun.security.validator.ValidatorException: PKIX path building failed:
      sun.security.provider.certpath.SunCertPathBuilderException:
        unable to find valid certification path to requested target  ...
  ...

I can think of at least two solution to circumvent this problem:
1. Install the missing certificates in the java keystore, or
2. Instruct the java instance to look into the general keystore.

There does not seem to be a straightforward way to discover the missing certificates and would probably require some user effort to install them into the java keystore using the java keytool (See Working with Certificates and SSL at https://docs.oracle.com/cd/E19830-01/819-4712/ablqw/index.html).

Instead, it seems easier to instruct java to look at the general or alternative keystores using the java.next.ssl.* properties (https://stackoverflow.com/questions/5871279/ssl-and-cert-keystore).

On Windows, which I assume most of users with such restrictive firewalls are sitting on, it is a just matter of setting a simple java property to instruct java to use the general windows keystore (https://stackoverflow.com/questions/41257366/import-windows-certificates-to-java):

javax.net.ssl.trustStoreType=Windows-ROOT

Given this, I was expecting the following to work on windows

clojure -J-D'javax.net.ssl.trustStoreType=Windows-ROOT' -P

but it doesn't. It turns out the $JvmOpts variable is not passed onto the prep command by the powershell script here https://github.com/clojure/brew-install/blob/b91fb78e321b5e39bedda594f5d578579d448d19/src/main/resources/clojure/install/ClojureTools.psm1#L388 :

& $JavaCmd -classpath $ToolsCp clojure.main -m clojure.tools.deps.alpha.script.make-classpath2 --config-user $ConfigUser --config-project $ConfigProject --basis-file $BasisFile --libs-file $LibsFile --cp-file $CpFile --jvm-file $JvmFile --main-file $MainFile --manifest-file $ManifestFile @ToolsArgs

Some options I can think of to utilise java properties to solve the general issue, from most intrusive/specific to the general case:

  1. Have the clojure.tools.deps.alpha.script.make-classpath2 fn which is responsible for downloading the dependencies set javax.net.ssl.trustStoreType=Windows-ROOT at runtime, if running on Windows,
    • .e.g. (System/setProperty "javax.net.ssl.trustStoreType" "Windows-ROOT").
    • cons:
      • This only works on windows.
      • forces to use of the general keystore, which might not be what the user always want.
        • This can be mitigated by having a new script option (say -W) that is passed onto clojure.tools.deps.alpha as an option. The drawback is that users/tooling has to always pass this flag on invocation.
      • setting the javax.net.ssl.tustStoreType property at runtime with System/setProperty could have no effect if any SSL connection was previously made from other parts of the code.
  2. Same as previous, but with a new script option (say -Sssl EDN) whose EDN are the javax.net.ssl.* properties, and will be passed as such to clojure.tools.deps.alpha for setting at runtime.
    • e.g. clojure -Sssl {:trustStoreType "Windows-ROOT"} -P
    • cons:
      • Same disadvantages as in #1, and moreover is more cumbersome for the user to type an edn map each time it invokes clojure.
  3. Invent a new deps.edn key (e.g. :clojure.tools.deps.alpha/ssl) whose pairs shall be java.net.ssl.* properties, that can be set in the user config deps.edn and parsed by clojure.tools.deps.alpha s make-classpath2 fn setting the properties using System/setPropertyat runtime.
    • e.g. example clj user config {:clojure.tools.deps.alpha/ssl {:trustStore "Windows-ROOT"}}
    • cons
      • setting the javax.net.ssl.* properties at runtime with System/setProperty might have no effect if any SSL connection was previously made from other parts of the code.
  4. Update script so that -J jvm options are passed onto the java invocation of the -P command (I have tested this to work).
    • e.g. clojure -J-D'javax.net.ssl.trustStoreType=Windows-ROOT' -P
    • cons:
      • User/tooling still have to provide the -J-Djavax.net.ssl.* options at each clojure invocation.
  5. Same as previous, but introduce a CLJ_JVMOPTS environment variable whose value is inserted in the script's $JvmOpts variable.
    • e.g. set CLJ_JVMOPTS=-D'javax.net.ssl.trustStoreType=Windows-ROOT', and then clojure -P

I'm more in favor of #5, since this does not seem to have any disadvantages and can be set once and be applied on all clojure invocations, creating a better experience for the user/tooling.

Let me know your thoughts. Happy to look after the changes in the scripts and/or tools.deps.alpha lib.

Thanks

PS: I have also tried to set the properties in the MAVEN_OPTS variable as pe r https://maven.apache.org/guides/mini/guide-repository-ssl.html, but it had no effect. I suspect this only take effect when invoking the mvn command line tool directly.

1 Answer

+1 vote
by

I think this is all correct analysis and we've run into a few cases where jvm properties may need to be set on the java call that generates the classpath, which as you've detected cannot currently be set.

We do have an existing ticket for this at https://clojure.atlassian.net/browse/TDEPS-165, but I haven't worked on it yet. I suspect the end solution will be to open up a new clj option or env var to allow passing jvm properties to the classpath construction java call.

by
It looks good with the caveat that an unconventional value might adversely affect the java invocation as mentioned in my earlier comment. It works well for my test case.

May I please enquire the purpose of the second new JAVA_OPTS env variable, and why the java invocations are split between this and CLJ_JVM_OPTS? I was expecting a single CLJ_JVM_OPTS env var passed to all java invocations.

https://github.com/clojure/brew-install/commit/7914954030ca21f7b928b6f064ace086efdc9057

Thanks
by
Ok, this is now released as 1.11.1.1165.

There are two properties because there are two contexts with (potentially) different needs and lots of cases where you might want a property on one but not the other.
by
Thanks!

And here how this example use case can be solved with the new environment variables, in case anyone else is interested.

To use the Windows certificate store when downloading project dependencies by setting the env variable in the PowerShell prompt:

$env:CLJ_JVM_OPTS = "-Djavax.net.ssl.trustStoreType=Windows-ROOT"

To use the Windows certificate store when running the project via the Clojure Tools, by setting the environment in the PowerShell prompt:

$env:JAVA_OPTS = "-Djavax.net.ssl.trustStoreType=Windows-ROOT"
by
Hi,

Hope I may jump in on this thread. Similar issue here. Corporate firewall that does SSL inspection and replaces SSL certificates (ZScaler). My easiest option is to install the certificates to the CA store.

"%JAVA_HOME%\bin\keytool.exe" -import -trustcacerts -cacerts -storepass XXX -file somefile.cer

And I can validate that it works (using https://github.com/MichalHecko/SSLPoke ):
> java.exe SSLPoke download.clojure.org 443
Successfully connected

But when installing clojure dependencies I get the same error ( PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target ) as the OP.

I don't think I need to use Windows-ROOT trust store. I need just just the default Java CA store ( at $JAVA_HOME/lib/security/cacerts ) and this should work, but it seems Clojure isn't detecting this.

Any advice on how I can debug this?

Thanks
by
I think this stackoverflow answer https://stackoverflow.com/a/5871352/7671 has a good overview of the properties that may need to be set, and you can set them via CLJ_JVM_OPTS.
...