Welcome! Please see the About page for a little more info on how this works.

+1 vote
in tools.deps by

When the classpath needed to execute {{clojure}} gets sufficiently large on Windows, it can exceed the maximum size of a single command-line argument, resulting in an error {{Program 'java.exe' failed to run: The filename or extension is too long}}.

There is some background on this issue in https://github.com/ajoberstar/ClojureTools/issues/2 and the other tickets linked from there. This turns out to be a fairly common problem in JVM-based development under Windows, eg in (link: https://github.com/bazelbuild/bazel/issues/2242 text: Bazel) and (link: https://tuhrig.de/gradles-bootrun-and-windows-command-length-limit/ text: gradle).

The usual solution is to create something known as a "pathing jar" - a jar file which has a {{Class-Path:}} entry in its MANIFEST.MD and no other content. Java can then be invoked via {{java ... -classpath pathing.jar clojure.main}}.

It recently occurred to me is that the pathing jar could be seen as the same information that's currently in the \~/.cpcache/<hash>.cp file, but formatted as a jar file instead of a text file. Therefore I propose as a solution to this that if the classpath is too long on Windows, the {{make-classpath}} Clojure code should produce a file ~/.cpcache/<hash>.jar containing this information. On the PowerShell side of things, the main {{clojure}} function can then check for the existence of this file and invoke java -cp \~/.cpcache/<hash>.jar clojure.main if it exists (vs the current behavior of using the contents of <hash>.cp as its argument).

I think I should be able to come up with a patch for this. I've messed around with this some on the Powershell side, details of which can be seen in the pull request linked from the ClojureTools ticket above, and the code to generate the actual jar file is pretty simple - a bit of string transformation and a call to {{jar cfm path.jar manifest.txt}}, which might not be necessary from the JVM side.

8 Answers

0 votes
by

Comment made by: timgilbert

Ok, so {{0001-Adding-cp-jar-argument.patch}} adds a command-line flag {{--cp-jar}} to the {{make-classpath}} bit of tools.deps. When this file is present, tools.deps will produce a pathing jar as described above, by converting the classpath to be a space-separated list of {{file://}} URLs, bunging that into a jar manifest, and then creating a jar file containing only the manifest. This code wound up being simpler than I expected it to be.

This will also need another bit of code to get it to work correctly - the scripts in {{brew-install}} or the Windows equivalent will need to pass this flag to the make-classpath script when they build the path, and then call {{java -cp .cpcache/1864468523.jar clojure.main}} to actually launch clojure.

I've tested this code from Windows in powershell and from WSL (ie, Linux) by calling {{(write-pathing-jar "path.jar" (slurp cached-cp-filename))}} from the repl. After that I was able to execute Clojure via {{java -cp path.jar clojure.main}} to launch a REPL, and then {{(require 'clojure.tools.deps.alpha.script.make-classpath)}} from inside that REPL to validate that I can require local namespaces.

0 votes
by

Comment made by: timgilbert

I've got a PR open to aoberstar's ClojureTools repo which implements the PowerShell side of this, and the combination of that patch and this one fixes the classpath problem for me:

https://github.com/ajoberstar/ClojureTools/pull/5

0 votes
by

Comment made by: alexmiller

Everything in ajoberstar's repo has been applied to the main repo (plus work beyond), so it would be best to build a patch on https://github.com/clojure/brew-install/tree/1.10.0

0 votes
by

Comment made by: timgilbert

Oh, cool. I'll generate another patch against that and attach it to this ticket.

0 votes
by

Comment made by: alexmiller

BTW, if you are in a github PR, just add .patch to the url to get a patch file.

0 votes
by

Comment made by: timgilbert

Ok, {{0002-Calling-cp-jar-to-create-pathing-jar.patch}} up there is the patch for brew-install, also on github as https://github.com/clojure/brew-install/pull/4.

0 votes
by

Comment made by: timgilbert

Oops! This newer 0002 patch fixes a syntax error around {{$ForceCP}} in the previous brew-install patch.

0 votes
by
Reference: https://clojure.atlassian.net/browse/TDEPS-120 (reported by timgilbert)
...