classpath "empty" when classpath 0.2.2 used with clojure maven plugin 1.3.20+

Question

I have a project that is storing data in resources that it expects to find on the classpath. When the project uses java.classpath 0.2.2 and clojure-maven-plugin 1.3.20+ calling (java.classpath/classpath) produces unexpected results - it shows some sun stuff, but no dependencies, no clojure, none of my code or resources. Clojure seems to be functioning though, it's just calls in to the classpath api that pose issues.

For comparison:

java.classpath 0.1.0 and c-m-plugin 1.3.9 behaves as expected.

java.classpath 0.1.0 and c-m-plugin 1.3.20 can create problems and sometimes only shows one item on the classpath that appears to be an empty manifest.

crosspost to clojure-maven-plugin here:
https://github.com/talios/clojure-maven-plugin/issues/81

Answer 1

Comment made by: drlivingston

also with this java version
Java version: 1.7.0_45, vendor: Oracle Corporation

Answer 2

Comment made by: stuart.sierra

I can confirm this behavior with all versions of java.classpath on all versions of clojure-maven-plugin 1.3.13 and later.

Starting with version 1.3.13 of clojure-maven-plugin, instead of specifying the Java classpath on the command line with {{-cp}}, the plugin generates a temporary JAR file with a manifest file containing the real classpath and launches Java with {{-jar}}.

See clojure-maven-plugin (link: https://github.com/talios/clojure-maven-plugin/pull/58 text: pull request #58) containing (link: https://github.com/talios/clojure-maven-plugin/commit/8d8e90e41806a6927c11347e6fc13344ba53c887 text: commit 8d8e90e4).

{{clojure.java.classpath/classpath}} looks at the classloader that loaded Clojure, which in this case is a child classloader of {{sun.misc.Launcher$AppClassLoader}}. AppClassLoader is a subclass of (link: http://docs.oracle.com/javase/6/docs/api/java/net/URLClassLoader.html text: URLClassLoader) but its (link: http://docs.oracle.com/javase/6/docs/api/java/net/URLClassLoader.html#getURLs() text: getURLs) method returns only the URL of the JAR file. There is no obvious API for getting the {{Class-Path}} property of the manifest file without opening and reading the contents of the JAR file.

Answer 3

Comment made by: stuart.sierra

On further digging in the OpenJDK source code, URLClassLoader reads JAR manifest files lazily, as needed, to load classes or resources. To find the list of all URLs that a URLClassLoader might open, you would have to reimplement this logic to read all the manifest files. It also has to be recursive (JAR manifest references another JAR with a manifest) and detect cycles.

Answer 4

Comment made by: drlivingston

Thank you for looking into this.
Am I understanding that this is the correct / desired behavior for what clojure-maven-plungin and clojure.java.classpath should be doing together?

So I was going into jar files, (although I've realized I don't think I'm looking into nested jar files - does the classloader look into nested files?) and now you are saying there's a third case I will have to cover too, right?: manifest files? Is there a utility method in clojure.java.classpath for reading them?

I was using code like this to identify everything accessible on the classpath:

https://github.com/drlivingston/kr/blob/master/kr-core/src/main/clojure/edu/ucdenver/ccp/utils.clj#L101

thanks,
Kevin

Answer 5

Comment made by: stuart.sierra

Currently, clojure.java.classpath handles neither JAR Manifest files nor nested JAR files. As I described in my previous comment, implementing support for this would be difficult and is not likely to be a priority for me any time soon. I would be willing to review a patch to add this capability, but it would have to prove that it protects against recursive cycles.

By my reading of the (link: http://docs.oracle.com/javase/7/docs/technotes/guides/jar/jar.html text: JAR File Specification), the behavior of clojure-maven-plugin could be considered against the spirit of the JAR spec, which decribes the "Class-Path" manifest attribute this way:

{quote}
"Class-Path: The value of this attribute specifies the relative URLs of the extensions or libraries that this application or extension needs." (emphasis added)
{quote}

The clojure-maven-plugin is creating a JAR manifest with a "Class-Path" attribute containing absolute URLs to other JAR files which are not embedded in the JAR containing the manifest.

I have always assumed that JAR manifest files with Class-Path attributes are used only when packaging complete applications or JDK extensions, not as a mechanism for specifying the class-path during development. Clearly it is possible, as the current clojure-maven-plugin demonstrates, but it is not a feature I would want to rely on.

In general, the JDK does not support enumerating all files on the classpath, since the classpath can include URLs to remote resources which are downloaded as needed.

Answer 6

Comment made by: drlivingston

Interesting. I didn't realize some of this was so flexible.
Thanks for your comments.

So iterating recursively over the CP seem like it might be a bad thing to do?, although I guess it's no worse than if you happened to ask for something to be loaded that's found in the very last, deepest corner of the CP, right?

I have been putting resource files on the CP and then having the application find them with a prefix of their names. e.g., find all "foo.bar" and I get "foo.bar.baz1" and "foo.bar.baz2" etc. without the application needing to be aware of what's there or the need for manifests to be generated or maintained. Is this an known anti-pattern for working with resources on the classpath?

Answer 7

Comment made by: stuart.sierra

{quote}
Is this an known anti-pattern for working with resources on the classpath?
{quote}

Yes.

Answer 8

Reference: https://clojure.atlassian.net/browse/CLASSPATH-6 (reported by alex+import)