Welcome! Please see the About page for a little more info on how this works.

0 votes
in Clojure by
edited by

I am having trouble with Clojure on the J9 JVM.

I'm interested in J9 in hopes Clojure won't limit one's choice of virtual machine.

The trouble happens in Windows and Linux, and both the "11" and "15" versions of the J9 JVM, and numerous (all?) Clojure versions including 1.10.1, 1.10.2, 1.7, and 1.5.1.

The trouble is an Exception or Error, of one kind or another, usually originating in clojure.core; frequently it is an absurd NullPointerException in reduce1 that is also thrown by Leiningen on the J9 JVM.

The problem is erratic. But something reliably goes wrong if J9 "always compiles" (-Xcomp) and it seems to work fine when J9 "always interprets" (-Xint). The clojure commands given in this report, for some reason, don't seem to honor the -J option so I modified the clojure script to pass along the necessary JVM options for this experiment.

I installed clojure with

./linux-install-1.10.2.774.sh --prefix ${HOME}/clo

I got AdoptOpenJDK, version 11, J9 JVM, for Linux, from adoptopenjdk.org.

On the first run of clojure

${HOME}/clo/bin/clojure

it threw NullPointerException. More about this below.

Trying various JVM options, I found that the -Xint JVM option allowed clojure to download various things and display the REPL prompt. According to java -X this option means "interpreted mode execution only". To use -Xint, I modified clojure to pass along the JVM option. (The -J option is honored by not all of clojure's invocations of Java.)

After the first successful run, subsequent runs of clojure usually succeeded in producing the REPL prompt.

However, the -Xcomp JVM option makes the error happen every time. It means "forces compilation of methods on first invocation".

I first ran into a J9 problem when trying to use the Windows version of Leiningen 2.9.5. Some of my discovery path is in a Github issue there: https://github.com/technomancy/leiningen/issues/2725

The NullPointerException stack trace was:

Exception in thread "main" java.lang.NullPointerException
	at clojure.core.protocols$naive_seq_reduce.invokeStatic(protocols.clj:62)
	at clojure.core.protocols$interface_or_naive_reduce.invokeStatic(protocols.clj:72)
     ...
	at clojure.pprint$pprint.invoke(pprint_base.clj:241)
	at clojure.lang.Var.invoke(Var.java:384)
	at clojure.main$report_error$fn__9186$fn__9187.invoke(main.clj:603)
	at clojure.main$report_error$fn__9186.invoke(main.clj:602)
	at clojure.main$report_error.invokeStatic(main.clj:601)
	at clojure.main$main.invokeStatic(main.clj:666)
	at clojure.main$main.doInvoke(main.clj:616)
	at clojure.lang.RestFn.applyTo(RestFn.java:137)
	at clojure.lang.Var.applyTo(Var.java:705)
	at clojure.main.main(main.java:40)

On a subsequent run, I got a different error:

Syntax error macroexpanding clojure.core/defn- at (clojure/tools/cli.cljc:124:1).
null
 Full report at:
/tmp/clojure-12652663517358293277.edn

and that file said:

{:clojure.main/message
 "Syntax error macroexpanding clojure.core/defn- at (clojure/tools/cli.cljc:124:1).\nnull\n",
 :clojure.main/triage
 {:clojure.error/phase :macro-syntax-check,
  :clojure.error/line 124,
  :clojure.error/column 1,
  :clojure.error/source "cli.cljc",
  :clojure.error/symbol clojure.core/defn-,
  :clojure.error/path "clojure/tools/cli.cljc",
  :clojure.error/class java.lang.NullPointerException},
 :clojure.main/trace
 {:via
  [{:type clojure.lang.Compiler$CompilerException,
    :message
    "Syntax error macroexpanding clojure.core/defn- at (clojure/tools/cli.cljc:124:1).",
    :data
    {:clojure.error/phase :macro-syntax-check,
     :clojure.error/line 124,
     :clojure.error/column 1,
     :clojure.error/source "clojure/tools/cli.cljc",
     :clojure.error/symbol clojure.core/defn-},
    :at [clojure.lang.Compiler checkSpecs "Compiler.java" 6976]}
   {:type java.lang.NullPointerException,
    :at [clojure.core$every_QMARK_ invokeStatic "core.clj" 2683]}],
  :trace
  [[clojure.core$every_QMARK_ invokeStatic "core.clj" 2683]
   [clojure.core$every_QMARK_ invoke "core.clj" 2676]
   [clojure.spec.alpha$pcat_STAR_ invokeStatic "alpha.clj" 1388]
   [clojure.spec.alpha$pcat_STAR_ invoke "alpha.clj" 1387]
   [clojure.spec.alpha$deriv invokeStatic "alpha.clj" 1542]
   [clojure.spec.alpha$deriv invoke "alpha.clj" 1528]
   [clojure.spec.alpha$re_conform invokeStatic "alpha.clj" 1669]
   [clojure.spec.alpha$re_conform invoke "alpha.clj" 1660]
   [clojure.spec.alpha$regex_spec_impl$reify__2510
    conform_STAR_
    "alpha.clj"
    1710]
   [clojure.spec.alpha$conform invokeStatic "alpha.clj" 171]
   [clojure.spec.alpha$conform invoke "alpha.clj" 167]
   [clojure.spec.alpha$macroexpand_check invokeStatic "alpha.clj" 708]
   [clojure.spec.alpha$macroexpand_check invoke "alpha.clj" 704]
   [clojure.lang.AFn applyToHelper "AFn.java" 156]
   [clojure.lang.AFn applyTo "AFn.java" 144]
   [clojure.lang.Var applyTo "Var.java" 705]
   [clojure.lang.Compiler checkSpecs "Compiler.java" 6974]
   [clojure.lang.Compiler macroexpand1 "Compiler.java" 6992]
   [clojure.lang.Compiler macroexpand "Compiler.java" 7079]
   [clojure.lang.Compiler eval "Compiler.java" 7165]
   [clojure.lang.Compiler load "Compiler.java" 7640]
   [clojure.lang.RT loadResourceScript "RT.java" 381]
   [clojure.lang.RT loadResourceScript "RT.java" 372]
   [clojure.lang.RT load "RT.java" 459]
   [clojure.lang.RT load "RT.java" 424]
   [clojure.core$load$fn__6857 invoke "core.clj" 6115]
   [clojure.core$load invokeStatic "core.clj" 6114]
   [clojure.core$load doInvoke "core.clj" 6098]
   [clojure.lang.RestFn invoke "RestFn.java" 408]
   [clojure.core$load_one invokeStatic "core.clj" 5897]
   [clojure.core$load_one invoke "core.clj" 5892]
   [clojure.core$load_lib$fn__6797 invoke "core.clj" 5937]
   [clojure.core$load_lib invokeStatic "core.clj" 5936]
   [clojure.core$load_lib doInvoke "core.clj" 5917]
   [clojure.lang.RestFn applyTo "RestFn.java" 142]
   [clojure.core$apply invokeStatic "core.clj" 669]
   [clojure.core$load_libs invokeStatic "core.clj" 5974]
   [clojure.core$load_libs doInvoke "core.clj" 5958]
   [clojure.lang.RestFn applyTo "RestFn.java" 137]
   [clojure.core$apply invokeStatic "core.clj" 669]
   [clojure.core$require invokeStatic "core.clj" 5996]
   [clojure.core$require doInvoke "core.clj" 5996]
   [clojure.lang.RestFn invoke "RestFn.java" 619]
   [clojure.tools.deps.alpha.script.make_classpath2$eval140$loading__6738__auto____141
    invoke
    "make_classpath2.clj"
    9]
   [clojure.tools.deps.alpha.script.make_classpath2$eval140
    invokeStatic
    "make_classpath2.clj"
    9]
   [clojure.tools.deps.alpha.script.make_classpath2$eval140
    invoke
    "make_classpath2.clj"
    9]
   [clojure.lang.Compiler eval "Compiler.java" 7181]
   [clojure.lang.Compiler eval "Compiler.java" 7170]
   [clojure.lang.Compiler load "Compiler.java" 7640]
   [clojure.lang.RT loadResourceScript "RT.java" 381]
   [clojure.lang.RT loadResourceScript "RT.java" 372]
   [clojure.lang.RT load "RT.java" 459]
   [clojure.lang.RT load "RT.java" 424]
   [clojure.core$load$fn__6857 invoke "core.clj" 6115]
   [clojure.core$load invokeStatic "core.clj" 6114]
   [clojure.core$load doInvoke "core.clj" 6098]
   [clojure.lang.RestFn invoke "RestFn.java" 408]
   [clojure.core$load_one invokeStatic "core.clj" 5897]
   [clojure.core$load_one invoke "core.clj" 5892]
   [clojure.core$load_lib$fn__6797 invoke "core.clj" 5937]
   [clojure.core$load_lib invokeStatic "core.clj" 5936]
   [clojure.core$load_lib doInvoke "core.clj" 5917]
   [clojure.lang.RestFn applyTo "RestFn.java" 142]
   [clojure.core$apply invokeStatic "core.clj" 669]
   [clojure.core$load_libs invokeStatic "core.clj" 5974]
   [clojure.core$load_libs doInvoke "core.clj" 5958]
   [clojure.lang.RestFn applyTo "RestFn.java" 137]
   [clojure.core$apply invokeStatic "core.clj" 669]
   [clojure.core$require invokeStatic "core.clj" 5996]
   [clojure.main$main_opt invokeStatic "main.clj" 514]
   [clojure.main$main_opt invoke "main.clj" 510]
   [clojure.main$main invokeStatic "main.clj" 664]
   [clojure.main$main doInvoke "main.clj" 616]
   [clojure.lang.RestFn applyTo "RestFn.java" 137]
   [clojure.lang.Var applyTo "Var.java" 705]
   [clojure.main main "main.java" 40]],
  :phase :macro-syntax-check}}

On yet another run, I got yet a different error message:

Exception in thread "main" java.lang.NullPointerException
	at clojure.core$reduce1.invokeStatic(core.clj:944)
	at clojure.core$reduce1.invokeStatic(core.clj:936)
	at clojure.core$merge.invokeStatic(core.clj:3052)
	at clojure.core$Throwable__GT_map$base__7474.invoke(core_print.clj:491)
	at clojure.core$map$fn__5885.invoke(core.clj:2757)
	at clojure.lang.LazySeq.sval(LazySeq.java:42)
	at clojure.lang.LazySeq.seq(LazySeq.java:51)
	at clojure.lang.RT.seq(RT.java:535)
	at clojure.lang.LazilyPersistentVector.create(LazilyPersistentVector.java:44)
	at clojure.core$vec.invokeStatic(core.clj:379)
	at clojure.core$Throwable__GT_map.invokeStatic(core_print.clj:492)
	at clojure.main$report_error.invokeStatic(main.clj:593)
	at clojure.main$main.invokeStatic(main.clj:666)
	at clojure.main$main.doInvoke(main.clj:616)
	at clojure.lang.RestFn.applyTo(RestFn.java:137)
	at clojure.lang.Var.applyTo(Var.java:705)
	at clojure.main.main(main.java:40)

The problem is also reproducible in Clojure 1.5.1:

$ java -Xcomp -cp clojure-1.5.1.jar clojure.main
Exception in thread "main" java.lang.ExceptionInInitializerError
	at java.base/java.lang.J9VMInternals.ensureError(J9VMInternals.java:185)
	at java.base/java.lang.J9VMInternals.recordInitializationFailure(J9VMInternals.java:174)
	at java.base/java.lang.Class.forNameImpl(Native Method)
	at java.base/java.lang.Class.forName(Class.java:413)
	at clojure.lang.RT.loadClassForName(RT.java:2098)
	at clojure.lang.RT.load(RT.java:430)
	at clojure.lang.RT.load(RT.java:411)
	at clojure.core$load$fn__5018.invoke(core.clj:5530)
	at clojure.core$load.doInvoke(core.clj:5529)
	at clojure.lang.RestFn.invoke(RestFn.java:408)
	at clojure.core$load_one.invoke(core.clj:5336)
	at clojure.core$load_lib$fn__4967.invoke(core.clj:5375)
	at clojure.core$load_lib.doInvoke(core.clj:5374)
	at clojure.lang.RestFn.applyTo(RestFn.java:142)
	at clojure.core$apply.invoke(core.clj:619)
	at clojure.core$load_libs.doInvoke(core.clj:5413)
	at clojure.lang.RestFn.applyTo(RestFn.java:137)
	at clojure.core$apply.invoke(core.clj:619)
	at clojure.core$require.doInvoke(core.clj:5496)
	at clojure.lang.RestFn.invoke(RestFn.java:408)
	at clojure.lang.Var.invoke(Var.java:415)
	at clojure.main.main(main.java:36)
Caused by: java.lang.NullPointerException
	at clojure.core$reduce1.invoke(core.clj:896)
	at clojure.core$expand_method_impl_cache.invoke(core_deftype.clj:474)
	at clojure.core$_cache_protocol_fn.invoke(core_deftype.clj:542)
	at clojure.core.protocols$fn__5979$G__5974__5992.invoke(protocols.clj:13)
	at clojure.core$reduce.invoke(core.clj:6177)
	at clojure.core$into.invoke(core.clj:6229)
	at clojure.main__init.load(Unknown Source)
	at clojure.main__init.<clinit>(Unknown Source)

2 Answers

0 votes
by
selected by
0 votes
by

Problems with the J9 JVM have been discussed quite a bit in the last week on Slack and the consensus seemed to be that Clojure is not compatible with it (it is not tested against it). I just searched for J9 and found half a dozen folks reporting pretty much random behavior on the J9 JVM across a bunch of channels.

by
In case of fire, it's super to be in a place with 2 exits.  Having two mainstream certified JVMs is like that.  You don't try the fire escape every year.  But, if you try the fire escape, and it's locked, uh-oh.  Especially if everyone else uses the fire escape fine, it seems to be locked only for you, presumably because of your special brain implant.

I bring the matter here to provide details to start a Jira issue.  That's our way of establishing conditions favorable to discovery of a workaround, or a crisp upstream bug ticket.

Having `reduce1` cited as the immediate cause, noticeably often, in J9 versions 11 & 15, for Clojure versions as far back as 1.5.1, makes me hope that the failure can be narrowed and either worked around or brought to the attention of the J9 bug fixers.
by
I doubt reduce1 is special, more likely it shows up because it's almost the first lazy seq code to be run in the JVM. Would not surprise me if the npe is related to locals clearing bytecode, which is emitted by the Clojure compiler but unusual from a Java->JVM standpoint and involved in most functions traversing a lazy seq.
by
A promising theory. There's another stack trace where the point-of-problem was in `some`.
...