Welcome! Please see the About page for a little more info on how this works.

0 votes
in Tools by
edited by

Background:

  1. My build runs in a container where I have no access to temporary files (they get thrown away as soon as the build completes, whether successfully or not). As a result I add --report stderr option to all invocations of the clojure command.
  2. My build constructs an uberjar, using Depstar, which requires the use of the -X:... option.
  3. I also use the -Srepro option.

Problem:

When a clojure command line includes the -X option, it doesn't appear that there's any way to also include --report stderr option. Here's some sample output:

$ clojure --report stderr -Srepro -X:uberjar
WARNING: Use of -A with clojure.main is deprecated, use -M instead
{:clojure.main/message
 "Execution error (FileNotFoundException) at java.io.FileInputStream/open0 (FileInputStream.java:-2).\n-Srepro (No such file or directory)\n",
 :clojure.main/triage
 {:clojure.error/class java.io.FileNotFoundException,
  :clojure.error/line -2,
  :clojure.error/cause "-Srepro (No such file or directory)",
  :clojure.error/symbol java.io.FileInputStream/open0,
  :clojure.error/source "FileInputStream.java",
  :clojure.error/phase :execution},
 :clojure.main/trace
 {:via
  [{:type java.io.FileNotFoundException,
    :message "-Srepro (No such file or directory)",
    :at [java.io.FileInputStream open0 "FileInputStream.java" -2]}],
  :trace
  [[java.io.FileInputStream open0 "FileInputStream.java" -2]
   [java.io.FileInputStream open "FileInputStream.java" 219]
   [java.io.FileInputStream <init> "FileInputStream.java" 157]
   [java.io.FileInputStream <init> "FileInputStream.java" 112]
   [clojure.lang.Compiler loadFile "Compiler.java" 7575]
   [clojure.main$load_script invokeStatic "main.clj" 475]
   [clojure.main$script_opt invokeStatic "main.clj" 535]
   [clojure.main$script_opt invoke "main.clj" 530]
   [clojure.main$main invokeStatic "main.clj" 664]
   [clojure.main$main doInvoke "main.clj" 616]
   [clojure.lang.RestFn applyTo "RestFn.java" 137]
   [clojure.lang.Var applyTo "Var.java" 705]
   [clojure.main main "main.java" 40]],
  :cause "-Srepro (No such file or directory)"}}

Execution error (FileNotFoundException) at java.io.FileInputStream/open0 (FileInputStream.java:-2).
-Srepro (No such file or directory)

With the first two options swapped:

$ clojure -Srepro --report stderr -X:uberjar
WARNING: Use of -A with clojure.main is deprecated, use -M instead
{:clojure.main/message
 "Execution error (FileNotFoundException) at java.io.FileInputStream/open0 (FileInputStream.java:-2).\n-X:uberjar (No such file or directory)\n",
 :clojure.main/triage
 {:clojure.error/class java.io.FileNotFoundException,
  :clojure.error/line -2,
  :clojure.error/cause "-X:uberjar (No such file or directory)",
  :clojure.error/symbol java.io.FileInputStream/open0,
  :clojure.error/source "FileInputStream.java",
  :clojure.error/phase :execution},
 :clojure.main/trace
 {:via
  [{:type java.io.FileNotFoundException,
    :message "-X:uberjar (No such file or directory)",
    :at [java.io.FileInputStream open0 "FileInputStream.java" -2]}],
  :trace
  [[java.io.FileInputStream open0 "FileInputStream.java" -2]
   [java.io.FileInputStream open "FileInputStream.java" 219]
   [java.io.FileInputStream <init> "FileInputStream.java" 157]
   [java.io.FileInputStream <init> "FileInputStream.java" 112]
   [clojure.lang.Compiler loadFile "Compiler.java" 7575]
   [clojure.main$load_script invokeStatic "main.clj" 475]
   [clojure.main$script_opt invokeStatic "main.clj" 535]
   [clojure.main$script_opt invoke "main.clj" 530]
   [clojure.main$main invokeStatic "main.clj" 664]
   [clojure.main$main doInvoke "main.clj" 616]
   [clojure.lang.RestFn applyTo "RestFn.java" 137]
   [clojure.lang.Var applyTo "Var.java" 705]
   [clojure.main main "main.java" 40]],
  :cause "-X:uberjar (No such file or directory)"}}

Execution error (FileNotFoundException) at java.io.FileInputStream/open0 (FileInputStream.java:-2).
-X:uberjar (No such file or directory)

Finally, the version of the CLI tools I'm using:

$ clojure --version
Clojure CLI version 1.10.3.855

So the question is: how can I ensure any uncaught exceptions thrown out of a -X function (in this case Depstar, but it could be anything) are written to stdout or stderr, rather than a temporary file that I can't access? Does -X bypass that (irritating) default behaviour of writing exception details to a temporary file, perhaps?

2 Answers

+1 vote
by

The --report option is a feature of clojure.main (not the clojure CLI). (And just FYI, there is an equivalent Java system property way to do this with clojure.main - see https://clojure.org/reference/repl_and_main#_as_launcher)

Using clj -X, you are not using clojure.main, so this option does not exist, but also -X should never do the thing that writes to a temp file, so I'm confused why you would need it at all with -X?

by
It’s confusing because folks unfamiliar with the internal implementation differences between -X, -M, -A (in the past), -P etc. etc., don’t know (or, tbqh, care) that there are (unnecessary) differences in the default exception handling behaviour of these options (some rely on clojure.main, others, presumably, on the JVM’s default exception handling mechanism).

It would be ideal if the principle of least surprise were applied here - that exception handling is always handled the same way regardless of which other clj/clojure options are used, and the default handling mechanism can be overridden via some consistent and universal option.
by
It would be good to think about whether the -X execution should take on some of the error reporting functionality of clojure.main. They are (intentionally) different but maybe there could be more overlap.
by
Since I cannot format code in a comment, I'm going to edit my answer to address this.
by
Yeah I can't think of any good reasons for responding to unhandled exceptions differently based on what was invoked and how.  That seems like a recipe for confusion.
by
Logged as https://clojure.atlassian.net/browse/TDEPS-193 to think about this more.
by
I am running `clojure -X:sql:dev development/cli-start` yet when this fails (due to a port not being free), I get the `Full report at: /var/folders/06/46ycckp94db7k29k022nxjp00000gn/T/clojure-8233560807290300750.edn` and changing the command to `env JAVA_OPTS='-Dclojure.main.report=stderr' clojure -X:sql:dev development/cli-start` makes the exception to appear inline. This is with `clojure --version` of 1.11.1.1189 (and Clojure 1.11.1). So this seems to contradict directly what you are saying, Alex?!
by
As I said in my answer:

Using -X does execute clojure.main but only for the CLI API stub which then invokes one or more "exec functions". You can't pass --report to it, but you can use the JVM option.
by
Things have changed in the implementation here a number of times. In current versions, -X/-T will go through clojure.main to exec functions. I would currently consider that an implementation detail (but probably one that will stick). We have a jira to consider this more fully at https://clojure.atlassian.net/browse/TDEPS-193
0 votes
by
edited by

depstar shells out to run the AOT compilation process and that does use clojure.main but there isn't an easy way to pass --report to that subprocess. You can, however, pass :jvm-opts to depstar which will in turn pass them to that subprocess:

:jvm-opts '["-Dclojure.main.report=stderr"]'

depstar itself will write to stdout/stderr:

(! 650)-> clojure -X:jar :jar test.jar :main-class foo.bar :aot true
[main] WARN hf.depstar.uberjar - :aot is not recommended for a 'thin' JAR!
[main] INFO hf.depstar.uberjar - Compiling foo.bar ...
Execution error (FileNotFoundException) at user/eval136 (REPL:1).
Could not locate foo/bar__init.class, foo/bar.clj or foo/bar.cljc on classpath.

Full report at:
/var/folders/p1/30gnjddx6p193frh670pl8nh0000gn/T/clojure-13601013524882075105.edn

[main] ERROR hf.depstar.uberjar - Compilation of foo.bar failed!
[main] ERROR hf.depstar.uberjar - AOT FAILED

Here's the same thing with that JVM option supplied:

(! 652)-> clojure -X:jar :jar test.jar :main-class foo.bar :aot true :jvm-opts '["-Dclojure.main.report=stderr"]'
[main] WARN hf.depstar.uberjar - :aot is not recommended for a 'thin' JAR!
[main] INFO hf.depstar.uberjar - Compiling foo.bar ...
{:clojure.main/message
 "Execution error (FileNotFoundException) at user/eval136 (REPL:1).\nCould not locate foo/bar__init.class, foo/bar.clj or foo/bar.cljc on classpath.\n",
 :clojure.main/triage
 {:clojure.error/class java.io.FileNotFoundException,
  :clojure.error/line 1,
  :clojure.error/cause
  "Could not locate foo/bar__init.class, foo/bar.clj or foo/bar.cljc on classpath.",
  :clojure.error/symbol user/eval136,
  :clojure.error/phase :execution},
 :clojure.main/trace
 {:via
  [{:type java.io.FileNotFoundException,
    :message
    "Could not locate foo/bar__init.class, foo/bar.clj or foo/bar.cljc on classpath.",
    :at [clojure.lang.RT load "RT.java" 462]}],
  :trace
  [[clojure.lang.RT load "RT.java" 462]
   [clojure.lang.RT load "RT.java" 424]
   [clojure.core$load$fn__6856 invoke "core.clj" 6115]
   [clojure.core$load invokeStatic "core.clj" 6114]
   [clojure.core$load doInvoke "core.clj" 6098]
   [clojure.lang.RestFn invoke "RestFn.java" 408]
   [clojure.core$load_one invokeStatic "core.clj" 5897]
   [clojure.core$compile$fn__6861 invoke "core.clj" 6125]
   [clojure.core$compile invokeStatic "core.clj" 6125]
   [clojure.core$compile invoke "core.clj" 6117]
   [user$eval136 invokeStatic "NO_SOURCE_FILE" 1]
   [user$eval136 invoke "NO_SOURCE_FILE" 1]
   [clojure.lang.Compiler eval "Compiler.java" 7181]
   [clojure.lang.Compiler eval "Compiler.java" 7136]
   [clojure.core$eval invokeStatic "core.clj" 3202]
   [clojure.main$eval_opt invokeStatic "main.clj" 488]
   [clojure.main$eval_opt invoke "main.clj" 482]
   [clojure.main$initialize invokeStatic "main.clj" 508]
   [clojure.main$null_opt invokeStatic "main.clj" 542]
   [clojure.main$null_opt invoke "main.clj" 539]
   [clojure.main$main invokeStatic "main.clj" 664]
   [clojure.main$main doInvoke "main.clj" 616]
   [clojure.lang.RestFn applyTo "RestFn.java" 137]
   [clojure.lang.Var applyTo "Var.java" 705]
   [clojure.main main "main.java" 40]],
  :cause
  "Could not locate foo/bar__init.class, foo/bar.clj or foo/bar.cljc on classpath."}}

Execution error (FileNotFoundException) at user/eval136 (REPL:1).
Could not locate foo/bar__init.class, foo/bar.clj or foo/bar.cljc on classpath.


[main] ERROR hf.depstar.uberjar - Compilation of foo.bar failed!
[main] ERROR hf.depstar.uberjar - AOT FAILED

Following on from the comments on Alex's answer:

Using -X does execute clojure.main but only for the CLI API stub which then invokes one or more "exec functions". You can't pass --report to it, but you can use the JVM option:

(! 667)-> cat src/thrower.clj 
(ns thrower)

(defn bang [_] (throw (ex-info "foo" {})))

(! 668)-> clojure -X thrower/bang
Execution error (ExceptionInfo) at thrower/bang (thrower.clj:3).
foo

Full report at:
/var/folders/p1/30gnjddx6p193frh670pl8nh0000gn/T/clojure-248093423269602852.edn
(! 669)-> clojure -J-Dclojure.main.report=stderr -X thrower/bang
{:clojure.main/message
 "Execution error (ExceptionInfo) at thrower/bang (thrower.clj:3).\nfoo\n",
 :clojure.main/triage
 {:clojure.error/class clojure.lang.ExceptionInfo,
  :clojure.error/line 3,
  :clojure.error/cause "foo",
  :clojure.error/symbol thrower/bang,
  :clojure.error/source "thrower.clj",
  :clojure.error/phase :execution},
 :clojure.main/trace
 {:via
  [{:type clojure.lang.ExceptionInfo,
    :message "foo",
    :data {},
    :at [thrower$bang invokeStatic "thrower.clj" 3]}],
  :trace
  [[thrower$bang invokeStatic "thrower.clj" 3]
   [thrower$bang invoke "thrower.clj" 3]
   [clojure.lang.AFn applyToHelper "AFn.java" 154]
   [clojure.lang.AFn applyTo "AFn.java" 144]
   [clojure.lang.Var applyTo "Var.java" 705]
   [clojure.core$apply invokeStatic "core.clj" 667]
   [clojure.core$apply invoke "core.clj" 662]
   [clojure.run.exec$exec invokeStatic "exec.clj" 40]
   [clojure.run.exec$exec doInvoke "exec.clj" 35]
   [clojure.lang.RestFn invoke "RestFn.java" 423]
   [clojure.run.exec$_main invokeStatic "exec.clj" 169]
   [clojure.run.exec$_main doInvoke "exec.clj" 157]
   [clojure.lang.RestFn applyTo "RestFn.java" 137]
   [clojure.lang.Var applyTo "Var.java" 705]
   [clojure.core$apply invokeStatic "core.clj" 667]
   [clojure.main$main_opt invokeStatic "main.clj" 514]
   [clojure.main$main_opt invoke "main.clj" 510]
   [clojure.main$main invokeStatic "main.clj" 664]
   [clojure.main$main doInvoke "main.clj" 616]
   [clojure.lang.RestFn applyTo "RestFn.java" 137]
   [clojure.lang.Var applyTo "Var.java" 705]
   [clojure.main main "main.java" 40]],
  :cause "foo",
  :data {}}}

Execution error (ExceptionInfo) at thrower/bang (thrower.clj:3).
foo
by
Thanks!  I've generally not gotten any exceptions out of depstar (for example I compile my code in an earlier step to make sure it's valid), so I'm less worried about it.  My bigger concern is looking at my bash build script in 6 months time and trying to recall why `--report stderr` is in some of the invocations of `clojure`, but not others.
by
Solution: use -J-Dclojure.main.report=stderr on all of them instead! :)
...