Node.js shim requires `node` invocation to be in the same directory as shim script

Question

Don't have time to provide a proper repro now but the basic issue can be illustrated by this:

`
~/c/boot-cljs-example (master=) node target/main.js
module.js:338

throw err;
      ^

Error: Cannot find module '/Users/martin/code/boot-cljs-example/out/goog/bootstrap/nodejs.js'

at Function.Module._resolveFilename (module.js:336:15)
at Function.Module._load (module.js:278:25)
at Module.require (module.js:365:17)
at require (module.js:384:17)
at Object.<anonymous> (/Users/martin/code/boot-cljs-example/target/main.js:6:1)
at Module._compile (module.js:460:26)
at Object.Module._extensions..js (module.js:478:10)
at Module.load (module.js:355:32)
at Function.Module._load (module.js:310:12)
at Function.Module.runMain (module.js:501:10)

~/c/boot-cljs-example (master=) cd target/
~/c/b/target (master=) node main.js
Starting...
`

This is compiled with boot because that was what I had at hand right now. The compiled shim looks like this:

`
var path = require("path");
try {

require("source-map-support").install();

} catch(err) {
}
require(path.join(path.resolve("."),"out","goog","bootstrap","nodejs.js"));
require(path.join(path.resolve("."),"out","cljs_deps.js"));
goog.global.CLOSURE_UNCOMPILED_DEFINES = {"cljs.core._STAR_targetSTAR":"nodejs"};
goog.require("boot.cljs.main");
goog.require("cljs.nodejscli");
`

The problem here is that {{path.resolve(".")}} will return the directory the {{node}} command was invoked in and not the directory of the shim. (See the "Cannot find module..." error above)

A solution could be to use {{__dirname}} which always resolves to the directory of the current file. This might result in some breakage for existing setups.

Answer 1

Comment made by: bensu

I have a proposed solution but I fear that supporting "run from anywhere" adds essential complexity to the resulting code. My thought process:

1. Relative paths are desirable to produce "context free code." If the user chooses absolute paths, then that behavior is respected and there's nothing to handle (no "path algebra") .

2. When dealing with relative paths, the whole system needs to establish a "frame of reference", a root path. The ClojureScript compiler assumes the path from which it is compiling to be that frame of reference, which usually coincides with the top root of the project. Though arbitrary, it is the only choice that makes sense.

3. The frame of reference is not explicit anywhere in the code, since it is defined as {{"."}}. If it were explicit, it would reveal context, as in {{"/home/some-user/their-folder/this-project/"}}.

4. When we approach the code from another reference point (executing the script from another directory), we first need to find the original compiler path (reference point,) and then resolve all paths from there. The compiler uses cljs.closure/path-relative-to for this purpose.

Path algebra:
{{compiler-path = __dirname - output-to}}

Node.js

var compiler-path = __dirname.replace(output-to, "") path.resolve (compiler-path, output-dir, "goog", "base.js") path.resolve (compiler-path, output-dir, "cljs_deps.js")

which assumes that if output-to was given relatively, then output-dir is also relative. If they are not in sync, more work needs to be done to keep them that way.

It's not up to me to decide if the extra complexity is worth the use-case. I actually hope there is a simpler solution to solve this that I'm not seeing.

Answer 2

Comment made by: karolmajta

I find this behavior really weird and would opt for switching to {{__dirname}}. I am also not sure i fully understand consequences of such switch (I have little cljs experience, speaking more from the perspective of nodejs user). My point is: current behavior renders clojurescript hard to use for commandline and desktop applications (namely electron).

For command line and desktop applications assumptions about CWD cannot be made. For now i run my applications through a bootstrap script JS script:

process.chdir(__dirname); require('./target/out');

I am lucky that my code does not have to use the real CWD, but it's a hack more than a real solution.

Speaking from nodejs perspective:

1. Using absolute paths is considered a bad practice anyway.
2. Nodejs programs that don't use external packages (don't depend on node_modules) can be run from any CWD
3. Nodejs programs that do depend on node_modules will break if run from a different directory than the one containing node_modules, but this is expected behavior.

Answer 3

Comment made by: pupeno

I just run into this problem when trying to develop an Electron application. The way it's working right now is essentially unpackageable. I think it would be nice to have this behavior even as an option and I'm happy to work on a patch.

Answer 4

Comment made by: pupeno

As far as I can see, this is the relevant code:

https://github.com/clojure/clojurescript/blob/cdaeff298e0f1d410aa5a7b6860232270d287084/src/main/clojure/cljs/closure.clj#L1410-L1411

Answer 5

Comment made by: pupeno

A potential workaround seems to use :simple optimization.

Answer 6

Comment made by: thatismatt

I have also just hit this issue in an electron app that I'm building. @pupeno did you get anywhere with your offer to work on a patch? I too would be happy to look in to a patch for this. Although I think I'll need some pointers to get started.

Answer 7

Comment made by: thatismatt

From a quick experiment this morning it looks like replacing {{path.resolve(".")}} with {{__dirname}} at (link: https://github.com/clojure/clojurescript/blob/aa5f001300e9aebd976cb180f5b7ccb37fcb6898/src/main/clojure/cljs/closure.clj#L1460-L1461) works for a couple of simple electron and node apps. By work I specifically mean that the error doesn't occur even when the app is started from a current working directory that is different to the compilation directory, i.e. the code is path independent.

I'll attach a patch for this once I've done some more complete testing. Before then any feedback on this approach would be appreciated.

Answer 8

Comment made by: thatismatt

Attaching patch for the suggested fix of using __dirname instead of "." in the generated script. I have tested this with nodejs 6.10.0 and electron 1.6.5.

Answer 9

Comment made by: thatismatt

Also see: (link: CLJS-1990)

Answer 10

Comment made by: matthiasn

Having the same problem where I can't use optimization :none when packaging with electron-builder. Any chance to accept the suggested patch? Thanks!

Answer 11

Reference: https://clojure.atlassian.net/browse/CLJS-1444 (reported by martinklepsch)