Powershell woes

Question

I've been pretty vocal on Slack about the issues introduced by the Powershell Clojure CLI command line on Windows. Alex Miller asked that the issues be described, augmented with community feedback, and that possible solutions were presented in some kind of a decision table.

While I'm not entirely certain that the format meets Alex Miller's requirements, here it goes :
Clojure CLI issues

In this document, I present two possible solutions.
- a binary wrapper around the Powershell script
- a portable binary rewrite of the bash/Powershell script

Both of these solutions require a fairly low coding effort, and I produced exploratory projects.
- A Clojure Powershel CLI wrapper. Tested on Windows.
- A portable Clojure CLI rewrite . Tested on Windows and Linux, it demonstrates that a portable solution isn't too hard to come by.

I hope to receive feedback on the issues presented, and those two projects.

Related jira: https://clojure.atlassian.net/browse/TDEPS-136

Answer 1

FWIW I started a cross platform implementation written in CLJS and distributed via npm a while ago. Not actually sure that is a good idea but given how widely distributed node is these days it be an option for a lot of people.

I sort of stopped working on it since it has this annoying problem on Windows which asks Terminate batch job (Y/N)? on CTRL+C every time. I didn't find a way to fix this and it is pretty annoying. Linux/macOS worked fine though.

See https://github.com/thheller/clojure-cli

Comments

Thanks for your input
Yes, that's an issue with batch files, many people are complaining about this. AFAIK there is no solution to this issue outside replacing the command interpreter.

Answer 2

I think the idea of a portable Clojure CLI wrapper sounds nice, but for most platforms: Linux, BSD, OSX, bash might actually be a better option, since it just works, and is fast enough. No need to compile to different targets, have a 32bit and a 64bit version, or download the appropriate version for your platform.

If GraalVM's native image was out of preview, especially on Windows, I'd personally lean towards that. Based on this benchmark: https://github.com/chocolateboy/startup-time it is as fast as bash, actually faster.

Based on the above benchmark and this one: https://github.com/bdrung/startup-time, it seems Pascal might actually have the smallest startup time of all, followed by C. It shows that Nim is almost as fast as C, faster then Go and Bash.

It does seem a bit weird to rely on another language, but then again, bash is also another language, though for some reason, it seems less weird to depend on. Windows not supporting a proper Bash is a real shame though.

I'd still vote for GraalVM native image if it is in a good enough shape to work on Windows. With the active work Oracle is doing on it, it should just keep getting better over time. And I don't think anything the Clojure CLI starter script is doing won't work with Graal. And it is Clojure all the way.

Otherwise, going with Nim or C or Pascal or Go instead of bash is a decision I feel only the maintainers should make. The bash pipeline is the easiest. Just package the script, no need to compile it for different targets and publish different installers. That's also way easier than Graal would be. I understand why that choice was made. The Windows case does make an argument to lean for something else though.

Anyways, just wanted to add some of the benchmark info here, especially that Graal native image should be competitive with bash in terms of startup time.

To make this more of an answer, I guess I'm answering that bash provides the simplest packaging of all the options, which made it a sound choice at first, but with Windows added to the mix, we see that bash isn't portable enough, and thus either we need two implementations, in this case, one in bash and one in powershell, or a more portable language like Java with GraalVM compilation, Nim, or C, would result in allowing a single code base to wok across platforms, but at the cost of a more complicated compiling and packaging pipeline.

Comments

Thanks for your feedback.
- Part of the decision of going with Nim was the ability to do do cross-compilation from a single OS to all targets.  Anyways any language will do, the full code is in the 3 or 400 lines area, that's nothing and can be rewritten at the drop of a hat.
- I agree with you that bash is very much good enough  on Posix systems.
- GraalVM sounds like a good idea, but there are two issues with it :
   - It's not yet ready for prime time on Windows
   - AFAIK it is not possible to get the full command line used to call a java program which is required on Windows to interpret it in a Posix compatible way. More research needed on my part there.
- I agree that startup times for GraalVM binaries seem to be good enough, binary size might be a different story, but do we really care about that ?
- Finally I agree with you that the final decision rests with the Clojure team. Though there is a bit of an urgency there. People are evaluating tools and maybe discarding these every day because of the shortcoming of the Powershell CLI. This may affect the adoption of deps.edn projects, at least with the Windows crowd.

---

ah-a !  https://github.com/profesorfalken/jProcesses can get the command line...

---

Didn't know that about Nim. That's a great point though, because one of the drawbacks for going with a natively compiled toolchain is the more complicated build and packaging needs. If Nim can build and package for all platforms without needing to run the build step under them, that can make things a lot easier. I actually don't know if that is the case for GraalVM.

---

As a test, I added cross-compilation to the portable rewrite (Linux side), looks like it's working like a charm.

Answer 3

I added a rust implementation to the existing Nim implementation. That one was rather harder to produce than I thought it would be , probably because i'm a novice there. But once it works, it does work !

Here are some more thoughts on this endeavor.

Worries about build process complexity

Cross compilation is key to address this issue. I researched this in depth. Though i didn't do Darwin, which is an important one in this community, but i don't have a mac ! Both projects are creating a windows installer and linux binary from linux. A docker file and build script are also provided to do this in a platform independent, reproducible way.

• If the rewrite was to be chosen, i would advise building from a mac + do the provided docker build and so get all 3 platforms covered from the same machine.
• There should be no change to the code in either project for a mac build (though i can't really be sure)
• If we only want to do windows, we can still do it either from windows, linux, linux with docker or mac with docker.

So to address the issue : a working docker is all the setup required, then a call to the provided docker_build script does all the building, packaging and even builds the windows installer.

Nim vs Rust

both projects have the same features

Nim :
- a joy to work with -> good
- easy to learn, read, write -> good
- found a bug in the zip file implementation (which i use to circumvent TDEP-120) -> worked around it but it is not good
- small community -> not good
- easy cross compilation -> good
- not the responsible choice -> bad
- concise -> good
- VS code support is good -> good
Rust :
- Bondage and discipline -> both good & not good
- rock solid -> good
- harder to write -> not good
- readable -> ok i guess
- a pain to work with -> bad
- easy cross compilation -> good
- the responsible choice -> good
- VS code support is good -> good
- emacs support is top notch -> who cares but me ?

Though Rust is more responsible, I had to go back to the Nim implementation for a couple fixes, and it was like breathing fresh air again after Rust.

The Rust version has obviously not been tested as extensively as the Nim version.

I'll of course keep the wrapper and these 2 rewrites maintained and up to date, but that's where I stop active work on this.

Answer 4

There is now a Clojure implementation built using GraalVM as a native executable for Linux, Mac and Windows as well: https://github.com/borkdude/deps.clj

In my opinion, this is the best path forward.