Interest in beginner-friendly variant of (pst)?

Question

Another State of Clojure survey and "error messages" is still the number one priority for improvement.

A discussion on Slack led to a number of specific pain points for beginners around exceptions and error messages. The one I'd like to focus on here is to provide a beginner-friendly alternative to (pst).

Currently, (pst) makes a reasonable attempt at demunging names, eliminating "noise", and shrinking the stack trace some -- but it still leaves a lot to be desired for beginners who struggle to understand certain exceptions and have difficulty navigating (Java-style) stack traces, especially compared to some other languages that make a big effort to provide user-friendly error messages and stack traces.

An example from Slack was:

(defn f [i]
  (fn [j]
    (/ j i)))

(run! #(% 1) (map f (range 3)))

The exception printed in the REPL is fine but (pst) shows:

user=> (pst)
ArithmeticException Divide by zero
        clojure.lang.Numbers.divide (Numbers.java:190)
        user/f/fn--16571 (NO_SOURCE_FILE:3)
        user/eval16576/fn--16577 (NO_SOURCE_FILE:1)
        clojure.core/run!/fn--8906 (core.clj:7849)
...

The elided part mentions clojure.lang.ArrayChunk.reduce and then has multiple references to clojure.core.protocols stuff (and doesn't go deep enough by default to show the original call to clojure.core/run!).

I think there's an opportunity here for a new clojure.repl/explain function, taking the same arguments as pst, that provides both a more detailed explanation of the failure and further reduces the noise in the stack trace that pst currently displays.

Ideally, this could be implemented with some basic cleanup in core but with some dynamic hooks that allow other tooling to "install" additional expansion and/or cleanup so that the community can provide libraries and functionality that further improve this aspect of the beginner experience.

A dynamic hook into ex-str, for example, would allow community-provided tooling to massage the exception message shown (both for the original REPL input and for both pst and explain) so that messages that beginners struggle with, such as class <whatever> cannot be cast to clojure.lang.IFn... could be rewritten into beginner-friendly language (Expected a function - found a <whatever>).

Similar dynamic hooks to filter stack frames, and to "print" them to strings, would allow much more friendly output for beginners.

Comments

I often run into really hard-to-decipher errors when dealing with macros or syntax errors (typos).

Personally when I see a "cannot be cast" error I'd love to see the actual value that "cannot be cast" -- instead of "class Symbol cannot be cast to ISeq" it'd be nice to see "class Symbol (`the-actual-symbol`) cannot be cast to ISeq" so it's easier to pinpoint where the typo/issue is. Of course many opaque classes can't be meaningfully toString'd but I think some is still better than none. I realize that these errors are from the Java side of things so adding extra info might not be trivial, but it'd still be nice.

Another issue I often find is the arcane and unreadable spec errors that either take long to parse by eye or I have to use some library to pretty-print them for me (and why do I need a library to make language errors readable?). It'd be nice to have a human-readable reason why "call to defn didn't conform to spec". Possibly in addition to (not instead of) the current output.

---

When you run into a syntax or spec case like this, please come here and file an issue!

The ClassCast exceptions are being generated by the jvm (in literally every function invocation), so it’s really not possible for us to enhance that particular case (unless we emit way more code around every cast). But even for these, if you encounter a confusing error, please come here and file an issue.

---

Emitting more code around those in REPL or tests would be great to be honest.

If Clojure defaulted to a run mode that wraps things so it can clearly show errors in terms of Clojure semantics and not it's implementation details, when in REPL or tests, that would be great for error messages I think.

Answer 1

There are at least two, maybe more, different problems / ideas here and it would be helpful to separate them. The ex-str / hook idea seems orthogonal.

Serious question - if the error messages are ok (you said it was here), and the tooling never shows the stack trace, why is this a beginner issue? Error messages try to show a good location for the source error (to the degree that they can given macros).

"Beginner-friendly" is, I think, not the right framing as it does not say anything about the problem. Seems like if a stack trace could be more illuminating, then it would be useful to both beginners and non-beginners, and we would be helped by concretely listing challenges in understanding stack traces. Off the top of my head...

1. Clojure function invocation often involves 2 or 3 frames per invocation
2. Anonymous functions have obscure names
3. Anonymous functions are implicitly created in some cases by the implementation
4. User calls macros, but stack traces deal in the expanded code (the code you write != the code you run) - this is a fundamental problem for both error messages and stack traces
5. Macro code often creates gensym names that become class names
6. REPL infrastructure is in the stack
7. Tooling infrastructure is in the stack (printers, nrepl, middleware)
8. Mixture of Clojure core lib and Clojure Java impl
9. Stack traces redundantly show file name in addition to Clojure ns or Java class
10. Line numbers don't correlate to source correctly
11. Laziness effects

A "simplified stack trace" function would need to be clear about which of these or other potential problems were actually important. Some of these are pretty straightforward, some are quite hard.

Keep in mind that showing an alternate view of reality will always be in tension with the risk of hiding the actual information that illuminates why you're looking at a stack trace in the first place.

If a simplified stack printer function is useful, then this can be done independent of core. Dev tooling in particular is in a great position to augment here.

Comments

I'll be honest, I'm just the messenger here.

Aside from ClassCastException which seems to crop up "often" for beginners who need a better explanation, I think the messages are OK and I was originally going to create a separate "ask" for the ex-str hook -- so, yes, I should have kept that separate.

I don't know how to phrase the question -- and I don't see other people framing it well either. It's always just "error messages are poor" and "stack traces are confusing/overwhelming". The fact that it's #1 every year on the survey speaks to some sort of "need" from... checks survey... nearly 50% of respondents (this year). I figured I'd just at least start a conversation.

The one point I will highlight (and agree with), because I hear it as the response every year:

"If a simplified stack printer function is useful, then this can be done independent of core" -- but it's "core" that beginners experience first and it's "core" that people are complaining about here.

Hopefully, beginners -- and people who teach beginners -- will chime in with specifics.

---

I continue to think that the biggest problem with stack traces is just that tools show them when they shouldn’t. By all means, more ask questions on specific problems are welcome (and keeping them as isolated as possible helps with votes and priorities).

---

Yes, it would be helpful if folks could provide details of their editor/REPL setup when providing feedback.

Emacs/CIDER still shows big stack traces, five years after all the core work on errors in 1.10, and it's still the most widely used editor setup...

(Alex has provided a crosstab of editor usage vs improvement areas to prioritize and it's clear that Emacs/CIDER's stack traces do not correlate to increased complaints with that editor -- error messages are about 50% of the complaints across all editors)