What is the technical or ideological reason for not allowing `recur` from inside a `catch` block?

Question

Currently, it's not possible to recur across a try block:

  (let [RETRIABLE-FUNCTION (fn [] (println "Function called") (throw (ex-info "Function failed" {})))]
    (loop [retries 3]
      (try
        (RETRIABLE-FUNCTION)
        (catch Exception ex
          (when (< 1 retries)
            (recur (dec retries)))))))

Syntax error (UnsupportedOperationException) compiling recur at (src/repl.clj:7:13). Can only recur from tail position

So instead on has to rewrite the code as something like this:

  (let [RETRIABLE-FUNCTION (fn [] (println "Function called") (throw (ex-info "Function failed" {})))]
    (loop [retries 3]
      (let [result (try
                     (RETRIABLE-FUNCTION)
                     (catch Exception ex
                       (when (< 1 retries)
                         ::retry)))]
        (if (= ::retry result)
          (recur (dec retries))
          result))))

— which is pretty unwieldy, especially in the cases of nested retries or more complex state machines.
Alternatively, one could use one of many retry macro libraries — which is a pretty heavy price for such simple use case, I think. And usually much less flexible than what one could do in-place.

"Ideologically", it seems to me that there should be problem there, as the last expression in catch look to me like a tail position.

Is there a technical limitation of the JVM that makes the recur-across-try impossible (or maybe has very low performance, so it's a good idea to discourage that)? Or is it just a low-priority quality-of-life feature, so it never got to an implementation?

Answer 1

try can have a finally which runs after the catch, so you can't recur from inside a catch as it is not necessarily the tail position.

Comments

But Alex, as I see it, expressions in `finally` are not in "tail position" (so don't change other places' tail position status), since their return values are ignored, so technically even if `finally` is present, ultimate expressions in each `catch` are in tail positions, as I understand it.

    (try
      (throw (ex-info "Exception" {}))
      1
      (catch Exception _ 2)
      (finally 3))

^ this evaluates to 2, and 3 in `finally` clause is ignored.

I assume, on the JVM byte-code level (which I am not very familiar with), they are not in tail position — but since JVM doesn't have tail calls anyway, "tail position" is mostly about Clojure-syntax level.

---

I'm sure it is in some cases possible to do so, but this is a particularly complicated part of both the Clojure compiler and the Java bytecode based on how exceptions are translated into bytecode (and that has actually changed since the compiler was written).

---

"tail position" is a property of the control flow graph, that can be approximated as a property of syntax. the control flow graph exists and is the same at the bytecode level. the jvm enforces some invariants of control flow (I forget exactly, but things like branch targets need to always be jumped to with the same abstract stack state).

the "tail position" in a control flow graph is control flow is transfered to somewhere else and then returns and then immedialty returns again, tail call optimization is collapsing those double returns into a single return. having a finally to execute between the two returns means it is no longer a tail. the finally clause's return value is ignored, but the finally is not ignored, it is executed. the finally is like (let [r e] (do-finally-stuff) r), do-finally-stuff is not a tail, but its existence means e is not a tail either.

in cases where there is no finally you could say the catch is a tail, but I am not sure if the jvm bytecode verifer would allow you to branch from there backwards.