Direct linking doesn't work on recursive calls

Question

It looks like self-recursive calls aren't optimized by direct linking, but if we redefine the same function twice, the Compiler is tricked into thinking that the call is not recursive and (rightfully) optimizes it into an invokeStatic.

I haven't investigated the cause but I suspect (and I might be wrong) it has to do with :arglist metadata potentially having different values when the Var is undefined vs when it's already bound.

`
[~]> cat test.clj
(ns test)

(defn a [x]
(a x))
[~]> clj
Clojure 1.8.0-master-SNAPSHOT
user=> (compile 'test)
test
user=> ^D
[~]> cd classes
[~/classes]> javap -c test\$a
Compiled from "test.clj"
public final class test$a extends clojure.lang.AFunction {
public static final clojure.lang.Var const__0;

public static {};

Code:
   0: ldc           #11                 // String test
   2: ldc           #13                 // String a
   4: invokestatic  #19                 // Method clojure/lang/RT.var:(Ljava/lang/String;Ljava/lang/String;)Lclojure/lang/Var;
   7: checkcast     #21                 // class clojure/lang/Var
  10: putstatic     #23                 // Field const__0:Lclojure/lang/Var;
  13: return

public test$a();

Code:
   0: aload_0
   1: invokespecial #26                 // Method clojure/lang/AFunction."<init>":()V
   4: return

public static java.lang.Object invokeStatic(java.lang.Object);

Code:
   0: getstatic     #23                 // Field const__0:Lclojure/lang/Var;
   3: invokevirtual #32                 // Method clojure/lang/Var.getRawRoot:()Ljava/lang/Object;
   6: checkcast     #34                 // class clojure/lang/IFn
   9: aload_0
  10: aconst_null
  11: astore_0
  12: invokeinterface #37,  2           // InterfaceMethod clojure/lang/IFn.invoke:(Ljava/lang/Object;)Ljava/lang/Object;
  17: areturn

public java.lang.Object invoke(java.lang.Object);

Code:
   0: aload_1
   1: aconst_null
   2: astore_1
   3: invokestatic  #41                 // Method invokeStatic:(Ljava/lang/Object;)Ljava/lang/Object;
   6: areturn

}`

Redefining the same function twice makes it work.

`
[~]> cat test.clj
(ns test)

(defn a [x]
(a x))

(defn a [x]
(a x))
[~]> clj
Clojure 1.8.0-master-SNAPSHOT
user=> (compile 'test)
test
user=> ^D
[~]> cd classes
[~/classes]> javap -c test\$a
Compiled from "test.clj"
public final class test$a extends clojure.lang.AFunction {
public static final clojure.lang.Var const__0;

public static {};

Code:
   0: ldc           #11                 // String test
   2: ldc           #13                 // String a
   4: invokestatic  #19                 // Method clojure/lang/RT.var:(Ljava/lang/String;Ljava/lang/String;)Lclojure/lang/Var;
   7: checkcast     #21                 // class clojure/lang/Var
  10: putstatic     #23                 // Field const__0:Lclojure/lang/Var;
  13: return

public test$a();

Code:
   0: aload_0
   1: invokespecial #26                 // Method clojure/lang/AFunction."<init>":()V
   4: return

public static java.lang.Object invokeStatic(java.lang.Object);

Code:
   0: aload_0
   1: aconst_null
   2: astore_0
   3: invokestatic  #30                 // Method invokeStatic:(Ljava/lang/Object;)Ljava/lang/Object;
   6: areturn

public java.lang.Object invoke(java.lang.Object);

Code:
   0: aload_1
   1: aconst_null
   2: astore_1
   3: invokestatic  #30                 // Method invokeStatic:(Ljava/lang/Object;)Ljava/lang/Object;
   6: areturn

}`

Answer 1

Comment made by: bronsa

I just took a quick look at this, not an easy one to fix as handling recursive calls would likely require 2 passes over the whole defn AST, one to determine whether the defn is direct-linkable and the other one to build the AST using StaticInvokeExpr rather than InvokeExpr (using a stub Method using the analysis info rather than reflecting)

Answer 2

Comment made by: alexmiller

Yeah, Rich is aware of this and it's not done yet due to the issues you mentioned. It's hard!

Answer 3

Reference: https://clojure.atlassian.net/browse/CLJ-1865 (reported by bronsa)