Silent truncation/downcasting of primitive type on reflection call to overloaded method (Math/abs)

Question

I realise relying on reflection when calling these kinds of methods isn't a great idea performance-wise, but it shouldn't lead to incorrect or dangerous behaviour.

Here it seems to trigger a silent downcast of the input longs, giving a truncated integer output:

user> (defn f (link: a b) (Math/abs (- a b)))
Reflection warning, NO_SOURCE_PATH:1:15 - call to abs can't be resolved.

'user/f

user> (f 1000000000000 2000000000000)
727379968
user> (class (f 1000000000000 2000000000000))
java.lang.Integer
user> (defn f (link: ^long a ^long b) (Math/abs (- a b)))

'user/f

user> (f 1000000000000 2000000000000)
1000000000000
user> (class (f 1000000000000 2000000000000))
java.lang.Long

Answer 1

Comment made by: matthjw

For an even simpler way to replicate the issue:

user> (#(Math/abs %) 1000000000000)
Reflection warning, NO_SOURCE_PATH:1:3 - call to abs can't be resolved.
727379968

Answer 2

Comment made by: jafingerhut

I was able to reproduce the behavior you see with these Java 6 JVMs on Ubuntu 12.04.2:

java version "1.6.0_27"
OpenJDK Runtime Environment (IcedTea6 1.12.5) (6b27-1.12.5-0ubuntu0.12.04.1)
OpenJDK 64-Bit Server VM (build 20.0-b12, mixed mode)

java version "1.6.0_45"
Java(TM) SE Runtime Environment (build 1.6.0_45-b06)
Java HotSpot(TM) 64-Bit Server VM (build 20.45-b01, mixed mode)

However, I tried two Java 7 JVMs, and it gave the following behavior which looks closer to what you would hope for. I do not know what is the precise difference between Java 6 and Java 7 that leads to this behavior difference, but this is some evidence that this has something to do with Java 6 vs. Java 7.

user=> (set! warn-on-reflection true)
true
user=> (defn f (link: a b) (Math/abs (- a b)))
Reflection warning, NO_SOURCE_PATH:1:15 - call to abs can't be resolved.

'user/f

user=> (f 1000000000000 2000000000000)
1000000000000
user=> (class (f 1000000000000 2000000000000))
java.lang.Long

Above behavior observed with Clojure 1.5.1 on these JVMs:

Ubuntu 12.04.2 plus this JVM:
java version "1.7.0_21"
Java(TM) SE Runtime Environment (build 1.7.0_21-b11)
Java HotSpot(TM) 64-Bit Server VM (build 23.21-b01, mixed mode)

Mac OS X 10.8.3 plus this JVM:
java version "1.7.0_15"
Java(TM) SE Runtime Environment (build 1.7.0_15-b03)
Java HotSpot(TM) 64-Bit Server VM (build 23.7-b01, mixed mode)

Answer 3

_Comment made by: matthjw_

Ah, interesting.
Maybe it's a difference in the way the reflection API works in java 7?

Here's the bytecode generated incase anyone's curious:

public java.lang.Object invoke(java.lang.Object);
  Code:
   0:    ldc    #14; //String java.lang.Math
   2:    invokestatic    #20; //Method java/lang/Class.forName:(Ljava/lang/String;)Ljava/lang/Class;
   5:    ldc    #22; //String abs
   7:    iconst_1
   8:    anewarray    #24; //class java/lang/Object
   11:    dup
   12:    iconst_0
   13:    aload_1
   14:    aconst_null
   15:    astore_1
   16:    aastore
   17:    invokestatic    #30; //Method clojure/lang/Reflector.invokeStaticMethod:(Ljava/lang/Class;Ljava/lang/String;[Ljava/lang/Object;)Ljava/lang/Object;
   20:    areturn

Answer 4

Comment made by: matthjw

Just an idea (and maybe this is what's happening under java 7?) but given it's a static method and all available overloaded variants are presumably known at compile time, perhaps it could generate code along the lines of:

(cond
(instance? Long x) (Math/abs (long x))
(instance? Integer x) (Math/abs (int x))
;; ...
)

Answer 5

Comment made by: jafingerhut

In Reflector.java method invokeStaticMethod(Class c, String methodName, Object(link: ) args) there is a call to getMethods() followed by a call to invokeMatchingMethod(). getMethods() returns the 4 java.lang.Math/abs methods in different orders on Java 6 and 7, causing invokeMatchingMethod() to pick a different one on the two JVMs:

java version "1.6.0_39"
Java(TM) SE Runtime Environment (build 1.6.0_39-b04)
Java HotSpot(TM) 64-Bit Server VM (build 20.14-b01, mixed mode)

user=> (pprint (seq (clojure.lang.Reflector/getMethods java.lang.Math 1 "abs" true)))
(#
#
#
#)
nil

java version "1.7.0_21"
Java(TM) SE Runtime Environment (build 1.7.0_21-b11)
Java HotSpot(TM) 64-Bit Server VM (build 23.21-b01, mixed mode)

user=> (pprint (seq (clojure.lang.Reflector/getMethods java.lang.Math 1 "abs" true)))
(#
#
#
#)
nil

That might be a sign of undesirable behavior in invokeMatchingMethod() that is too dependent upon the order of methods given to it.

As you mention, type hinting is good for avoiding the significant performance hit of reflection.

Answer 6

Comment made by: alexmiller

Not reproducible on 1.8.0-alpha3.

Answer 7

Comment made by: bronsa

Alex, I could reproduce using 1.8.0-master-SNAPSHOT and jdk 1.8:

[~]> java -version java version "1.8.0_45" Java(TM) SE Runtime Environment (build 1.8.0_45-b14) Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode) [~]> clj Clojure 1.8.0-master-SNAPSHOT user=> (set! *warn-on-reflection* true) true user=> (#(Math/abs %) 1000000000000) Reflection warning, NO_SOURCE_PATH:2:3 - call to static method abs on java.lang.Math can't be resolved (argument 727379968 user=> (class *1) java.lang.Integer

Andy's last comment mentions that clojure.lang.Reflector.invokeStaticMethod is dependant on the order of methods passed to it and that that order can change between jdk versions so maybe that's why you couldn't reproduce it

Answer 8

Comment made by: jafingerhut

Possibly the same as, or at least some commonality with, CLJ-1921.

Answer 9

Reference: https://clojure.atlassian.net/browse/CLJ-1212 (reported by alex+import)