Clojure Q&A - Recent questions tagged direct-linking

Can direct-linking be enabled for my own code and not for libraries?

Tue, 27 Feb 2024 12:47:39 +0000

Blanketly enablind direct-linking can break libraries, and I can't control what vars in them are affected, but I'd still like to improve performance at least in parts of my own code.

Would the following work?

(binding [clojure.core/*compiler-options* 
  {:direct-linking true}]
  ,,,
  )

Here's the docs: https://clojuredocs.org/clojure.core/*compiler-options*

Finer-grained control over direct-linking

Sun, 18 Dec 2022 19:44:03 +0000

Currently, the approach to enabling direct linking (DL) is mostly all-of-nothing with pinpoint opt-out (^:redef). In my experience, it makes using DL somewhat awkward in production cases where you still want to retain the ability to jack into the application for debugging and investigation purposes. Knowing upfront which functions you would need to recompile is cumbersome and mentally taxing. In fact, I end up not enabling DL at all because of it.

I wonder if having a more flexible way to tune DL would make it more popular. Here are my thoughts pertaining to this.

Currently, the decision whether to DL is made at the callsite depending on the value of *compiler-options* at the moment of compiling that callsite. The alternative I envision is when the decision to DL is made at the function definition time, and all the future invocations of that function honor that decision.

Let me give you an example. Clojure itself compiles clojure.core namespace with DL enabled. This means that functions within clojure.core statically link to other clojure.core functions. However, if the developer has DL disabled, then the calls to clojure.core functions would go through Var resolution. I personally can't envision a scenario where this precise behavior is desired. Would someone want to redefine a clojure.core function for the code they run while being content with an old version still being used by some other code? Is this something that the language design wants to encourage?

My second point is that a blanket DL usually brings quite unsubstantial performance improvements except when used for specific functions called in tight loops that are amenable to JIT cascade optimizations due to inlining. Such performance-critical functions can be reliably identified by the developer with a help of a profiler, and then it makes sense to only enable DL for those functions. As of now, the only way to make a function that is guaranteed to be JIT-inlinable without going buying into DL wholesale is to use the semi-obscure :inline function which is just a macro, after all.

On the implementation side, this could be implemented by a combination of a special value of *compiler-options* :direct-linking (kinda similar to how *unchecked-math* can just be true or :warn-on-boxed) and a metadata on Vars (could reuse the currently noop ^:static).

TL;DR: I'd like a mode of DL when all callsites of a function compiled with DL to be directly linked automatically, and a way to explicitly mark a function to always use DL. Does anybody else find something like this useful? Would love to hear what you think.

Are protocol methods guaranteed to not be directly linked?

Mon, 23 Aug 2021 23:20:02 +0000

As shown in the following example, the protocol method, pfoo doesn't appear to be directly linked.

$ clj -J-Dclojure.compiler.direct-linking=true
Clojure 1.10.1
user=> (defprotocol Foo (pfoo [this]))
Foo
user=> (defn libfn [x] (pfoo x))
#'user/libfn
user=> (extend-protocol Foo String (pfoo [_] :dude))
nil
user=> (libfn "hello")
:dude
user=> (alter-var-root #'user/pfoo (fn [old] (fn [this] (prn :this) (old this))))
#object[user$eval178$fn__179$fn__180 0x18eec010 "user$eval178$fn__179$fn__180@18eec010"]
user=> (libfn "hello")
:this
:dude

The reason I ask is because I'm trying to find a generic way to patch protocol methods to work within sci + graal native-image. The constraints are:

works under direct-linking (important for native-image use)
use with 3rd party libraries
- avoid source code changes to target protocols (can't add ^:dynamic, ^:redef meta)

I have a solution that works here, but it relies on the fact that protocol vars can be altered using alter-var-root at compile time even under direct linking. Is that a safe assumption?