AReference#meta() is synchronized

Question

We use Clojure for a "rules engine". Each function represents a rule and metadata describes the rule and provides some static configuration for the rule itself. The system is immutable and concurrent.

If two or more Threads invoke the same Var concurrently they end up blocking each other because AReference#meta() is synchronized (see attached screenshot, the red dots).


(defn
  ^{:rule {:remote-address "127.0.0.1"}}
  example
  [request]
  (let [rule (:rule (meta #'example))]
    (= (:remote-address rule) (:remote-address request))))


*Approach:* Replace synchronized block with a rwlock for greater read concurrency. This approach removes meta read contention (see real world example in comments). However, it comes with the downsides of:

* extra field for every AReference (all namespaces, vars, atoms, refs, and agents)
* adds construction of lock into construction of AReference (affects perf and startup time)

*Patch:* clj-1888-2.patch replaces synchronized with a rwlock for greater read concurrency

*Alternatives:*

* Use volatile for _meta and synchronized for alter/reset. Allow read of _meta just under the volatile - would this be safe enough?
* Extend AReference from ReentrantReadWriteLock instead of holding one - this is pretty weird but would have a different (potentially better) footprint for memory/construction.

Answer 1

Comment made by: alexmiller

A volatile is not sufficient in alterMeta as you need to read/update/write atomically.

You could however use a ReadWriteLock instead of synchronized. I've attached a patch that does this - if you have a reproducible case I'd be interested to see how it affects what you see in the profiler.

There are potential issues that would need to be looked at - this will increase memory per reference (the lock instance) and slow down construction (lock construction) at the benefit of more concurrent reads.

Answer 2

_Comment made by: rkapsi_

Hey Alex,

I do have a reproducible case. The blocking has certainly disappeared after applying your patch (see attached picture). The remaining blocking code on these "WorkerThreads" is {{sun.nio.ch.SelectorImpl.select(long)}} (i.e. not clojure related).

You can repro it yourself by executing something like the code below concurrently in an infinite loop.


(defn
  ^{:rule {:remote-address "127.0.0.1"}}
  example
  [request]
  (let [rule (:rule (meta #'example))]
    (= (:remote-address rule) (:remote-address request))))


Suggestions for the patch: Make the meta lock a {{final}} field and maybe pull the read/write locks into local variables to avoid the double methods calls.


alterMeta(...)
  Lock w = _metaLock.writeLock();
  w.lock();
  try {
    // ...
  } finally {
    w.unlock();
  }
}

Answer 3

Comment made by: alexmiller

Marking pre-screened,

Answer 4

Reference: https://clojure.atlassian.net/browse/CLJ-1888 (reported by rkapsi)