Welcome! Please see the About page for a little more info on how this works.

+57 votes
in Java Interop by
closed by

Moving to Java 8 as a baseline allows us to consider built-in ties to critical java.util.Function interfaces like Function, Predicate, Supplier, etc. Needs assessment about what's possible and what automatic integration it would open for users.

https://docs.oracle.com/javase/8/docs/api/java/util/function/package-summary.html

closed with the note: Released in Clojure 1.12.0-alpha12
by
Why are we in 2023 without any official response of when is this going to be implemented or dealed?
by
We have been looking at this off and on for several releases now. There are a bunch of aspects to it and several options for how to proceed, several of which we have built prototypes for, but we have not decided yet what to do. It's in our scope list for 1.12.

13 Answers

+4 votes
by

This is what a Kafka Streams app written in Java looks like:

      sb.table("input", Consumed.with(sl, sl))
            .groupBy((k, v) -> KeyValue.pair(k / 10, v), Grouped.with(sl, sl))
            .aggregate(() -> 0L,
                    (k, v, acc) -> acc + v,
                    (k, v, acc) -> acc - v,
                    Materialized.with(sl, sl))
            .toStream()
            .to("output", Produced.with(sl, sl));

The same app written in Clojure looks like this:

    (-> sb
        (.table "input" (topic->consumed data-in))
        (.groupBy (key-value-mapper
                    (fn [k v] (KeyValue/pair (long (/ k 10)) v)))
                (serdes->grouped "groupie" data-in))
        (.aggregate (reify Initializer
                    (apply [_] 0))
                    (reify Aggregator
                    (apply [_ k v acc]
                        (+ acc v)))
                    (reify Aggregator
                    (apply [_ k v acc]
                        (- acc v)))
                    (serdes->materialised ...))
        (.toStream)
        (.to "output" (topic->produced data-out)))

If we had the ability to use Lambdas where SAM types are expected, we could write this instead:

    (-> sb
        (.table "input" (topic->consumed data-in))
        (.groupBy (fn [k v] (KeyValue/pair (long (/ k 10)) v))
                (serdes->grouped "groupie" data-in))
        (.aggregate (constantly 0)
                    (fn [k v acc] (+ acc v))
                    (fn [k v acc] (- acc v))
                    (serdes->materialised ...))
        (.toStream)
        (.to "output" (topic->produced data-out)))
by
Congrats and thank you for having implemented this in Clojure 1.12:
https://clojure.atlassian.net/browse/CLJ-2799
+3 votes
by

Comment made by: jwhitlark

If I could just use IFn's anywhere a java.util.function.* was needed, that would be fantastic!

+2 votes
by

Comment made by: marctrem

Moving to Java 8 as a baseline allows us to use default interface methods.

The some-java-fns-interface.patch patch implements Consumer, Function, Predicate and Supplier on IFn.

If you want to go this route, I would be very happy to implement all interfaces under java.util.function on IFn as well as the accompanying tests. I currently use this code to play with FoundationDB through its Java client and it works well for me.

https://github.com/marctrem/clojure/commit/97742493f674edd8f6c034ee94da84fa62a76bad

+2 votes
by

Someone made a patch that addresses this in the best way I've seen yet, which is simply that it does exactly what Java does for lambdas, but it does it for Clojure FN

https://clojure.atlassian.net/plugins/servlet/mobile?originPath=/browse/CLJ-2637#issue/CLJ-2637

by
Oops we better allocate a "question" for CLJ-2637 lest the comments here get all tangled up between the two approaches
+1 vote
by

Having just bumped into this, I thought I'd provide some usage/context information.

When interfacing with async Java code written using CompletableFuture and CompletionStage, one needs to provide arguments that implement Function, Consumer, BiFunction and others.

I'm using macros like these:

(defmacro as-function [f]
  `(reify java.util.function.Function
     (apply [this arg#]
       (~f arg#))))

(defmacro as-consumer [f]
  `(reify java.util.function.Consumer
     (accept [this arg#]
       (~f arg#))))

but this quickly becomes tedious, as variants are needed depending on function arguments.

java.util.function defines lots of interfaces, but from my (limited) point of view, I'd need the most common ones, notably those required by CompletionStage and CompletableFuture.

by
Additionally just basic java interfaces like java.util.Map.  In order to use a concurrent hash map in the most efficient manner you need to use compute, computeIfPresent, computeIfAbsent - (https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ConcurrentHashMap.html#compute-K-java.util.function.BiFunction-).
+1 vote
by

Request for data

Setting aside java.util.stream.Stream, and retrofits of existing JDK APIs like new Thread(() -> doSomething(x)), what are examples of lambda/SAM-using libraries that are awkward to consume from Clojure?

0 votes
by
_Comment made by: jwhitlark_


;; I dug this out of some scratch code experimenting with kafka streams.  All the reify's were filled with java8 lambdas in the original.

;; I'll dig up another example that shows examples using stuff from java.utils.funstion.*

;;Some of this was lifted from a franzy example or something?

;; Note that, for example,
;; https://kafka.apache.org/0102/javadoc/org/apache/kafka/streams/kstream/Predicate.html
;; is different from
;; https://docs.oracle.com/javase/8/docs/api/java/util/function/Predicate.html

(ns utils
  (:import (org.apache.kafka.streams.kstream Reducer KeyValueMapper ValueMapper Predicate))

(defmacro reducer [kv & body]
  `(reify Reducer
     (apply [_# ~(first kv) ~(second kv)]
       ~@body)))

;; public interface KeyValueMapper<K,V,R>
;; apply(K key, V value)
(defmacro kv-mapper [kv & body]
  `(reify KeyValueMapper
     (apply [_# ~(first kv) ~(second kv)]
       ~@body)))

;; public interface ValueMapper<V1,V2>
;; apply(V1 value)
(defmacro v-mapper [v & body]
  `(reify ValueMapper
     (apply [_# ~v]
       ~@body)))

(defmacro pred [kv & body]
  `(reify Predicate
     (test [_# ~(first kv) ~(second kv)]
       ~@body)))

;; I used it something like this:

(ns our-service.kafka-streams
  (:require
   [our-service.util :as k]
   [clojure.string :as str]
  (:import
           (org.apache.kafka.streams StreamsConfig KafkaStreams KeyValue)
           (org.apache.kafka.streams.kstream KStreamBuilder ValueMapper)))

(defn create-word-count-topology []
  (let [builder (KStreamBuilder.)
        init-stream (.stream builder (into-array ["streams-str-input"]))
        wc (-> init-stream
            (.flatMapValues (k/v-mapper [& value]
                                        (str/split (apply str value) #"\s")))
            (.map (k/kv-mapper [k v]
                               (KeyValue/pair v v)))
            (.filter (k/pred [k v]
                             (println v)
                             (not= v "the")))
            (.groupByKey)
            (.count "CountStore")
            show-item
            ;; this needs to be mapValues
            (.mapValues (reify ValueMapper
                          (apply [_ v]
                            (println v)
                            (str v))))
            (.toStream)
            (.to "wordcount-output"))]
    [builder wc]))
0 votes
by

Comment made by: gshayban

The JLS infers lambda types by hunting for matching functional interfaces AKA "Single Abstract Method" classes (link: 1) (whether they're interfaces or abstract classes.) We could have a reify-like helper that detects these classes (link: 2). You would have to hint the target class. We don't really need things that are both IFn and j.u.f.Predicate, etc.

`
(import '[java.util.function Predicate Consumer])

(let [orig [1 2 3]

  st (atom [])]

(.forEach orig (jfn Consumer [x] (swap! st conj x)))
(= @st orig))
`

(link: 1) https://docs.oracle.com/javase/specs/jls/se8/html/jls-9.html#jls-9.8
(link: 2) spike https://gist.github.com/ghadishayban/0ac41e81d4df02ff176c22d16ee8b972

0 votes
by

Comment made by: jwhitlark

Well, that would be an improvement. The practical problem I run into is that I'm frequently deep in a fluent interface, and don't necessarily know the exact class. That said, it's usually only in a few places. Would it make sense to have a registry? Perhaps something like:

(auto-infer-lambda (link: java.util.function, org.apache.kafka.streams.kstream))

0 votes
by

Comment made by: gshayban

Do you ever use SAM classes that are abstract classes and not interfaces?

0 votes
by

Comment made by: ajoberstar

Here's an alternative approach in my (link: https://github.com/ajoberstar/ike.cljj/blob/master/src/main/clojure/ike/cljj/function.clj text: ike.cljj) library. This uses MethodHandles (i.e. java.lang.invoke package) instead of regular reflection. I'm not sure if I tested this on abstract classes yet.

The usage looks similar to what Ghadi posted:

`
(defsam my-sam
java.util.function.Predicate
[x]
(= x "it matched"))

(-> (Stream/of "not a match" "it matched")

(.filter my-sam)
(.collect Collectors/toList)

(-> (IntStream/range 0 10)

(.filter (sam* java.util.function.IntPredicate odd?))
(.collect Collectors/toList)

`

It uses (link: https://docs.oracle.com/javase/8/docs/api/java/lang/invoke/MethodHandleProxies.html#asInterfaceInstance-java.lang.Class-java.lang.invoke.MethodHandle- text: MethodHandleProxies.asInterfaceInstance) to create a proxy instance of the interface that calls a method handle calling a Clojure function. It doesn't try to validate parameter counts, it just treats it as varargs delegating to IFn.applyTo(ISeq). Not sure if it's the most efficient but it was effective for my needs.

I think the (link: https://docs.oracle.com/javase/8/docs/api/index.html?java/lang/invoke/MethodHandles.html text: LambdaMetaFactory) may be the preferred way to meet this type of use case. It was harder for me to follow exactly how to use that though so I didn't end up looking to deep into it.

The main functional issue I have with my approach (and Ghadi's) is that you have to explcitly provide the interface you want to proxy. Java lambdas and Groovy Closures can be used against methods that expect a SAM and it just coerces based on what the method expects. Ideally this would be supported by Clojure too.

Instead of having to do:

`
(-> (IntStream/range 0 10)

(.filter (sam* java.util.function.IntPredicate odd?))
(.collect Collectors/toList)

`

I'd like to do this:

`
(-> (IntStream/range 0 10)

(.filter odd?)
(.collect Collectors/toList)

`

0 votes
by

Comment made by: gshayban

Another possible avenue is to extend java.util.function.Supplier to Clojure functions with an explicit 0-arity. That interface is becoming increasingly common in practice; it might be valuable to special-case it. (We shouldn't & couldn't do a similar thing for defrecords, as they already have a method called get, which clashes with Supplier's only method.)

0 votes
by
Reference: https://clojure.atlassian.net/browse/CLJ-2365 (reported by alexmiller)
...