Welcome! Please see the About page for a little more info on how this works.

+1 vote
in Clojure by
recategorized by

I recently thought that I ran into a bug in clojure.spec because it threw an exception when I put a lambda like (fn [x] x) or #(identity %) into the -> threading macro without wrapping it first in an extra parens like ((fn [x] x)). This is the most common repeated "mistake" I've made in Clojure over the years, and I've heard from others that it's a common misconception about the threading macros. This "mistake" in my case is based on my assumption that the lambda is a fundamental "thing" or value in the language. But, in fact, Clojure is just data, mostly macros, and each macro handles its input data in different ways.

The -> threading macro treats a lambda as a list instead of as a fundamental "function value." When one does (-> :hmm (fn [x] x)) it is expanded like: (fn :hmm [x] x) which obviously is not what is intended. However, it does have a special convenience case for a non-list thing like a symbol, keyword, etc, which it assumes is something that can behave as a function with one argument, and wraps it in parens before threading. This special case adds value because it interprets the intention of the user and avoids forcing them to add parens when they're unnecessary.

I assert that the basic lambda forms (fn) and #() should and easily can be handled similarly to a symbol, by interpreting the intention of the user. The value from doing so would make the threading macros simpler and more intuitive to use, would remove an unnecessary exception that affects numerous users, and would advance the language towards treating the lambda forms as core "values." This last point is a philosophical one, but important in my opinion for making the language "just work" the way a new or non-expert user expects when getting stuff done.

The existing special case of wrapping a non-list form already demonstrates the value of providing some interpretation of the intention of the user, but there now exists an inconsistency between this special case, and the absence of it for two of the most common forms of arguably one of the most foundational "values" in the language, the function itself.

Have you been bitten by this "mistake"?

If you'd like to try it side by side with the existing -> macro, here's a version called t->:

(defmacro t->
  "Threads the expr through the forms. Inserts x as the
  second item in the first form, making a list of it if it is a lambda or not a
  list already. If there are more forms, inserts the first form as the
  second item in second form, etc."
  {:added "1.0"}
  [x & forms]
  (loop [x x, forms forms]
    (if forms
      (let [form (first forms)
            threaded (if (and (seq? form) (not (#{'fn 'fn*} (first form))))
                       (with-meta `(~(first form) ~x ~@(next form)) (meta form))
                       (list form x))]
        (recur threaded (next forms)))
      x)))

I've prepared a patch for -> and ->> that handles this use case and has no breaking impact that I can conceive of, keeping the existing behavior, while adding the new. It passes all of the core Clojure tests, but I have not yet tested it on outside code beyond my own. That would obviously be necessary before it could be considered a possible change in core.

From 686831062a574486413022af31e8c7a07b78cd24 Mon Sep 17 00:00:00 2001
From: Thomas Spellman <thos37@gmail.com>
Date: Mon, 13 Jan 2020 20:39:45 -0800
Subject: [PATCH] thread macros

---
 src/clj/clojure/core.clj             | 12 ++++++------
 test/clojure/test_clojure/macros.clj | 12 ++++++++++++
 2 files changed, 18 insertions(+), 6 deletions(-)

diff --git a/src/clj/clojure/core.clj b/src/clj/clojure/core.clj
index 8e98e072..fe43289b 100644
--- a/src/clj/clojure/core.clj
+++ b/src/clj/clojure/core.clj
@@ -1670,42 +1670,42 @@
   (. (. System (getProperties)) (get \"os.name\"))
 
   but is easier to write, read, and understand."
   {:added "1.0"}
   ([x form] `(. ~x ~form))
   ([x form & more] `(.. (. ~x ~form) ~@more)))
 
 (defmacro ->
-  "Threads the expr through the forms. Inserts x as the
-  second item in the first form, making a list of it if it is not a
+  "Threads the expr through the forms. Inserts x as the second item
+  in the first form, making a list of it if it is a lambda or not a
   list already. If there are more forms, inserts the first form as the
   second item in second form, etc."
   {:added "1.0"}
   [x & forms]
   (loop [x x, forms forms]
     (if forms
       (let [form (first forms)
-            threaded (if (seq? form)
+            threaded (if (and (seq? form) (not (#{'fn 'fn*} (first form))))
                        (with-meta `(~(first form) ~x ~@(next form)) (meta form))
                        (list form x))]
         (recur threaded (next forms)))
       x)))
 
 (defmacro ->>
-  "Threads the expr through the forms. Inserts x as the
-  last item in the first form, making a list of it if it is not a
+  "Threads the expr through the forms. Inserts x as the last item
+  in the first form, making a list of it if it is a lambda or not a
   list already. If there are more forms, inserts the first form as the
   last item in second form, etc."
   {:added "1.1"}
   [x & forms]
   (loop [x x, forms forms]
     (if forms
       (let [form (first forms)
-            threaded (if (seq? form)
+            threaded (if (and (seq? form) (not (#{'fn 'fn*} (first form))))
               (with-meta `(~(first form) ~@(next form)  ~x) (meta form))
               (list form x))]
         (recur threaded (next forms)))
       x)))
 
 (def map)
 
 (defn ^:private check-valid-options
diff --git a/test/clojure/test_clojure/macros.clj b/test/clojure/test_clojure/macros.clj
index ce17bb38..9fb1fa9e 100644
--- a/test/clojure/test_clojure/macros.clj
+++ b/test/clojure/test_clojure/macros.clj
@@ -106,8 +106,20 @@
   (is (nil? (loop []
               (as-> 0 x
                 (when-not (zero? x)
                   (recur))))))
   (is (nil? (loop [x nil] (some-> x recur))))
   (is (nil? (loop [x nil] (some->> x recur))))
   (is (= 0 (loop [x 0] (cond-> x false recur))))
   (is (= 0 (loop [x 0] (cond->> x false recur)))))
+
+(deftest ->lambda-test
+  (is (= 'a (-> 'a ((fn [x] x)))))
+  (is (= 'a (-> 'a (fn [x] x))))
+  (is (= 'a (-> 'a #(identity %))))
+  (is (= 'a (-> 'a (#(identity %))))))
+
+(deftest ->>lambda-test
+  (is (= 'a (->> 'a ((fn [x] x)))))
+  (is (= 'a (->> 'a (fn [x] x))))
+  (is (= 'a (->> 'a #(identity %))))
+  (is (= 'a (->> 'a (#(identity %))))))
-- 
2.21.0 (Apple Git-122.2)

Logged: https://clojure.atlassian.net/browse/CLJ-2553

2 Answers

+1 vote
by

This is a list.

(fn [x] x)

This is a reader macro that evaluates to an anonymous function form, also a list:

#(identity %)

;;becomes
(fn* [p1__152#] (identity p1__152#))

The expansion for #() happens at read time, so it's transforming structure of the code (the form following #) into an s-expression. So, it's like a shortcut (e.g. syntactic sugar) that expands into a common Clojure form.

->

Is a macro that operates on s-expressions. Its purpose is to restructure s-expressions so that we can write more linear, pipeline versions of the typical inside-out nested s-expression flow for complex expressions. This is very handy in general.

Your proposal changes the fundamental behavior of -> since it now no longer operates on mere structural forms (aside from elevating non-list forms to singleton lists), we have new implicit rules. In fact, there are many times where I "want" to structurally modify lists (including ones that define function forms like (fn [] blah) directly, e.g. when writing macros or other meta programming. It's both more general and simpler to restrict the behavior of -> to purely structural modification of lists, and leave any special higher rules up to the caller (e.g. you need to wrap functions in an extra () if you want them to be invoked, or come up with another means to do so).

The good news is....it's not hard to deviate from -> and write your own variant for your own needs. This is the perfect use case for a macro in a library. Just like there are variants of -> that include some notion of continuity/failure, like some->, you could easily define your own lift-functions-> macro or something that would implement the semantics you described. This solution clears up the semantics for your use (and possibly others if they opt in) while retaining the simplicity and generality of the original macro. I think this is a great example of leveraging macros to extend the language to suit your needs.

I do not think the proposed semantic gain is worth the deviation in consistency and simplicity, since you're applying more magic to infer what the expansions of reader macros and fn forms mean. I'd rather just manipulate lists, and where necessary, write macros to implement custom semantics. I think the existing -> and ->> style accomplishes list manipulation in the simplest, least-magic way possible.

0 votes
by

Would you post an example of the thing that did not work as you expected? Anonymous functions can be very useful with -> and ->>.

The threading macros "thread" a running result through a bunch of forms at a certain spot, first argument or last argument. So, to start with an example that involves a function represented by a symbol:

user=> (-> 42 (+ 1))
43

We can see exactly how it works, by looking at the macro expansion:

user=> (macroexpand-1 '(-> 42 (+ 1)))
(+ 42 1)

Going back to the threading, we can use (fn...) in place of + by the rule of referential transparency:

user=> (-> 42 ((fn [a b] (+ a b)) 1))
43

This example makes -> thread the running value into the invocation of the anonymous function, which we can see more clearly in the macro expansion:

user=> (macroexpand-1 '(-> 42 ((fn plus [a b] (+ a b)) 1)))
((fn plus [a b] (+ a b)) 42 1)

I find one particular idiom with anonymous functions inside -> or ->> very useful, now and then: to log the running value. I can insert something like the following between two steps:

...
            ((fn [x] (println "running value:" x) x)) 
...
by
edited by
Hi @pbwolf, thanks for your response.  I've updated my original post with more clarification.  I agree with all your ideas of why anonymous functions (what I call lambdas) are useful and important within the threading macros.  My point is simply that it should be unnecessary to have to double wrap them in parens for a lambda with one argument.  For example, one of your examples:

    (-> 42 ((fn [a b] (+ a b)) 1))

That takes 2 arguments and so would be necessary to wrap in parens.  However, regarding your second example:

    (-> "hello" ((fn [x] (println "running value:" x) x)))

My suggestion is that with a small modification to the -> macro, you could simply do:

    (t-> "hello" (fn [x] (println "running value:" x) x))

Another example:

    (t-> {:some :data} #(assoc % :some-other :data))

In other words it makes working with the threading macro more intuitive and enjoyable and eliminates a common error caused by a common "mistake" that I think should not be treated as a mistake, but rather as a reasonable use case.

Please try it out and see what you think:

    (defmacro t->
      "Threads the expr through the forms. Inserts x as the second item
      in the first form, making a list of it if it is a lambda or not a
      list already. If there are more forms, inserts the first form as the
      second item in second form, etc."
      {:added "1.0"}
      [x & forms]
      (loop [x x, forms forms]
        (if forms
          (let [form (first forms)
                threaded (if (and (seq? form) (not (#{'fn 'fn*} (first form))))
                           (with-meta `(~(first form) ~x ~@(next form)) (meta form))
                           (list form x))]
            (recur threaded (next forms)))
          x)))
by
I myself have delusionally put #(...) into a threading form.  I have some sympathy for the cause.  But, by the light of day, I even more strongly cherish the (objective) simplicity of threading macros that apply the same pattern to every form regardless of its contents.  Also, someday, I will make my own "enhanced" fn form, and I will get offended if the core threading macros demote it to second class by favoring core's fn and fn* only.
by
I think it's not delusional to expect that a core macro would have a special case for two core function forms of arguably the most core thing -> the function.  It's funny that you're protective of your hypothetical future possibility of being resentful of your theoretical non-core enhanced fn being deprived of a core special case!  That's great!  But really, when you're that much of a power coder, I'm sure you'll already have an enhanced threading macro to handle it too!  

My main point is that this change actually makes the use of the language **simpler**, especially for new-comers and non-experts, and it deprives no one of something they already have, it breaks nothing, and it only *adds value!*
by
edited by
> My main point is that this change actually makes the use of the language **simpler**,

I disagree, it may make it easier or more convenient in most situations, but this is a more complex threading macro, and not simpler. To my point, which of the two can be used to implement the other? The answer reveals which one is simpler.

Having said that, making it more convenient in this case could be a good thing, and warrant the additional complexity. I doubt core would embrace it though, so I'd just roll your own if I was you.
by
edited by
Didier A. wrote: > I disagree, it may make it easier or more convenient in most situations, but this is a more complex threading macro, and not simpler

We are referring to two different kinds of "simple" and "easy."  I'm referring to simpler usage and you're referring to simpler implementation.  I'm aiming for simpler (and easier in my opinion) usage, which makes the implementation more complex.  You're aiming for simpler implementation which makes usage harder and more complex.

I appreciate that you think it might be worth it to add a bit of complexity to make usage simpler.  I agree that it's a grey area.  I think that each case should be analyzed individually, and in this particular case it's worth doing.  However, I think the biggest down side to making this change would be a lack of backwards compatibility with older versions of Clojure.  But is this a significant down-side?
...