:or defaults should refer to enclosing scope in map destructuring

Question

Michael Blume noticed that :or defaults can depend on the values of other keys, see https://groups.google.com/d/msg/clojure/6kOhpPOpHWM/ITjWwQFS_VQJ

Michael's Gist https://gist.github.com/MichaelBlume/4891dafdd31f0dcbc727 displays a case where an associative form involving :keys and :or compiles or not depending on the order of symbols in :keys. By tweaking that case one can arrive at expressions which always compile, but produce different values depending on :keys:


(let [foo 1
       bar 2
       {:keys [bar foo]
        :or {foo 3 bar (inc foo)}} {}]
  {:foo foo :bar bar})
;= {:foo 3, :bar 4}

(let [foo 1
      bar 2
      {:keys [foo bar]
       :or {foo 3 bar (inc foo)}} {}]
  {:foo foo :bar bar})
;= {:foo 3, :bar 2}


I believe that the most natural solution is to demand that :or defaults be evaluated in an enclosing scope where none of the destructuring-introduced locals are present. This approach is taken by the 0001 patch.

Answer 1

Comment made by: michaelblume

I suspect that this is the right thing to do but I think it's important to note that this will break existing code https://github.com/ngrunwald/ring-middleware-format/blob/master/src/ring/middleware/format_params.clj#L214

Answer 2

Comment made by: michaelblume

Update on my previous comment -- ring-middleware-params has updated so that it doesn't depend on this behavior. I think we should definitely merge this patch so no one else depends on it.

Answer 3

Comment made by: mpenet

Since this involves :or keys evaluation, this might be worth checking if this should/could have an impact on http://dev.clojure.org/jira/browse/CLJ-1676 as well.

Answer 4

Comment made by: stu

This is a behavior change, the docs do not promise the requested behavior and existing code may depend on the current behavior.

Answer 5

Comment made by: jafingerhut

Isn't this a case where if existing code works, it works by accident of the seq order of an unordered map? If so, any code that depends upon the existing behavior sometimes breaks, sometimes does not break, when the Clojure seq order on maps changes, which occurred Clojure 1.5.1 to Clojure 1.6.0, and again from 1.6.0 to 1.7.0.

Answer 6

Comment made by: michaelblume

Yes, it does, and I've seen existing code break due to those changes, hence the discussion that lead to this ticket.

Answer 7

Comment made by: michaelblume

Updating this patch

Answer 8

_Comment made by: michalmarczyk_

@Stuart:

To substantiate what was said above, here is the same code snippet evaluated at a Clojure 1.6 REPL and then again at a Clojure 1.7 REPL, both REPLs freshly started, with different results:


Clojure 1.6.0
(let [foo 1 bar 2
      {:keys [foo bar]
       :or {foo 3 bar (inc foo)}} {}]
  [foo bar])
[3 2]

Clojure 1.7.0
(let [foo 1 bar 2
      {:keys [foo bar]
       :or {foo 3 bar (inc foo)}} {}]
  [foo bar])
[3 4]


It is certainly not promised in the docs that there will be no surprising interactions between {{:or}} and {{:keys}}, but as demonstrated above, any existing code that depended on 1.6 behaviour has already been broken by 1.7. Specifying *some* behaviour and sticking to it in the future would prevent such surprises going forward.

I also think that the current behaviour is "random" in the sense that there is no principled reason why one might expect it – hence the proposal to make {{:or}} defaults refer to the enclosing scope that I've implemented in the patch.

Answer 9

Comment made by: michaelblume

Updated patch to apply to master

Answer 10

Reference: https://clojure.atlassian.net/browse/CLJ-1613 (reported by michalmarczyk)