Why different symbol handling by tagged literals vs elements (reader vs edn)

Question

The reader reference docs state "tagged literals are Clojure’s implementation of edn tagged elements."

But the two are different in at least one regard: a symbol produced by a tagged literal will be resolved, but a symbol produced by a tagged element and read by clojure.edn/read-string will not.

This means you can get different output if you read EDN containing a given tag vs. if you use the same tag to produce a literal in your source.

Example:

data_readers.cj

{example/symbolize clojure.core/symbol}

Then:

user> (edn/read-string {:readers {(quote example/symbolize) clojure.core/symbol}}
                       "#example/symbolize\"hello\"")   
hello
user> #example/symbolize "hello"
Syntax error compiling at (*cider-repl clojure/foragr:localhost:46533(clj)*:0:0).
Unable to resolve symbol: hello in this context
user>

Is there any particular reason for this discrepency? Is there a scenario where it would be good to resolve symbols from a tagged literal? And if so, why not do the same from tagged elements and edn/read-string?

Answer 1

The first one is just read.
The second is read, then eval (because R E PL)

So, no discrepancy. The REPL uses the same read in both cases. If you want read without eval, then read without eval, for example by quoting:

'#example/symbolize "hello"

Comments

?

The repl seems orthogonal here. You can see it did not eval the output of `edn/read-string` even though it’s a symbol. I also tested the literal in source of a clj file before I posted and the result is the same.

---

Source files are also read and evaluated, so it's entirely relevant.

---

Some additional background here: https://clojure.org/guides/learn/syntax#_evaluation

---

Thanks for replying on a Sunday. My takeaway is that tagged literals are eval-ed and thus work substantially differently from tagged elements. For whatever reason I did not get this at all from the docs as written.

Now I understand that it’s quite a different interface for tagged literals vs tagged elements. I can go out of my way to tell people “if you use this tag I made in edn and read it with my library it works like <this> but if you try to use that literal in your source you need to quote it like <that>”. I just think it’s suboptimal that the two interfaces are that different but I guess that probably goes without saying. I’m sure there are use cases I have never considered.

Thanks again.

---

Put another way - despite the word “read”, with `edn/read-string` one always ends up with a data structure (or object), full stop. It’s read in but only to data - a true reader read would produce symbols for immediate automatic evaluation (right?) and run eval per ` clojure.core/read` not just give you data?

I understand this from a safety perspective but you end up with different interface than tagged literals. My naive expectation since edn is expressly cited in the reader reference that discusses tagged literals is that the two would “read” in the same sense particularly since the potential upside from eval-ing tagged literals seems so minor that one would tend to choose interface consistency.  But this is where I go back to saying I’m sure there are many use cases I have not considered. Not complaining here just explaining why this was a speed bump to me.

---

No, tagged literal literals and tagged elements are the same thing. They are a read-time construct. Whether they are evaluated depends on the context. Code at the repl and in source files is read and evaluated. read and read-string will read (but not eval). Quoting is another technique to read but not eval. There is no difference here - it's all how you use it.

---

Ya maybe we are talking past each other at the end of the weekend here.

What I’m saying is if you use a given tag X linked to function Y in an edn file and read it in and give it to `edn/read-string` you get unevaluated symbols. If you  use that exact X/Y combo in a tagged literal you get evaluated symbols. Basically what my repl session above shows.

That’s two different interfaces from where I sit but I perhaps we have different terminology here. “What you get depends on the context” -I’m with you there.  I realize if I use ` clojure.core/read-string` I’d get the other behavior. I’m not complaining about the difference just noting it.

I don’t expect you’ll “agree” with what I’ve said above or not btw, I’ve taken enough of your time please enjoy the rest of the weekend and thanks again.

Answer 2

Alex is correct and I think you're misunderstanding his answer. See if this REPL session convinces you:

(~/clojure)-(!2003)-> cat src/data_readers.clj
{example/symbolize clojure.core/symbol}

Sun Aug 27 17:02:45
(~/clojure)-(!2004)-> clj
Clojure 1.12.0-alpha4
user=> (require '[clojure.edn :as edn])
nil
user=> (edn/read-string {:readers {(quote example/symbolize) clojure.core/symbol}}
                       "#example/symbolize\"hello\"")
hello ; edn/read-string produces a symbol
user=> (read-string "#example/symbolize\"hello\"")
hello ; core/read-string also produces a symbol
user=> hello ; this symbol is not bound to anything
Syntax error compiling at (REPL:0:0).
Unable to resolve symbol: hello in this context
user=> #example/symbolize "hello"
;; Read (produces symbol hello) Eval (tries to lookup the symbol's value and fails)
Syntax error compiling at (REPL:0:0).
Unable to resolve symbol: hello in this context
user=> #example/symbolize "\"hello\""
Syntax error compiling at (REPL:0:0).
;; Note the subtle difference in the error: Read produced a SYMBOL again, spelled "hello"
;; and there's no bound "hello" symbol either
Unable to resolve symbol: "hello" in this context
user=>

Comments

Hi Sean, Thanks for this.

I think the distinction I've been missing is that (of course, and I should have known this as they're documented in the clojure reader reference) tagged literals apply at read time, so of course they will be evaled - it's not possible to just leave them as data. Whereas tagged elements -- in the context of an edn file you're reading -- will be consumed at runtime through an explicit read call and so won't be evaled unless you go out of your way to do so.

I think seeing your clojure.core/read-string example helped crystalize this so thanks!

(I had a longer more confused comment here which I'm replacing with this!)

---

The only comment I'll make here is that they are all "tagged literals". The reader -- whether it's the EDN reader or the Clojure reader -- will read the form that follows the tag (so it must be valid EDN or Clojure data) and then invoke the specified function on that form: purely symbolic evaluation as part of the reading process.

If the context is such that a form will be evaluated after being read, then the symbolic form produced by the reader will be evaluated.

This isn't specific to tagged literals. The readers turn text into (EDN or Clojure) forms. The resulting forms may be evaluated (if you are in the REPL or loading a source file for a namespace, or specifically evaluated a form from your editor).

In many ways, this is similar to how macros work: the reader turns text into forms, the symbolic forms are passed to the macro (function) which returns a new symbolic form, and that new form is evaluated if the context requires it.