Welcome! Please see the About page for a little more info on how this works.

+4 votes
in Compiler by
edited by

Teaching Old eval New Tricks

Tagged Literals Promise Extensibility

  • EDN and Tagged Literals:
    - In EDN, tagged literals deliver complete extensibility.
    - When processing EDN data, custom types created with tagged readers achieve parity with built-in literals.

  • Clojure’s Syntax and Tagged Literals:
    - Clojure's syntax, as a superset of EDN, includes tagged literals.
    - Key Differences:
    - EDN is for reading data, while Clojure's syntax is expected to be eval'ed.
    - Tagged literals in Clojure code are treated as second-class citizens compared to built-in types because:

    1. The compiler inherently knows how to compile built-in types,
       but it does not understand other types returned by tagged
       readers.
    
  • Compiler Behavior and Challenges:
    - By default:
    - The compiler embeds the instance as a string in the emitted bytecode, recreating the instance at runtime via the reader.
    - Issues with Collections:
    - Container types like third party collections are problematic because their interior data is not evaluated.
    - Common advice suggests returning a form that, when evaluated, creates an instance of the type. However:

    - This makes the result no longer a literal, as:
      - The reader does not produce the expected type.
      - Macros don't see the expected type
      - You only get the expected type after eval.
    - For mixed environments (EDN without eval vs. Clojure with
      eval), separate tagged readers for the same tag may be
      required.
    

Towards a Solution

  • Compiler Extensions:
    - A potential solution involves creating an extension point to teach the compiler how to compile new types.
    - Special care is required for cross-compiling dialects, such as ClojureScript.
    - JVM collections might need to self-compile for ClojureScript.

  • Backward Compatibility:
    - Ideally, any solution would:
    - Retain the existing "self-quoting" behavior.
    - Allow types to opt into new functionality.

by
a useful thing to search for in clojurians slack to see these issues discussed is "ordered" because of  https://github.com/clj-commons/ordered a library for insertion ordered variants of clojure collections that provides tagged literals. If you are allergic to premature solution discussion, be warned I am there doing that.
by
https://clojurians.slack.com/archives/C03S1KBA2/p1698763672356679 a long discussion thread that starts with a mali issue because it provides a tagged literal syntax for its schemas, but to work around the lack of eval, if I recal correctly, it calls eval in its tagged reader function, which can result in multiple evals

Please log in or register to answer this question.

...