Welcome! Please see the About page for a little more info on how this works.

0 votes
in ClojureScript by

Dirac DevTools REPL relies on proper source-maps(link: 1). When I type "#js {}" into the REPL prompt, evaluation works, but generated javascript code snippet has wrong source map. Associated source-map's sourcesContent is something like #object(link: cljs.tagged_literals.JSValue 0x34d692f3 "cljs.tagged_literals.JSValue@34d692f3"). And it is not what user entered. And this is not a valid cljs source I could parse again for code-completions.

The problem is somewhat subtle. When generating source-map info, the REPL code given a form tries to look at :source metadata(link: 2) and if not present, it simply prints the form using pr. We get Clojure-style printing because JSValue does not implement ClojureScript printing as reported in CLJS-733.

Instead of implementing the rejected idea from CLJS-733. I focused on the first-case code path with :source metadata. It turned out that :source gets added if given form supports metadata protocols. Easiest idea was to change JSValue from deftype to defrecord.

I confirm that this fixed the REPL issue in Dirac. Now I'm getting back the original source code as entered by user.

Note that similar issue is present with #uuid and #inst tags as well. Those mostly work because their printer is compatible with both Clojure and ClojureScript. But fundamentally user might get back different source-map source than entered into the REPL. For example whitespace will be missing. This might cause confusion because line/column information reported by reader which might not match this artificial source-map content generated by printing.

I believe a fix would be to wrap UUID and Date forms created by tagged literals into something similar to JSValue and work with this wrapped value instead. Same with #queue. This would make cljs.tagged-literal code consistent, because every form originated from data readers would be inside a wrapper. It would enable arbitrary metadata on wrappers.

As a side note:
I noticed that reader's metadata get lost. They are not transplanted from form produced by the reader to generated tagged literal forms.
My second goal was to copy reader's metadata. Unfortunately this wasn't possible with current implementation of tools.reader. I can get reader metadata from passed form, but that is not the full picture, it only describes content after the tag was already consumed. I would need metadata from tag itself, passed into data-reader call somehow(link: 3).

I don't know about any real-world problem caused by missing reader metadata on tagged literals, but I think ideally it should be fixed. For example when implementing error-reporting in cljs-oops, I had to handle this edge case(link: 4), because line/column metadata was missing in some edge cases.

(link: 1) https://github.com/binaryage/dirac/releases/tag/v0.8.0
(link: 2) https://github.com/clojure/clojurescript/blob/960bb9b778190aa7359acb2f74cc61d452cef2ae/src/main/clojure/cljs/repl.cljc#L476
(link: 3) https://github.com/clojure/tools.reader/blob/3f36a18a6c5d53a4fc718822131ee75625fd44dc/src/main/cljs/cljs/tools/reader.cljs#L834
(link: 4) https://github.com/binaryage/cljs-oops/blob/d530a3cdf8cbab39bd2699c36caded4414224c50/src/lib/oops/reporting.clj#L29

3 Answers

0 votes

Comment made by: darwin

I just noticed that some tests broke because of the defrecord change. My original implementation was using deftype with clojure.lang.IObj implementation. But it had a shortcoming of reusing val as the target for holding metadata. Which might not be IObj in general case. And also a separate implementation would have to be done for bootstrapped. Also I didn't want to add another field into deftype. I will leave it open for your suggestion how to properly implement this.

0 votes

Comment made by: darwin

Another stab at it - this time I rely on the fact that val field must be a vector or a map. So it can carry metadata for JSValue without changing adding new fields to deftype.

Tests passing.

0 votes
Reference: https://clojure.atlassian.net/browse/CLJS-1898 (reported by darwin)