Preventing `clojure.tools.namespace.repl/refresh` from loading namespaces that haven't been loaded before

Question

Two issues that I see in the current implementation of clojure.tools.namespace.repl/refresh:

1. Upon the very first call, it reloads absolutely all Clojure files on the classpath. Its docstring hints at it, but it doesn't explain for why such a behavior is needed. At the very least, it makes the first refresh call much slower than needed.

2. It will attempt to (re)load even the files that haven't been loaded before. A very real situation - I have some Clojure files on classpath that require dependencies that aren't available. That alone makes it impossible to use refresh. Perhaps, it can be altered to avoid loading files that haven't been loaded before.

Currently, I'm using these workarounds:

(alter-var-root #'clojure.tools.namespace.repl/refresh-tracker
                assoc :clojure.tools.namespace.dir/time (System/currentTimeMillis))

(alter-var-root #'clojure.tools.namespace.repl/remove-disabled
                (fn [orig-remove-disabled]
                  (fn [tracker]
                    (let [filter-loaded #(filter find-ns %)]
                      (-> tracker
                          orig-remove-disabled
                          (update :clojure.tools.namespace.track/unload filter-loaded)
                          (update :clojure.tools.namespace.track/load filter-loaded))))))

Of course, the workaround assumes that each files has a proper ns form and doesn't create other namespaces. But seems like this assumption is also made by tools.namespace, namely by the usage of clojure.tools.namespace.file/read-file-ns-decl.

Answer 1

Upon the very first call, it reloads absolutely all Clojure files on the classpath.

Correction: it doesn't reload ever ns in the classpath. It is limited to directories (as opposed to .clj files contained in .jar files, which also form part of the classpath).

Anyway, that's its behavior when a user hasn't performed set-refresh-dirs. tools.namespace cannot make a best guess of what the refresh-dirs value should be, so the best practice is to set it oneself intentfully in advance.

This is reflected in Sierra's reloaded template.

At the very least, it makes the first refresh call much slower than needed.

tools.namespace is aimed at programmers seeking an automated code-reloading, project-oriented workflow.

It assumes that you want to work with your project (as opposed to with an arbitrary set of scripts), and cannot guess which project namespace will be necessary first, or what will you do first (run tests, start a server?). So it loads every ns.

It's worth noting that a REPL session can last days, if not weeks. If something goes wrong, you can (clear) and start over.

I have some Clojure files on classpath that require dependencies that aren't available. That alone makes it impossible to use refresh.

This issue is entirely gone when using set-refresh-dirs.

Of course, the workaround assumes that each files has a proper ns form and doesn't create other namespaces.

That is a good assumption to do for every ns that lives under src, test. Arbitrary tooling can break on namespaces that fail to follow this de-facto standard.

Again, this is why set-refresh-dirs exists - to distinguish the 'main' source paths (src/, test/) from other directories containing examples, scripts, drafts, and such. It in fact dangerous to perform require over arbitrary namespaces in a given project (as those can be production-oriented scripts etc).

Comments

To be clear, I welcome every approach to productivity and I happen to know what kind of workflow you were seeking (as I was around when this q was shared in Clojurians Slack).

If performing a t.n modification accomplishes so, that's great!

But the way this question is posted is overassuming and can easily leave a misleading trail of information, which is why I wanted to reflect better why some people can be compelled to use tools.namespace as designed.

Probably you can find more info on t.n's rationale in Sierra's writing or talks.

---

> tools.namespace is aimed at programmers seeking an automated code-reloading, project-oriented workflow.

My workaround fixes the slow code loading. And it still preserves automated code-**re**loading just fine.

> This issue is entirely gone when using set-refresh-dirs.

It's not gone at all if the two files are in the same dir.

---

> It's not gone at all if the two files are in the same dir.

It is poor practice to place non-loadable namespace under `src/` or `test/`. It directly affects other users and tools.

If you have non-loadable namespaces, it's better to place them under `examples/`, `drafts/`  or such.