Could tools.namespace refresh get more sophisticated directory/file filtering?

Wed, 05 Jan 2022 18:03:20 +0000

TL;DR:
currently the only way of affecting which source files are loaded by clojure.tools.namespace.repl/refresh is setting refresh dirs, which is an allowlist-based system. There is no way currently to block specific directories/files/patterns from being loaded but allowing everything else. This presents an issue when there are clojure source files in source directories on the classpath, which are not meant to be loaded at all.

Long story:

We bumped into an interesting issue with clojure.tools.namespace/refresh vs clj-kondo hooks .

A one-liner repro case is:

clj -Srepro -Sdeps '{:deps {org.clojure/tools.namespace {:mvn/version "1.2.0"} seancorfield/next.jdbc {:git/url "https://github.com/seancorfield/next-jdbc/" :git/sha "24bf1dbaa441d62461f980e9f880df5013f295dd"}}}' -M -e "((requiring-resolve 'clojure.tools.namespace.repl/refresh-all))"

This will fail with:

:error-while-loading hooks.com.github.seancorfield.next-jdbc
Could not locate hooks/com/github/seancorfield/next_jdbc__init.class, hooks/com/github/seancorfield/next_jdbc.clj or hooks/com/github/seancorfield/next_jdbc.cljc on classpath. Please check that namespaces with dashes use underscores in the Clojure file name.

For context, clj-kondo is a static analyzer/linter. In order to be able to analyze custom macros properly, it lets libraries distribute clj files (describing how these macros should be analyzed) as resources under a specific directory.

The above example fails due to the combination of the following:

referring a lib by git coordinates, which means that it will appear as a directory on the classpath
said lib exporting clj-kondo hooks
clj-kondo hooks themselves being .clj files
as these are for clj-kondo to use, they are not meant to be loaded in the app using the lib
as a result these files normally having a namespace inside them that does not appear in code proper, for example: https://github.com/seancorfield/next-jdbc/blob/24bf1dbaa441d62461f980e9f880df5013f295dd/resources/clj-kondo.exports/com.github.seancorfield/next.jdbc/hooks/com/github/seancorfield/next_jdbc.clj#L1
tools.namespace by default reloads all clj source files found in classpath-directories (referring back to the first point)

As there is currently no way to tell tools.namespace to not load certain files I had to come up with a somewhat hacky workaround, which tries to set the refresh dirs to classpath-directories minus problematic ones, but it would be much much nicer if there was a way to be able to set this with a blocklist or predicate.

Workaround in case anyone else bumps into it:

(defn remove-clj-kondo-exports-from-tools-ns-refresh-dirs
  "A potential issue from using this is that if the directory containing the clj-kondo.exports folder
  also directly contains to-be-reloaded clojure source files, those will no longer be reloaded."
  []
  (->> (clojure.java.classpath/classpath-directories)
       (mapcat
        (fn [^File classpath-directory]
          (let [children   (.listFiles classpath-directory)
                directory? #(.isDirectory ^File %)
                clj-kondo-exports?
                           #(= "clj-kondo.exports" (.getName ^File %))
                has-clj-kondo-exports
                           (some (every-pred clj-kondo-exports? directory?) children)]
            (if has-clj-kondo-exports
              (->> children
                   (filter directory?)
                   (remove clj-kondo-exports?))
              [classpath-directory]))))
       (apply clojure.tools.namespace.repl/set-refresh-dirs)))

;; call in user.clj
(remove-clj-kondo-exports-from-tools-ns-refresh-dirs)

Clojure Q&A - Recent questions tagged clj-kondo

Could tools.namespace refresh get more sophisticated directory/file filtering?