Clojure Q&A - Recent questions in tools.namespace

tools.namespace remove-node does not remove outgoing dependencies from :dependents

Mon, 20 Jan 2025 02:32:44 +0000

Problem

I noticed that when a dependency, example.one, was removed from another namespace, example.two, example.one still showed up under [::track/deps :dependents 'example.two] in the tracker after an invocation of repl/scan.

After digging into this for some time, I saw that the implementation of remove-node in MapDependencyGraph didn't make any changes to the :dependents attribute.

This seems like a mistake to me, as the remove-node docstring reads:

Removes the node from the dependency graph without removing it as a
dependency of other nodes. That is, removes all outgoing edges from
node.

The implementation of remove-node follows:

(remove-node [graph node]
  (MapDependencyGraph.
    (dissoc dependencies node)
    dependents))

This removes outgoing edges from dependencies, but if dependents is effectively an inverse of dependencies, then I'd expect to see some sort of removal from dependents as well. Instead, dependents remains unmodified.

Proposed Fix

If this is indeed a bug, and not a misunderstanding on my part, I have a fix here along with a couple tests on my behavioral-tests branch that removes node from values in the dependents map.

Does this align with the expected behavior of remove-node, or am I misunderstanding the intended behavior of this function?

tools.namespace no longer returns declared metadata on namespace

Sat, 27 May 2023 20:35:10 +0000

Problem

v1.4.0 of tools.namespace returns metadata from namespace declarations. Newer versions do not.

Reproduction

Given ./src/foo.clj:

(ns ^{:doc "foo" :no-doc true} foo)

And ./test.clj:

(require '[clojure.tools.namespace.find :as f]
         '[clojure.java.io :as io])

(binding [*print-meta* true]
  (pr (f/find-ns-decls-in-dir (io/file "src"))))

If I use v1.4.0, I get the foo ns :doc and :no-doc metadata:

❯ clj -Sdeps '{:deps {org.clojure/tools.namespace {:mvn/version "1.4.0"}}}' -M test.clj
((ns ^{:doc "foo", :no-doc true} foo))

But if I use v1.4.4, the ns metadata includes the new generated :dir and :file metadata from tools.namespace, but not the :doc and :no-doc metadata:

❯ clj -Sdeps '{:deps {org.clojure/tools.namespace {:mvn/version "1.4.4"}}}' -M test.clj
((ns ^{:dir "src", :file "foo.clj"} foo))

This has been true since v1.4.1 when the jar/dir and file metadata feature was introduced:

❯ clj -Sdeps '{:deps {org.clojure/tools.namespace {:mvn/version "1.4.1"}}}' -M test.clj
((ns ^{:dir "src", :file "foo.clj"} foo))

Additional Info

The same issue should exhibit for find-ns-decls-in-jarfile.

Usage Scenario

Cljdoc uses tools.namespace to learn about namespaces before attempting to load them. It skips namespaces that have :no-doc and/or :skip-wiki metadata.

clojure.tools.namespace throws casting error when calling find-ns-decls-in-jarfile

Tue, 07 Mar 2023 17:15:20 +0000

If I call find-ns-decls-in-jarfile jarfile (example below) with any jar I get the exception:

Error printing return value (ClassCastException) at clojure.tools.namespace.find/read-ns-decl-from-jarfile-entry (find.clj:158).
class java.util.jar.JarFile cannot be cast to class java.io.File (java.util.jar.JarFile and java.io.File are in module java.base of loader 'bootstrap')

I can reproduce this with this example:


    (let [filename (format "%s/.m2/repository/org/clojure/tools.namespace/1.4.2/tools.namespace-1.4.2.jar" (System/getenv "HOME"))
      jarfile (JarFile. (io/file filename))]
  (clojure.tools.namespace.find/find-ns-decls-in-jarfile jarfile))

I think the issue is the type hint on line 159. I think this should instead be ^java.util.jar.JarFile.

How to restrict namespaces reloaded by tools.namespace to only dependencies of a root ns?

Fri, 03 Mar 2023 20:17:19 +0000

Hey,

At my organisation, we use Component and would like to start using the reloaded workflow to reload the code more easily.

The problem we’re facing is that our dev classpath contains too much. In addition to the app proper (loaded by (require 'our-org.repl)), there are tests and some one-off code: migrations and illustrative REPL sessions that we keep around as documentation.

Some of these auxiliary namespaces have names that don’t match their location in the filesystem, so they are not loaded correctly by tools.namespace. We’d like to prevent them from being reloaded at all, even though they are present on the classpath.

We could use clojure.tools.namespace.repl/set-refresh-dirs, but I feel this is clunky (need to explicitly opt-in for directories) and not quite expressive enough. What we’d really want is to reload only the namespaces that are (possibly indirect) dependencies of our-org.repl.

To this end, I’ve monkey-patched tools-namespace:

(defn relevant-deps [deps ns]
  (conj
   (ns.dependency/transitive-dependencies deps ns)
   ns))

(alter-var-root
 #'ns.repl/remove-disabled
 (fn [orig-impl]
   (fn [{:keys [::ns.track/deps] :as tracker}]
     (-> tracker
         (orig-impl)
         (update ::ns.track/unload (partial filter (loaded-libs)))
         (update ::ns.track/load
                 #(set/intersection (set %1) %2)
                 (relevant-deps deps 'our-org.repl))))))

This works, but is obviously kludgy. Is there a better way to achieve the same goal? Perhaps the functionality of restricting dependencies could be added to tools.namespace?

Should tools.namespace find still cope with namespace-less sources?

Tue, 21 Feb 2023 20:17:47 +0000

Issue
Tools.namespace find 1.4.1 does not handle sources that don't include a ns declaration.

Reproduction
Given file ./src/foo.clj:

;; no ns declaration here

And test script ./test.clj:

(require '[clojure.tools.namespace.find :as f]
         '[clojure.java.io :as io])

(println "found nses:" (f/find-ns-decls-in-dir (io/file "src")))

We can see that tools.namespace 1.4.0 returns an empty sequence for this scenario:

$ clj -Sdeps '{:deps {org.clojure/tools.namespace {:mvn/version "1.4.0"}}}' -M --report stderr test.clj
found nses: ()

But version 1.4.1 throws an exception for this same test:

$ clj -Sdeps '{:deps {org.clojure/tools.namespace {:mvn/version "1.4.1"}}}' -M --report stderr test.clj
{:clojure.main/message
 "Execution error (NullPointerException) at clojure.tools.namespace.find/find-ns-decls-in-dir$fn (find.clj:93).\nCannot invoke \"clojure.lang.IObj.withMeta(clojure.lang.IPersistentMap)\" because \"x\" is null\n",
 :clojure.main/triage
 {:clojure.error/class java.lang.NullPointerException,
  :clojure.error/line 93,
  :clojure.error/cause
  "Cannot invoke \"clojure.lang.IObj.withMeta(clojure.lang.IPersistentMap)\" because \"x\" is null",
  :clojure.error/symbol
  clojure.tools.namespace.find/find-ns-decls-in-dir$fn,
  :clojure.error/source "find.clj",
  :clojure.error/phase :execution},
 :clojure.main/trace
 {:via
  [{:type clojure.lang.Compiler$CompilerException,
    :message
    "Syntax error macroexpanding at (/home/lee/proj/oss/-verify/tns-test/test.clj:4:1).",
    :data
    {:clojure.error/phase :execution,
     :clojure.error/line 4,
     :clojure.error/column 1,
     :clojure.error/source
     "/home/lee/proj/oss/-verify/tns-test/test.clj"},
    :at [clojure.lang.Compiler load "Compiler.java" 7665]}
   {:type java.lang.NullPointerException,
    :message
    "Cannot invoke \"clojure.lang.IObj.withMeta(clojure.lang.IPersistentMap)\" because \"x\" is null",
    :at [clojure.core$with_meta__5485 invokeStatic "core.clj" 220]}],
  :trace
  [[clojure.core$with_meta__5485 invokeStatic "core.clj" 220]
   [clojure.core$with_meta__5485 invoke "core.clj" 219]
   [clojure.tools.namespace.find$find_ns_decls_in_dir$fn__1437
    invoke
    "find.clj"
    93]
   [clojure.core$keep$fn__8649 invoke "core.clj" 7409]
   [clojure.lang.LazySeq sval "LazySeq.java" 42]
   [clojure.lang.LazySeq seq "LazySeq.java" 51]
   [clojure.lang.RT seq "RT.java" 535]
   [clojure.core$seq__5467 invokeStatic "core.clj" 139]
   [clojure.core$print_sequential invokeStatic "core_print.clj" 53]
   [clojure.core$fn__7391 invokeStatic "core_print.clj" 174]
   [clojure.core$fn__7391 invoke "core_print.clj" 174]
   [clojure.lang.MultiFn invoke "MultiFn.java" 234]
   [clojure.core$pr_on invokeStatic "core.clj" 3675]
   [clojure.core$pr invokeStatic "core.clj" 3678]
   [clojure.core$pr invoke "core.clj" 3678]
   [clojure.lang.AFn applyToHelper "AFn.java" 154]
   [clojure.lang.RestFn applyTo "RestFn.java" 132]
   [clojure.core$apply invokeStatic "core.clj" 667]
   [clojure.core$pr invokeStatic "core.clj" 3691]
   [clojure.core$pr doInvoke "core.clj" 3678]
   [clojure.lang.RestFn applyTo "RestFn.java" 139]
   [clojure.core$apply invokeStatic "core.clj" 667]
   [clojure.core$prn invokeStatic "core.clj" 3715]
   [clojure.core$prn doInvoke "core.clj" 3715]
   [clojure.lang.RestFn applyTo "RestFn.java" 137]
   [clojure.core$apply invokeStatic "core.clj" 667]
   [clojure.core$println invokeStatic "core.clj" 3734]
   [clojure.core$println doInvoke "core.clj" 3734]
   [clojure.lang.RestFn invoke "RestFn.java" 421]
   [user$eval1480 invokeStatic "test.clj" 4]
   [user$eval1480 invoke "test.clj" 4]
   [clojure.lang.Compiler eval "Compiler.java" 7194]
   [clojure.lang.Compiler load "Compiler.java" 7653]
   [clojure.lang.Compiler loadFile "Compiler.java" 7591]
   [clojure.main$load_script invokeStatic "main.clj" 475]
   [clojure.main$script_opt invokeStatic "main.clj" 535]
   [clojure.main$script_opt invoke "main.clj" 530]
   [clojure.main$main invokeStatic "main.clj" 664]
   [clojure.main$main doInvoke "main.clj" 616]
   [clojure.lang.RestFn applyTo "RestFn.java" 137]
   [clojure.lang.Var applyTo "Var.java" 705]
   [clojure.main main "main.java" 40]],
  :cause
  "Cannot invoke \"clojure.lang.IObj.withMeta(clojure.lang.IPersistentMap)\" because \"x\" is null",
  :phase :execution}}

Execution error (NullPointerException) at clojure.tools.namespace.find/find-ns-decls-in-dir$fn (find.clj:93).
Cannot invoke "clojure.lang.IObj.withMeta(clojure.lang.IPersistentMap)" because "x" is null

Usage Scenario

Cljdoc uses tools.namespace to find namespaces during API analysis.

Various libs include all sorts of weirdnesses including namespace-less sources.

Previous versions of tools.namespace handled this weirdness for us.

We can certainly work around this if it is decided that tools.namespace should not handle namespace-less sources.

Exposing namespaces that will be reloaded from c.t.n.repl

Fri, 23 Sep 2022 22:42:10 +0000

Hello! I've recently copied some logic out of clojure.tools.namespace.repl that determines the next set of namespaces that need refreshing: https://github.com/clojure/tools.namespace/blob/master/src/main/clojure/clojure/tools/namespace/repl.clj#L91 This information is highly useful to external consumers who may need to manage some stateful resources prior to refreshing, depending on their state management solution.

I opened a PR here, which was closed because I apparently can't read contribution guidelines: https://github.com/clojure/tools.namespace/pull/13 This just pulls out functions that are invoked via alter-var-root to a standalone function. (vemv also helpfully pointed out that I forgot to propagate an argument).

Would this be a reasonable, small change to make to c.t.n.repl?

Thank you :)

tools.namespace does not recognise ns deps implied by data_readers.cljc

Fri, 29 Apr 2022 14:52:37 +0000

i found an interesting issue in our codebase around some tagged-literals seemingly violating the principle of least surprise

we have been creating some pure-data descriptions of processes including some defrecord objects from a custom tagged-literal reader declared in data_readers.cljc, and having those defrecord objects participate in a protocol. it's all been working just fine until i ran across this issue

someone had put one of these tagged literals into a def (which seemed like a reasonable thing to do at first blush):

(def foo #ctx/event-path [:blah])

which was ok... until c.t.n.r/refresh was called, after which point everything broke

it turned out that the defrecord object foo had a stale class (presumably because the namespace dependencies induced by the tagged-literal parsers declared in data_readers.cljc are not recognised by tools.namespace, and namespaces were therefore recompiled out of order), and so no longer participated in the protocol

should tools.namespace be able to recognise data_readers.cljc induced dependencies ? or is that a step too far for it ?

Could tools.namespace refresh get more sophisticated directory/file filtering?

Wed, 05 Jan 2022 18:03:20 +0000

TL;DR:
currently the only way of affecting which source files are loaded by clojure.tools.namespace.repl/refresh is setting refresh dirs, which is an allowlist-based system. There is no way currently to block specific directories/files/patterns from being loaded but allowing everything else. This presents an issue when there are clojure source files in source directories on the classpath, which are not meant to be loaded at all.

Long story:

We bumped into an interesting issue with clojure.tools.namespace/refresh vs clj-kondo hooks .

A one-liner repro case is:

clj -Srepro -Sdeps '{:deps {org.clojure/tools.namespace {:mvn/version "1.2.0"} seancorfield/next.jdbc {:git/url "https://github.com/seancorfield/next-jdbc/" :git/sha "24bf1dbaa441d62461f980e9f880df5013f295dd"}}}' -M -e "((requiring-resolve 'clojure.tools.namespace.repl/refresh-all))"

This will fail with:

:error-while-loading hooks.com.github.seancorfield.next-jdbc
Could not locate hooks/com/github/seancorfield/next_jdbc__init.class, hooks/com/github/seancorfield/next_jdbc.clj or hooks/com/github/seancorfield/next_jdbc.cljc on classpath. Please check that namespaces with dashes use underscores in the Clojure file name.

For context, clj-kondo is a static analyzer/linter. In order to be able to analyze custom macros properly, it lets libraries distribute clj files (describing how these macros should be analyzed) as resources under a specific directory.

The above example fails due to the combination of the following:

referring a lib by git coordinates, which means that it will appear as a directory on the classpath
said lib exporting clj-kondo hooks
clj-kondo hooks themselves being .clj files
as these are for clj-kondo to use, they are not meant to be loaded in the app using the lib
as a result these files normally having a namespace inside them that does not appear in code proper, for example: https://github.com/seancorfield/next-jdbc/blob/24bf1dbaa441d62461f980e9f880df5013f295dd/resources/clj-kondo.exports/com.github.seancorfield/next.jdbc/hooks/com/github/seancorfield/next_jdbc.clj#L1
tools.namespace by default reloads all clj source files found in classpath-directories (referring back to the first point)

As there is currently no way to tell tools.namespace to not load certain files I had to come up with a somewhat hacky workaround, which tries to set the refresh dirs to classpath-directories minus problematic ones, but it would be much much nicer if there was a way to be able to set this with a blocklist or predicate.

Workaround in case anyone else bumps into it:

(defn remove-clj-kondo-exports-from-tools-ns-refresh-dirs
  "A potential issue from using this is that if the directory containing the clj-kondo.exports folder
  also directly contains to-be-reloaded clojure source files, those will no longer be reloaded."
  []
  (->> (clojure.java.classpath/classpath-directories)
       (mapcat
        (fn [^File classpath-directory]
          (let [children   (.listFiles classpath-directory)
                directory? #(.isDirectory ^File %)
                clj-kondo-exports?
                           #(= "clj-kondo.exports" (.getName ^File %))
                has-clj-kondo-exports
                           (some (every-pred clj-kondo-exports? directory?) children)]
            (if has-clj-kondo-exports
              (->> children
                   (filter directory?)
                   (remove clj-kondo-exports?))
              [classpath-directory]))))
       (apply clojure.tools.namespace.repl/set-refresh-dirs)))

;; call in user.clj
(remove-clj-kondo-exports-from-tools-ns-refresh-dirs)

Resolution for bug TNS-6 in clojure.tools.namespace

Mon, 24 May 2021 17:39:12 +0000

Hi there,

So I believe I have a resolution for TNS-6 in clojure.tools.namespace. I have a consistent repro and a resolution in the form of a one-line patch (see the patch for more detail). All the tests still pass, though I'm not sure they cover the changes. I have also tested the resolution in a number of local dependent repos, and it resolves the issue.

However, I am not a Clojure contributor so I cannot comment or submitt the patch or comment on relevant Jira ticket.

From b5a62f894e049a09d5fcf099ac7b4d6591fb4f18 Mon Sep 17 00:00:00 2001
From: zalky <zalan.k@gmail.com>
Date: Mon, 24 May 2021 13:08:30 -0400
Subject: [PATCH] Fix TNS-6: fully remove ns from deps graph

The clojure.tools.namespace.dependency/remove-node fn removes a
namespace's :dependencies set from the graph (outgoing edges), but
does not remove that namespace from the :dependents set of other
namespaces (ingoing edges). If a namespace is removed and one of its
dependencies is changed at the same time, as commonly occurs when
switching branches, then the removed namespace is still in the
:dependents of the changed namespace when it is reloaded. As all the
of the dependents of a changed namespace are also reloaded, this
throws a missing namespace error.

The resolution is to use clojure.tools.namespace.depedency/remove-all
to remove both outgoing (:dependencies) and ingoing (:dependents)
edges in the dependency graph.

However, care must be taken when choosing the point of change, because
clojure.tools.namespace.track/add relies on the partial removal
behaviour of clojure.tools.namespace.track/remove-deps. It uses
remove-deps to wipe the dependencies of a node, and cannot know how to
re-insert dependents if lost. Therefore the point of change should be
in clojure.tools.namespace.track/remove.
---
 src/main/clojure/clojure/tools/namespace/track.cljc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/main/clojure/clojure/tools/namespace/track.cljc b/src/main/clojure/clojure/tools/namespace/track.cljc
index 683c35d..ce666d5 100644
--- a/src/main/clojure/clojure/tools/namespace/track.cljc
+++ b/src/main/clojure/clojure/tools/namespace/track.cljc
@@ -96,7 +96,7 @@
          :or {load (), unload (), deps (dep/graph)}} tracker
         known (set (dep/nodes deps))
         removed-names (filter known names)
-        new-deps (remove-deps deps removed-names)
+        new-deps (reduce dep/remove-all deps removed-names)
         changed (affected-namespaces deps removed-names)]
     (assoc tracker
       ::deps new-deps
-- 
2.30.1 (Apple Git-130)

[Patch updated after further testing]

Should tools.namespace.repl/refresh exclude code found in ~/.gitlibs?

Fri, 26 Mar 2021 14:43:59 +0000

If you have a deps.edn project with a git dep in it, in a project where you're also using clojure.tools.namespace.repl/refresh then by default the refresh call will load and refresh all the clojure namespaces in your git dep.

I think it does this because prior to tools.deps; essentially all library dependencies were distributed as jars, so it would assume that any directories in the classpath contain your projects source code, and would benefit from reloading via refresh.

This assumption changes in tools.deps projects as deps can now be stored as files on the filesystem in the ~/.gitlibs directory. These deps are essentially mistaken as project dependencies, and are all loaded and refreshed, even if your project does not actually require them.

Given that ~/.gitlibs are versioned and intended to be immutable, arguably it doesn't make sense to automatically require and refresh them.

It is quite easily avoided by including a call to explicitly set your projects refresh-dirs with something like: (tnsrepl/set-refresh-dirs "src" "test"), however I wonder if it would be better for tools.namespace to exclude anything under ~/.gitlibs by default, as tools.namespace's starting/default assumption no longer holds.

NOTE this is also likely related to this issue.

Preventing `clojure.tools.namespace.repl/refresh` from loading namespaces that haven't been loaded before

Tue, 02 Mar 2021 19:51:03 +0000

Two issues that I see in the current implementation of clojure.tools.namespace.repl/refresh:

Upon the very first call, it reloads absolutely all Clojure files on the classpath. Its docstring hints at it, but it doesn't explain for why such a behavior is needed. At the very least, it makes the first refresh call much slower than needed.
It will attempt to (re)load even the files that haven't been loaded before. A very real situation - I have some Clojure files on classpath that require dependencies that aren't available. That alone makes it impossible to use refresh. Perhaps, it can be altered to avoid loading files that haven't been loaded before.

Currently, I'm using these workarounds:

(alter-var-root #'clojure.tools.namespace.repl/refresh-tracker
                assoc :clojure.tools.namespace.dir/time (System/currentTimeMillis))

(alter-var-root #'clojure.tools.namespace.repl/remove-disabled
                (fn [orig-remove-disabled]
                  (fn [tracker]
                    (let [filter-loaded #(filter find-ns %)]
                      (-> tracker
                          orig-remove-disabled
                          (update :clojure.tools.namespace.track/unload filter-loaded)
                          (update :clojure.tools.namespace.track/load filter-loaded))))))

Of course, the workaround assumes that each files has a proper ns form and doesn't create other namespaces. But seems like this assumption is also made by tools.namespace, namely by the usage of clojure.tools.namespace.file/read-file-ns-decl.

Would be nice if tools.namespace.find functions return file info in ns-decl meta

Tue, 05 Nov 2019 13:35:01 +0000

Currently calling tools.namespace.find/find-ns-decls returns a seq of ns-decl for a given dir or collection of paths but there is no way to know where did they came from (can also be coming from paths inside jar files).

Looks pretty straight forward to add file info as meta of the ns-decl form. Something like https://github.com/jpmonettas/tools.namespace/commit/1431215324ba7f76f78342742ce3716ec8c5abb8

What do you guys think?

(Filed jira at https://clojure.atlassian.net/browse/TNS-55)

How does tools.namespace.track/add is supposed to work?

Fri, 18 Oct 2019 21:22:25 +0000

I'm trying to figure out if I can use tools.namespace.track for a project but I don't understand the :unload :load functionality.

(require '[clojure.tools.namespace.track :as ns-track])

;; define a empty tracker and add some dependencies
(def tracker (-> (ns-track/tracker)
                 (ns-track/add '{alpha #{beta}
                                 beta  #{gamma delta}})))

;; now add a new dependency and check what we need to unload and load
(-> tracker
    (ns-track/add '{epsilon #{}})
    (select-keys [:clojure.tools.namespace.track/unload
                  :clojure.tools.namespace.track/load]))

#:clojure.tools.namespace.track{:unload (epsilon alpha beta),
                                :load (epsilon beta alpha)}

I don't understand why alpha and beta are required to unload/load but maybe I'm missing something.

I'm using the latest version.

Thanks!

Is it usual to require all application namespaces via tools.namespace?

Sun, 04 Aug 2019 16:39:19 +0000

We have to require namespaces with side effects like multimethod or protocol implementation.
So we have a lot of noisy require forms in one initial namespace.

Is it better to use tools.namespace for require all app namespaces?

Java 11 and Tools Namespace

Wed, 31 Jul 2019 14:50:56 +0000

I've run into an issue where refresh from tools.namespace (v. 0.3.0) seems to work with Java 8 but not Java 11. I tried with both Clojure 1.10.0 and 1.10.1. I was able to reproduce the problem on a clean virtual machine. In both cases, I was able to reload the namespaces on Java 8 but not Java 11.

It's weird though -- in some of my other projects, I can reload namespaces on Java 11. I haven't been able to track down the difference. (I'm also new to tools.deps, so it could be I'm doing something wrong there.)

I put together a reproduction here: https://github.com/thomascothran/namespace-repro

I'm assuming it's likely an error on my end, but the only thing I've been able to narrow it down to is the Java version.

Reference: https://clojure.atlassian.net/browse/TNS-54

Breakage with tools.reader 1.3.0

Tue, 16 Oct 2018 09:04:51 +0000

I've noticed this problem after bumping the tools.reader dep in cider-nrepl to 1.3.0:

Oct 15, 2018 5:50:23 PM clojure.tools.logging$eval579$fn__582 invoke
SEVERE: Unhandled REPL handler exception processing message {:op init-debugger, :print-level 10, :print-length 10, :session cc619163-753b-46b1-9334-cf6e9459aac4, :id 8}
java.lang.ClassNotFoundException: clojure.tools.reader.reader_types.SourceLoggingPushbackReader, compiling:(cider/inlined_deps/toolsreader/v1v3v0/clojure/tools/reader.clj:1:1)
    at clojure.lang.Compiler.load(Compiler.java:7526)
    at clojure.lang.RT.loadResourceScript(RT.java:379)
    at clojure.lang.RT.loadResourceScript(RT.java:370)
    at clojure.lang.RT.load(RT.java:460)
    at clojure.lang.RT.load(RT.java:426)
    at clojure.core$load$fn__6548.invoke(core.clj:6046)
    at clojure.core$load.invokeStatic(core.clj:6045)
    at clojure.core$load.doInvoke(core.clj:6029)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.core$load_one.invokeStatic(core.clj:5848)
    at clojure.core$load_one.invoke(core.clj:5843)
    at clojure.core$load_lib$fn__6493.invoke(core.clj:5888)
    at clojure.core$load_lib.invokeStatic(core.clj:5887)
    at clojure.core$load_lib.doInvoke(core.clj:5868)
    at clojure.lang.RestFn.applyTo(RestFn.java:142)
    at clojure.core$apply.invokeStatic(core.clj:659)
    at clojure.core$load_libs.invokeStatic(core.clj:5925)
    at clojure.core$load_libs.doInvoke(core.clj:5909)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at clojure.core$apply.invokeStatic(core.clj:659)
    at clojure.core$require.invokeStatic(core.clj:5947)
    at clojure.core$require.doInvoke(core.clj:5947)
    at clojure.lang.RestFn.invoke(RestFn.java:421)
    at cider.inlined_deps.toolsnamespace.v0v3v0_alpha4.clojure.tools.namespace.parse$eval2867$loading__6434__auto____2868.invoke(parse.cljc:9)
    at cider.inlined_deps.toolsnamespace.v0v3v0_alpha4.clojure.tools.namespace.parse$eval2867.invokeStatic(parse.cljc:9)
    at cider.inlined_deps.toolsnamespace.v0v3v0_alpha4.clojure.tools.namespace.parse$eval2867.invoke(parse.cljc:9)

    ... skipping some lines ...
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: clojure.tools.reader.reader_types.SourceLoggingPushbackReader
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at clojure.lang.DynamicClassLoader.findClass(DynamicClassLoader.java:69)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at clojure.lang.DynamicClassLoader.loadClass(DynamicClassLoader.java:77)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:348)
    at clojure.lang.RT.classForName(RT.java:2204)
    at clojure.lang.RT.classForNameNonLoading(RT.java:2217)
    at cider.inlined_deps.toolsreader.v1v3v0.clojure.tools.reader$eval3532$loading__6434__auto____3533.invoke(reader.clj:9)
    at cider.inlined_deps.toolsreader.v1v3v0.clojure.tools.reader$eval3532.invokeStatic(reader.clj:9)
    at cider.inlined_deps.toolsreader.v1v3v0.clojure.tools.reader$eval3532.invoke(reader.clj:9)
    at clojure.lang.Compiler.eval(Compiler.java:7062)
    at clojure.lang.Compiler.eval(Compiler.java:7051)
    at clojure.lang.Compiler.load(Compiler.java:7514)
    ... more

After first going to the tools.reader tracker (https://dev.clojure.org/jira/browse/TRDR-55) I was told this is a problem that has to be fixed in tools.namespace, so I'm filing this ticket here as well.

Generating namespace forms

Mon, 01 Oct 2018 12:00:06 +0000

Since tools.namespace supports extracting dependencies from a namespace form (https://github.com/clojure/tools.namespace/blob/master/src/main/clojure/clojure/tools/namespace/parse.cljc#L123), would it also be in scope for tools.namespace to support generating namespace forms?

That is, could we add a function that, given a list of dependent namespaces, java classes, etc., generates a namespace form in a canonical style (e.g., https://stuartsierra.com/2016/clojure-how-to-ns.html)?

My use case is linting/reformatting a few hundred namespace declarations across large enterprise project.
I'd like to do this programmatically --- since I haven't found any tools so far, I thought I'd write my own, and would like to make it open source if possible.

Thought I'd run it by y'all first in case it'd be in scope for tools.namespace and there were specific design considerations I should keep in mind before writing the code.
(And no worries if this is out of scope because it's too bike-sheddy, etc.)

Support namespaces as strings in require statements

Sat, 28 Jul 2018 19:41:23 +0000

Since July 2017, in clojurescript, one can require npm modules using the following syntax:
(:require (link: "honeybadger-js" :as honeybadger))

The npm module to required is passed as a string.

See https://clojurescript.org/news/2017-07-12-clojurescript-is-not-an-island-integrating-node-modules for the full details.

Currently, tools.namespace consider this form as invalid.

`require :reload` fails where `load-file` succeeds

Thu, 17 May 2018 22:07:42 +0000

Here are steps to reproduce at the REPL:
- (c.t.n.r/refresh) succeeds
- Modify any file in the classpath
- (c.t.n.r/refresh) fails from not having found certain namespace(s) (not always the same one(s))

This always happens with a particular project, but no (known) others of relatively similar complexity. AOT compilation/execution works with the project in question, as do requires without using (c.t.n.r/refresh). For (c.t.n.r/refresh), load-file works in place of require :reload in clojure.tools.namespace.reload/track-reload-one. It is not known what the root cause of this bug is beyond require :reload, or why load-file works instead, but it's a simple enough patch with no known downsides. For now I'm hot-swapping code every time at the REPL.

A fork is available at https://github.com/alexandergunnarson/tools.namespace from which I can submit a patch if desired.

Show full path for circular dependency exceptions

Thu, 23 Nov 2017 21:50:10 +0000

I use {{org.clojure/tools.namespace "0.3.0-alpha4"}} to detect circular dependencies among arbitrary objects. It works fine, but the error could be more informative for a certain case:

{:type clojure.lang.ExceptionInfo
   :message "Circular dependency between x and y"
   :data {:reason :clojure.tools.namespace.dependency/circular-dependency, :node x, :dependency y}
   :at [clojure.core$ex_info invokeStatic "core.clj" 4617]}

{{x}} and {{y}} are redacted data.

The problem is that neither x or y depend on each other directly, but via an indirect path.

It would be tremendously useful if the exception itself informed of the full faulty path, instead of just the beginning and end.

Example of desired exception:

:data {:reason :clojure.tools.namespace.dependency/circular-dependency, :node x, :dependency y, :path [x a b c y]}

Ability to track files that are not namespaces

Fri, 03 Nov 2017 07:23:48 +0000

Support tracking non-clojure files when performing a refresh. The idea here is to make the reloaded workflow work well for libraries like HugSQL (which may depend on non-clojure files in the project).

See https://github.com/layerware/hugsql/issues/72 for some additional context

Add project.clj for easier development

Tue, 13 Sep 2016 22:27:43 +0000

Tools.namespace doesn't currently have have a Leiningen configuration which would make it easy to start a repl and start developing tools.namespace.

Attached patch adds simple project.clj file with deps and source-paths.

File in invalid path will mark namespace for reload

Tue, 13 Sep 2016 21:15:40 +0000

Having a cljc file at path {{public/js/out/foo/bar.cljc}} with ns form {{(ns foo.bar)}} will result in namespace {{foo.bar}} being reloaded.

This is problematic because ClojureScript compiler will copy all the input files to {{:output-dir}} for source-map use. Lately as more libraries have started using cljc, and this has started causing problems in cases where library is used on Clojure env. Cljs compilation will result in reload of the library code, which can redefine protocols etc. and break the Clojure env.

I think it would make sense for tools.namespace to ignore changes where file path and namespace don't match.
Another question and perhaps way to fix this, is to understand why dependency resolution doesn't work in this case: namespaces which depend on the protocols at a cljc file on output-dir aren't reloaded.

ns-decl? treats (ns) as valid

Mon, 28 Mar 2016 12:10:01 +0000

If you try to declare an empty namespace with (ns), you get an ArityException, but ns-decl? does not test for this.

The attached patch simply tests whether it is at least two items long.

I have previously signed the Clojure CA.

scan-dirs / scan-all can return incorrect dependencies when both clj and cljc files define same namespace

Wed, 30 Dec 2015 22:13:22 +0000

Originally filed as an issue for function scan-all, which at the time I was unaware was deprecated in 0.3.0 alphas. I have tested against the newer scan-dirs as well, and it has the same problem.

When there are both .clj and .cljc files defining the same namespace, tools.namespace scan-dirs and scan-all can use the dependencies from the .cljc file, ignoring those in the .clj file, whereas for Clojure/Java the opposite should be done.

See sample project and its README here for steps to reproduce: https://github.com/jafingerhut/tnstest

Note that Clojure/Java's behavior is to prefer .clj file over a .cljc file defining the same namespace, even if the .clj file is in a directory that is later in the classpath than a .cljc file. According to Alex Miller email on clojure-dev Google group, this behavior is by design. Link: https://groups.google.com/d/msg/clojure-dev/d5Hb1E7zfHE/sPybIAxgCwAJ

Single classpath argument for c.t.n.move

Fri, 06 Nov 2015 21:01:33 +0000

{{clojure.tools.namespace.move/move-ns}} requires that the caller provide both 1) the directory containing the file to be moved; and 2) the collection of directories containing all source files to be updated with the new name. This is redundant.

Instead, we can search for the file in all source directories, move it within the same root directory, then update all files in all directories.

In addition, with java.classpath in TNS-36, the set of directories to search can default to directories on the classpath. This will be much more convenient for REPL use.

This change can be made without breaking older arities of {{move-ns}}.

c.t.n.move: does not support to move .cljc files

Sat, 05 Sep 2015 22:19:57 +0000

it seems that c.t.n.move did not get any reader conditionals love. I use this namespace to power (link: https://github.com/benedekfazekas/mranderson text: mranderson): a source inlining tool used by cider-nrepl and refactor-nrepl in order to avoid clashes with the code/projects their users work on. lately we started adding dependencies to refactor-nrepl (well, tools.namespace itself) which may use .cljc files. source inlining started failing for these source files.

Broken tracker after mis-named namespace

Sun, 07 Sep 2014 17:23:39 +0000

A tools.namespace dependency tracker can get into an inconsistent state after attempting to load a file with a namespace declaration that does not match the file's name.

Steps to reproduce:

In a new Clojure project, create a file {{foo.clj}} on the classpath containing the namespace declaration {{(ns wrong-ns-declaration-for-foo)}}

Then at the REPL:

`
user=> (use 'clojure.tools.namespace.repl)
nil
user=> (refresh)
:reloading (wrong-ns-declaration-for-foo)
:error-while-loading wrong-ns-declaration-for-foo

Edit the file {{foo.clj}} so that its namespace declaration is correct: {{(ns foo)}}

But at the REPL, {{refresh}} still doesn't work:

`
user=> (refresh)
:reloading (foo wrong-ns-declaration-for-foo)
:error-while-loading wrong-ns-declaration-for-foo

Since tools.namespace 0.2.5, the workaround is to call {{clear}}:

user=> (clear) {} user=> (refresh) :reloading (foo) :ok

Stack overflow on refresh after circular dependency detected

Wed, 17 Apr 2013 16:34:08 +0000

`
user> (refresh)
Exception Circular dependency between com.example.foo and com.example.bar clojure.tools.namespace.dependency.MapDependencyGraph (dependency.clj:74)
user> (refresh)
StackOverflowError clojure.lang.PersistentHashMap$BitmapIndexedNode.index (PersistentHashMap.java:576)
user> (clojure.repl/pst)
StackOverflowError

clojure.lang.PersistentHashMap$BitmapIndexedNode.index (PersistentHashMap.java:576)
clojure.lang.PersistentHashMap$BitmapIndexedNode.find (PersistentHashMap.java:676)
clojure.lang.PersistentHashMap$ArrayNode.find (PersistentHashMap.java:396)
clojure.lang.PersistentHashMap.valAt (PersistentHashMap.java:152)
clojure.lang.PersistentHashMap.valAt (PersistentHashMap.java:156)
clojure.lang.RT.get (RT.java:645)
clojure.tools.namespace.dependency/transitive (dependency.clj:52)
clojure.tools.namespace.dependency/transitive/fn--7043 (dependency.clj:51)
clojure.core.protocols/fn--6022 (protocols.clj:79)
clojure.core.protocols/fn--5979/G--5974--5992 (protocols.clj:13)
clojure.core/reduce (core.clj:6177)
clojure.tools.namespace.dependency/transitive (dependency.clj:52)

nil
`

Attempt to reload deleted file

Fri, 14 Dec 2012 13:56:02 +0000

I can't identify the exact circumstances, but I have seen events where a source code file has been deleted but clojure.tools.namespace.repl/refresh still tries to reload it. Because the file doesn't exist, there's an exception when you try to load it, so you're stuck.