parse-props in server.clj throws ConcurrentModificationException

Question

Looks like the iterator version of Java system properties isn't thread-safe. This can be confirmed using the code below:

import java.io.FileWriter;
import java.util.Map;
import java.util.Properties;
import java.util.Random;

public class JavaPropertiesTest {
  public static void main(String[] args) throws Exception {
    new Thread(() -> {
      while (true) {
        Properties properties = System.getProperties();
        try (FileWriter fileOutputWriter = new FileWriter("/dev/null")){
          for (Map.Entry<Object, Object> entry : properties.entrySet()) {
            fileOutputWriter.append(entry.getKey().toString());
          }
        } catch (Exception e) {
          e.printStackTrace();
        }
      }
    }).start();

    new Thread(() -> {
      Random gen = new Random();
      while (true) {
        System.getProperties().setProperty(Integer.toString(gen.nextInt()), Integer.toString(gen.nextInt()));
      }
    }).start();
  }
}

stack trace in clojure 1.8.x, and it is still the same for the most recent release
Caused by: java.util.ConcurrentModificationException at java.util.Hashtable$Enumerator.next(Hashtable.java:1387) at clojure.lang.RT$4.invoke(RT.java:509) at clojure.lang.LazySeq.sval(LazySeq.java:40) at clojure.lang.LazySeq.seq(LazySeq.java:49) at clojure.lang.ChunkedCons.chunkedNext(ChunkedCons.java:59) at clojure.lang.ChunkedCons.next(ChunkedCons.java:43) at clojure.lang.RT.next(RT.java:688) at clojure.core$next4341.invokeStatic(core.clj:64) at clojure.core.protocols$naive_seq_reduce.invokeStatic(protocols.clj:63) at clojure.core.protocols$interface_or_naive_reduce.invokeStatic(protocols.clj:72) at clojure.core.protocols$fn6755.invokeStatic(protocols.clj:168) at clojure.core.protocols$fn6755.invoke(protocols.clj:124) at clojure.core.protocols$fn6710$G6705__6719.invoke(protocols.clj:19) at clojure.core.protocols$seq_reduce.invokeStatic(protocols.clj:31) at clojure.core.protocols$fn6732.invokeStatic(protocols.clj:75) at clojure.core.protocols$fn6732.invoke(protocols.clj:75) at clojure.core.protocols$fn6684$G66796697.invoke(protocols.clj:13) at clojure.core$reduce.invokeStatic(core.clj:6545) at clojure.core.server$parse_props.invokeStatic(server.clj:143) at clojure.core.server$start_servers.invokeStatic(server.clj:154) at clojure.core.server$start_servers.invoke(server.clj:154) at clojure.lang.Var.invoke(Var.java:379) at clojure.lang.RT.doInit(RT.java:482) at clojure.lang.RT.

Answer 1

Are you actually seeing this during Clojure startup somehow?

This is called in a single thread during Clojure startup so I'm not sure how this would actually happen. What other thread is modifying the props?

Comments

Yes, it actually happened in my EMR jobs which use uses clojure as a component. I have no idea what other threads are actually doing in this case, but looks like it's related to EMRFS(s3 client) set up. Unfortunately, I can't paste all the stacktrace here due to editor limitation.

Answer 2

Logged as https://clojure.atlassian.net/browse/CLJ-2686