Separate text from JSON to process JSON

Question

Hi,

I'm starting with Clojure, better yet, re-starting after trying it several years ago.
There are some new tools like Babashka that I'm trying to use for this.

The challenge is to do the following:

1. Start ansible virtual environment

2. Run an external process with 'ansible-inventory' against one host
   at a time, or all hosts, let's see...

3. The interesting part of the output of 'ansible-inventory' goes to
   stderr, so read it.

4. It consists of lines of text and then some JSON.

5. Parse the JSON contents.

It will be more elaborate than this, but it's enough for the case.

Let's forget about 1.

Instead of 2., just do a simple 'cat a.txt' file built with some text and some JSON like

line1
line 2
{
  'field': 'value'
}

What I've been able to build was this:

#!/usr/bin/env bb

(require '[babashka.process :refer [shell]])
(require '[cheshire.core :as json])
(require '[clojure.string :as str])

(-> (shell {:out :string} "cat" "a.txt")
    :out json/parse-string)

What I'do in an imperative language, that I use to use would be:

• Read line by line. In Clojure (str/split-lines)

• Discard any line until a line consisiting only of a '{' is found.

• From there, accumulate all lines in a string until a '}' is found.

• Parse the JSON from that string.

How can I approach this in Clojure?

Answer 1

So you already came up with str/split-lines for your first bullet, to get a sequence of lines. For the second bullet you can use drop-while to get rid of all the lines before the one that is just your opening {. At that point I would probably just rejoin the rest of the lines and feed that entire string to a JSON parser, telling it to parse one object. But you could use take-while to get all the lines up to (but not including) the one that is }, then join those and tack the } back on as well.

With the caveat that your chosen JSON parser does not seem to like strings enclosed in single quotes. so I had to edit a.txt to use double quotes instead, as well as adding some trailing post-JSON lines to test that they get stripped properly too, the following seems to work:

#!/usr/bin/env bb

(require '[babashka.process :refer [shell]])
(require '[cheshire.core :as json])
(require '[clojure.string :as str])

(defn extract-json [s]
  (let [json-lines (->> s
                        str/split-lines
                        (drop-while #(not (re-matches #"\s*\{\s*" %)))
                        (take-while #(not (re-matches #"\s*\}\s*" %)))
                        (str/join "\n"))]
    (println (json/parse-string (str json-lines "}")))))


(-> (shell {:out :string} "cat" "a.txt")
    :out extract-json)`

And for completeness, here is my revised a.txt:

line1
line 2
{
  "field": "value"
}
line 3

Comments

Thank you James!  Regarding JSON, it should have double quotes.  I used the parser that saw referenced.  I'm not used to the Java and jvm worlds :-)