Clojure Spec for GeoJSON

Clojure Spec is a new feature added to Clojure 1.9 that allows you to define the structure of your data, validate your data, and generate your data. Clojure being dynamically typed often finds those that are familiar with a type system, struggle to know what each function expects. Spec helps to bridge this gap by describing the set of allowed values. After playing with Clojure spec, I came to the realization that a type system still leaves a lot of work to assert the input values that are typed.

One of the best features is the ability to generate data based on a spec. Since my day job is to write software and develop tools for the GIS (Geographic Information Systems) domain, I will walk through how you can use spec to generate and validate geojson sample data. Below is an example of an point represented in geojson.

{
"type": "Feature",
"geometry": {
"type": "Point",
"coordinates": [125.6, 10.1]
},"properties": {
"name": "Dinagat Islands"
}
}

Lets start from the inside out. Fire up the REPL. First we need two doubles for latitude and longitude.

(require '[clojure.spec :as s])
(s/def ::longitude double?)
(s/def ::latitude double?)

Now we need to combine these into two the value of the coordinates key.

(s/def :wgs/coordinates (s/tuple ::longitude ::latitude))

(s/valid? :wgs/coordinates [1.0 2.0])
=> true
(s/explain :wgs/coordinates [1.8 "d"])
In: [1] val: "d" fails spec: :clojure-spec-meetup-talk.core/latitude at: [1] predicate: double?
=> nil

Lets take our first run at generating data.

(gen/generate (s/gen :wgs/coordinates))
=> [-134.5 -1.0]

Lets look at the types of data it can generate

(type (clojure.spec.gen/generate (s/gen :wgs/coordinates))) 
=> clojure.lang.PersistentVector
(type (clojure.spec.gen/generate (s/gen :cart/coordinates)))
=> clojure.lang.LazySeq

Now that we have the ability to generate data, lets start making a spec for GeoJSON.

(s/def ::type #{"Point"})
(s/def ::geometry (s/keys :req-un [::type :wgs/coordinates]))

and generate some data

(gen/generate (s/gen ::geometry))
=> {:type "Point", :coordinates [-0.01104752100945916 -0.017578125]}

or sample the spec multiple times

(gen/sample (s/gen ::geometry))
=>
({:type "Point", :coordinates [-2.0 2.0]}
{:type "Point", :coordinates [-0.5 -2.0]}
{:type "Point", :coordinates [-2.0 1.0]}
{:type "Point", :coordinates [0.0 1.0]}
{:type "Point", :coordinates [0.5 0.5]}
{:type "Point", :coordinates [-2.0 -1.75]}
{:type "Point", :coordinates [2.75 0.0]}
{:type "Point", :coordinates [1.0 -1.75]}
{:type "Point", :coordinates [-3.0 -1.0]}
{:type "Point", :coordinates [2.53125 6.375]})

Now we know that there are reasonable ranges to a latitude and longitude. This is where you can integrate the business logic of your domain into your specs.

(s/def ::longitude (s/double-in :min -180.0 :max 180 :NaN false :infinite? false))
(s/def ::latitude (s/double-in :min -90.0 :max 90 :NaN false :infinite? false))
(s/def ::coordinates (s/tuple ::longitude ::latitude))
(s/def ::geometry (s/keys :req-un [::type ::coordinates]))

Now that the coordinates are defined, we need to add the other pieces of the spec.

(s/def :feature/type #{"Feature"})
(s/def ::properties map?)
(s/def ::feature (s/keys :req-un [:feature/type ::geometry ::properties]))
(gen/sample (s/gen ::feature))
=> {:id "zOWGcd4B9",
:type "Feature",
:geometry {:id "77", :type "Point", :geometry {:type "Point", :coordinates [0.5 3.9921875]}, :properties {\B 4/3}},
:properties {\y #uuid"6af3de71-717b-475d-b818-55a9a6dbc332",
:bHlh_k .4,
z?7+4!3oQ 0.75,
-0.8974609375 -1.375,
:JHb?.*65U96!71*.m2.+_86x_F/?TA1fFL!I1 3/8,
-4 :raW6_*_.u._+50h.WY89+.bI2cs!/SR}}

This is why clojure.spec and generative testing are awesome. Looking at the properties we can see that won’t be valid JSON. We can redefine it like so:

(s/def ::properties (s/map-of string? (s/or :s string? :n number? :m map? :c coll?)))
=>
{:id "Kg3K2id2",
:type "Feature",
:properties {"6y" {:wQ6b!--.L+P127J._1-.lP3!-4.k_1LrI3jNf.M/*6r21*u2m A*.!5-L,
*3Q_-.-K._2kxd!OO?*._-*a1*.U.og1-W*+9r6-.DI3p/i_- #uuid"029e4e58-3613-4667-ae18-5027798121aa",
false true,
:V9C-!7 q-3.r.*C13IS-+!-/_**FbC-,
#uuid"ff256ba0-a0b6-4d46-b69f-e5cc55881edd" \V,
\X zh-D89+7A.ir24*l.o?.*H_-!4.F2**kx/Zu.Bq,
-1 :g_-8.BA.+ifII.Mz_Hd.k2_!nvMD*.hP8t!OcnW.R7.Ir-!/?XnA!}}}

properties key in the map is a string, but we didn’t require the value to also be valid GeoJSON. Lets try further defining what a valid JSON map is.

(s/def ::jsonobj (s/map-of string? (s/or :s string? :n number? :m ::jsonobj :c (s/coll-of ::jsonobj))))

We can use a spec here to define the spec. The only issue is that the randomness of this will peg your CPU because it will create a random depth to this nesting.

(s/def ::properties ::jsonobj) 
(s/def ::id (s/or :p pos-int? :s (s/and string? #(not (str/blank? %)))))
(s/def ::feature (s/keys :req-un [::id :feature/type ::geometry ::properties]))
(gen/sample (s/gen ::feature)) ; pegging the CPU time

So we will need to think about the downsides of generating random data. There is no current way I know of to control the depth of the nested JSON maps so you will need to resort to map? for now.

Now lets define some of the geometry types in GeoJSON with specs.

; namespace point
(s/def :pt/type #{"Point"})
(s/def :pt/geometry (s/keys :req-un [:pt/type ::coordinates]))
(s/def :pt/feature (s/keys :req-un [::id :pt/type :pt/geometry ::properties]))
; namespace poly
(s/def :poly/type #{"Polygon"})
(defn circle-gen [x y]
(let [vertices (+ (rand-int 8) 4)
radius (rand 3) ;2 dec degrees radius length
rads (/ (* 2.0 Math/PI) vertices)
pts (map (fn [r]
[(+ x (* radius (Math/cos (* r rads))))
(+ y (* radius (Math/sin (* rads r))))])
(range vertices))]
(conj pts (last pts))))
(s/def :poly/coordinates (s/with-gen
coll?
#(gen/fmap (fn [[lon lat]] (list (circle-gen lon lat)))
(gen/tuple (s/gen ::longitude) (s/gen ::latitude)))))
(s/def :poly/geometry (s/keys :req [:poly/type :poly/coordinates]))
(s/def :poly/feature (s/keys :req-un [::id :poly/geometry :poly/type ::properties]))
(s/def :feat/geometry (s/or :poly/feature :pt/feature))
(s/def ::feature (s/keys :req-un [::id :feature/type :feat/geometry ::properties]))
(gen/sample (s/gen :poly/feature))
(s/def :gfeature/type (s/or :pt/type :poly/type))

circle-gen is a simple way for our generator to generate polygons. Theses are not going to cut it for a complex GIS system because these are simple convex polygons that won’t help you too much in the real world. There are two sides to using spec: validating data and generating data.

And to finish it off, here is a how wed define it in a feature.

(s/def ::feature-spec (s/keys :req-un
[::id :gfeature/type
:feat/geometry ::properties]))
(s/def :gj/features (s/coll-of ::feature-spec))
(s/def :fc/type #{"FeatureCollection"})
(s/def ::featurecollection-spec (s/keys :req-un [:fc/type :gj/features]))
(gen/sample (s/gen ::featurecollection-spec))

I’ve started making a geometry library for Clojure here . The goal is to make is work with JTS and JSTS for Clojure and Clojurescript and stay up with the latest.

One clap, two clap, three clap, forty?

By clapping more or less, you can signal to us which stories really stand out.