Transform JSON data with jq!

Published in

Learning.com Tech Blog

3 min readAug 30, 2018

I’ve been using jq for a while, simply as a way to “pretty-print” JSON data in the terminal, but I’ve barely been scratching the surface.

According to the jq website,

jq can mangle the data format that you have into the one that you want with very little effort, and the program to do so is often shorter and simpler than you’d expect.

…but, that appears to be underselling it a bit, according to Wikipedia:

jq is a very high-level functional programming language with support for backtracking and managing streams of JSON data. It is related to the Icon and Haskell programming languages.

A “Klein Bottle” model; related to the “Impossible Torus” of the JSON logo. ;-)

Pretty-printing with curl and jq

Here’s an example of JSON returned from https://status.github.com/api.json raw, and the same JSON piped through jq:

Examples

The real power comes from applying filters and streams to the source JSON. Let’s use the following Gist as an example:

Get the value of the “id” property at the object root:

$ jq '.id' lcom-techblog-jq.json"Ag8EAQAKCwAPAAwIBAoJAQ"

'.' returns the input unchanged, while '.id' returns the value of the “id” property.

jq ‘.id’

Get the first object in the “links” array:

$ jq '.links[0]' lcom-techblog-jq.json{
  "active": true,
  "iden": "CwgODwQAAwwCBAcNBAcGBQ",
  "created": 1534960753.141877,
  "modified": 1534970458.090077,
  "type": "link",
  "title": "Example Domain",
  "url": "https://example.com/"
}

jq ‘.links[0]’

Get all of the “url” values in the “links” array:

$ jq '.links[].url' lcom-techblog-jq.json"https://example.com/"
"https://www.iana.org/domains/reserved"
"https://tools.ietf.org/html/rfc2606"

jq ‘.links[].url’

…as a new array:

$ jq '[ .links[].url ]' lcom-techblog-jq.json[
  "https://example.com/",
  "https://www.iana.org/domains/reserved",
  "https://tools.ietf.org/html/rfc2606"
]

jq ‘[ .links[].url ]’

Select only the objects in the “links” array that are active:

$ jq '[ .links[] | select(.active == true) ]' lcom-techblog-jq.json[
  {
    "active": true,
    "iden": "CwgODwQAAwwCBAcNBAcGBQ",
    "created": 1534960753.141877,
    "modified": 1534970458.090077,
    "type": "link",
    "title": "Example Domain",
    "url": "https://example.com/"
  },
  {
    "active": true,
    "iden": "BAENAwEBAQQBAgMDBAcECA",
    "created": 1534900109.6203609,
    "modified": 1534953681.504821,
    "type": "link",
    "title": "IANA — IANA-managed Reserved Domains",
    "url": "https://www.iana.org/domains/reserved"
  }
]

jq ‘[ .links[] | select(.active == true) ]’

…as a new array, with only selected properties:

$ jq '[ .links[] | select(.active == true) | { created, title, url } ]' lcom-techblog-jq.json[
  {
    "created": 1534960753.141877,
    "title": "Example Domain",
    "url": "https://example.com/"
  },
  {
    "created": 1534900109.6203609,
    "title": "IANA — IANA-managed Reserved Domains",
    "url": "https://www.iana.org/domains/reserved"
  }
]

jq ‘[ .links[] | select(.active == true) | { created, title, url } ]’

…with “created” converted to an ISO-8601 string:

$ jq '[ .links[] | select(.active == true) | { created: (.created|todate), title, url } ]' lcom-techblog-jq.json[
  {
    "created": "2018-08-22T17:59:13Z",
    "title": "Example Domain",
    "url": "https://example.com/"
  },
  {
    "created": "2018-08-22T01:08:29Z",
    "title": "IANA — IANA-managed Reserved Domains",
    "url": "https://www.iana.org/domains/reserved"
  }
]