Bash is good for processing text, but nowadays a lot of stuff is JSON: configs, APIs, etc. And bash is horrible with manipulating JSON. Here we have various tool for processing JSON, and I think your heard or used one of them — jq
But when I want to do something more complex with it I always fail as it has unfamiliar syntax, and, in fact, it is completely independent language. I wanted more simple and more familiar tool for working with JSON.
What is the best language to work with JSON? I think the answer is contained in its name — JavaScript Object Notation. So I created new tool 😁
In fact, two tools — in JavaScript and in Go:
• fx (JavaScript, Node.js)
• xx (Go, Otto, a JavaScript interpreter in Go)
Main idea what you can pipe any JSON in it and pass anonymous function as argument. That’s it. Nothing to remember. 🤗
$ echo '{"foo": [{"bar": "value"}]}' | fx 'x => x.foo[0].bar'
"value"Let’s benchmark fx/xx with jq. For it we will be using awesome hyperfine. We take jq’s example from tutorial on crunching GitHub API commit response.
cat commits.json | jq "[.[] | {message: .commit.message, name: .commit.committer.name, parents: [.parents[].html_url]}]"cat commits.json | fx "this.map(x => ({message: x.commit.message, name: x.commit.committer.name, parents: x.parents.map(p => p.html_url)}))"cat commits.json | xx "this.map(function (x) { return {message: x.commit.message, name: x.commit.committer.name, parents: x.parents.map(function (p) { return p.html_url;}) };})"
Node.js version was significantly slower then jq or xx. But this is due to big startup time of any node application. 🤔What if we benchmark on a really big file?
ll big.json
-rw-r--r-- 1 anton staff 36M 27 jun 00:07 big.json
fx performs twice faster than jq written in C, and more than 3 times faster xx. But how JavaScript version can be faster? It’s because v8 faster jq, and otto.
So should we abandon jq? No, of cause not. But I hope that these new tools can be useful for somebody. And they will find their users 👐.
