Basho: Shell scripting and automation with plain JavaScript

Update: Basho’s README on GitHub is more recent and accurate. Basho itself has changed significantly, and now evaluates lazily and only as needed.

I was faced with a rather mundane task last week. I had to clone my work from GitHub to GitLab, clean up the projects list and append some text to all the READMEs. With about 150 public and private repositories (of which I wanted to keep 50), manually doing this was not one of the best ways to spend a day.

If your bash foo is up to the task, this would have seemed achievable. Mine wasn’t. I have written midsize scripts before, but all I could remember was the pain in dealing with arrays and strings. Since I’ve been writing a good bit of JS for a while, it would be nice if I could do the heavy lifting in JS and interleave shell commands wherever needed.

Opportunity for a tool that makes life better. I’ll call it basho.

npm install -g basho

Spec and Documentation

Basho evaluates a pipeline of instructions left to right. Instructions can be JavaScript code, reference to an external JS file, or a shell command. To evaluate a JavaScript expression, use the option -j. Let’s start with a single item in the pipeline, a JavaScript constant.

# Prints 100 
basho -j 100 
# Prints true 
basho -j true 
# Prints 100 
basho -j 10**2

Good news is, the option -j can be omitted for the first expression.

# This works
basho 100 
# Prints 100 
basho 10**2

The option -p avoids printing the final result. I am not sure where you’d need to use it, but it’s there.

# Prints nothing 
basho -p 100

Working with strings is a little difficult. Since bash will chew the quotes for itself, you’d need to either use single quotes around your double quotes. So we made a shorthand for this, the -q option.

# Prints hello, world 
basho '"hello, world"' 
# Here's a better way to do this 
basho -q hello, world

You can pipe an expression into a subsequent expression. The variable ‘x’ is always used as a placeholder for receiving the previous input.

# Prints 10000 
basho 100 -j x**2

Execute shell commands with the -e option. The shell command is expanded as a JS template string, with the variable ‘x’ holding the input from the preceding command in the pipeline. Remember to quote or escape characters which hold a special meaning in your shell, such as $, >, <, |, () etc.

# Prints 1000. Escape the $. 
basho 1000 -e echo \${x}

You can import a function from a JS file or an npm module with the -i option. The -i option takes two parameters; a filename or module name and an alias for the import. An import is available in all subsequent expressions throughout the pipeline.

# cat square.js 
module.exports = function square(n) { return n ** 2; } 
# prints 100. Imports square.js as sqr. 
basho 10 -i square.js sqr -j "sqr(x)" 
# Prints 40000. Does sqr(10), then adds 100, then sqr(200) 
basho 10 -i square.js sqr -j "sqr(x)" -j x+100 -j "sqr(x)"

basho can receive input via stdin. As always, ‘x’ represents the input.

# Prints 100 
echo 10 | basho parseInt(x)**2

If the input to an expression is an array, the expression or command is executed for each item in the array.

# echo 1; echo 2; echo 3; echo 4 
basho [1,2,3,4] -e echo \${x}

An input can also be an object, which you can expand in the template string.

basho "{ name: 'jes', age: 100 }" -e echo \${x.name}, \${x.age}

You can use an Array of objects.

# echo kai; echo niki 
basho "[{name:'kai'}, {name:'niki'}]" -e echo \${x.name}

Array of arrays, sure.

# echo 1 2 3; echo 3 4 5 
basho "[[1,2,3], [3,4,5]]" -e echo \${x[0]} \${x[1]} \${x[2]}

A command can choose to receive the entire array at once with the -a option.

# echo 4 
basho [1,2,3,4] -a x.length -e echo \${x}

That’s useful for filtering arrays.

# echo 3; echo 4 
basho [1,2,3,4] -a "x.filter(x => x > 2)" -e echo \${x}

There’s a shorthand for filter, the option -f.

# echo 3; echo 4 
basho [1,2,3,4] -f x>2 -e echo \${x}

There’s reduce too. Here’s the long form.

# Prints the sum 10 
basho [1,2,3,4] -a "x.reduce((acc,x)=>acc+x,0)" -e echo \${x}

Shorthand for reduce, the option -r. The first parameter is the lambda, the second parameter is the initial value of the accumulator.

# Prints the sum 10 
basho [1,2,3,4] -r acc+x 0 -e echo \${x}

Btw, you could also access an array index in the template literal as the variable ‘i’ in lambdas and shell command templates.

# echo a1; echo b2; echo c3 
basho "['a','b','c']" -e echo \${x}\${i}

You can extend the pipeline further after a shell command. The shell command output becomes the input for the next command.

# echo 110 - which is (10^2) + 10 
basho 10 -j x**2 -e echo \${x} -j "parseInt(x)+10" -e echo \${x}

There’s nothing stopping you from using all the piping magic built into your shell.

# Prints 100 
basho 10 x**2 | echo

Promises! If an JS expression evaluates to a promise, it is resolved before passing it to the next command in the pipeline.

# Prints 10 
basho "Promise.resolve(10)" -e echo \${x} 
# Something more useful basho -i node-fetch fetch \
 -j "fetch('http://oaks.nvg.org/basho.html')" \
 -e echo \${x}

Typing basho without any parameters does nothing but might make you happy. Or sad.

Advanced

You can reference the output of any previous expression in a pipeline with the --stack option. The parameter can be an index indicating how many steps you want to go back, or it can be a range. Examples below.

# Prints [2,3,4,5]
basho [1,2,3,4] -j x+1 -j x+2 — stack 1
# Prints [2,3,4,5]
basho [1,2,3,4] -j x+1 -j x+2 — stack 1,2 -j x

To turn off saving previous results (for performance reasons), use the nostack option. Turning it off is hardly ever required, except when you’re dealing with huge text transforms.

basho --nostack ["a.txt", "b.txt"] -e cat \${x} -j x.length

Tip

If you need the fetch module (or any other) often, you’re better off creating an alias for basho in .bashrc (or .bash_profile on a Mac).

# in .bashrc
alias basho='basho -i node-fetch fetch'
# now you could just do
basho "fetch('example.com/weather')" -j x.temperature

Real world examples

Count the number of occurences of a word in a string or file.

echo hello world hello hello | \
  basho "(x.match(/hello/g) || []).length"

Get the weather in Bangalore.

echo Bangalore,in | basho "fetch(\`http://api.openweathermap.org/data/2.5/weather?q=\${x}&appid=YOURAPIKEY&units=metric\`)" -j "x.json()" -j x.main.temp

That’s all folks. Code is on GitHub for now. Ping me on Twitter if you need anything.