Using Node.js to Write Safer Bash Scripts

It’s easy to get Bash wrong, it’s hard to debug and mistakes could have disastrous consequences

Pure Bash Scripts are Dangerous

Valve deletes all of your files — Scary Bash story number 1

On January 15th, 2015, Github user keyvin submitted an issue in Valve Software’s steam-for-linux repository.

Moved ~/.local/share/steam. Ran steam. It deleted everything on system owned by user. · Issue #3671…

[…] Ran steam. It deleted everything on system [...]

Oh, no... Can you imagine being one of Valve’s developers and seeing this issue on Github?

The cause of this terrifying bug was a shell script used to launch the Steam client, that contained the following snippet:

# figure out the absolute path to the script being run
[...]
STEAMROOT="$(cd "${0%/*}" && echo $PWD)"
[...]
# Scary!
rm -rf "$STEAMROOT/"*

The script had a bug. Under certain circumstances the variable STEAMROOT was empty, and the command rm -rf "$STEAMROOT/"* turned into the infamousrm -rf /*. You can tell whoever wrote this script knew this was frightening code, they even included a comment calling it "# Scary!".

Bash deletes all of my workweek — Scary Bash story number 2

On October 26th this year (sadly, a Friday), our very own CI builds started failing due to a test’s results:

AssertionError: expected 0 to equal 3

I spent the better part of a day researching what made the test suddenly fail. No code or configuration change relevant to that test was made. Going through the logs of our CircleCI build took a while, but eventually I found an error in one of the 918k lines of our log for the mock-data-load step before the tests are run:

2019-10-27T07:55:14.621Z - error: An unexpected error has occurred 
{ cmd: 'elasticdump ...some-params...' }

elasticdump is a CLI tool we use to load static data into our elasticsearch’s indices for testing purposes. It had a minor version update that same day which had a bug, causing the load step to fail. Our CI scripts, written in Bash, very helpfully output the error message and then gleefully continued, with this bug waiting silently. Eventually the test failed with an error message that was not helpful in finding the actual bug, wasting a lot of useful development time. (By the way, elasticdump’s maintainers had responded very quickly to the issue I’ve posted on their repository and we had green build the very same day!)

Why did this happen?

Almost all software organizations use shell scripting in one way or another. Maybe it’s a 3-line script that just does “npm install, npm test, npm publish”, or a 20-line script meant to install everything a coworker needs on their 1st day. Maybe it’s a 50-line script, calling one of three other scripts as a function of some environment variable, which in turn use ansible to remotely execu — … nevermind on that last one. The point is — we all use shell scripting in some manner.

There are very good reasons for this:

• It runs (almost) everywhere without dependencies
• A lot of tools we need are CLI-only, no language-specific package for them exists which makes it a fantastic “glue” language for chaining other tools
• It’s super easy to bootstrap a new script — just touch script.sh, chmod +x script.sh and you’re done

However, shell languages (specifically, Bash) have some very surprising pitfalls. The languages are archaic and have very unintuitive syntax and behavior for people who are used to more modern language. It’s possible to learn, but most of us only write Bash when we need it — we never spend time actually learning it.

My proposed solution is to thinly wrap Bash executions with a tiny package that abstracts away a lot of the Bash pitfalls many of us never knew existed. I’ll show you such a wrapper for Node.js, but first let me try and convince that pure Bash will cause bugs in your software, if it hasn’t already.

Surprising Bash Pitfalls

Errors mid-script are a-okay

Look at this innocent script that could act as a super simplified continuous deployment script for some open source package:

npm install
npm test
npm publish

You would expect that if npm test fails, the script fails. However, Bash’s default behavior is to march on, and perform npm publish even if the tests fail. Not good. You could solve this by adding set -e to the beginning of your script which makes the script exit immediately after a failed command with that command’s (non-zero) exit code.

Errors mid-pipe are just as fine

Even if you’re enough of a Bash wizard to set the -e flag so your scripts exit on failure, you’re still out of luck if you have a failure in a pipe chain. The following command, which runs lint on all the .spec files in the current folder and saves the result into a file, has a typo:

ls | grepp .spec | eslint --stdin-filename | tee result.txt

The command grepp does not exist, eslint would not run because it didn’t receive a file, and tee would create the file results.txt but write nothing into it. The worst part — the script would continue to run. By default, Bash does not count a failure mid-pipe as a command failure.

To make scripts exit if they encounter an error mid-pipe, you need to add set -o pipefail to the beginning of your script.

Using undefined variables is completely acceptable

Bash allows you to read from environment variables you’ve never set. Example:

rm -rf $BUILD_FOLDER/*

This dangerous code would run just fine even if you forgot to set the value of BUILD_FOLDER. Very similarly to the 1st example in this article, this would end up running rm -rf /*… By default, Bash allows you to use undefined variables and considers their value to be an empty string.

To make your scripts exit when they encounter an undefined variable, you can add set -u to the beginning of your scripts.

The 80/20 Solution

You might be tempted at this point to say - “Okay, Bash does have some behavior that’s surprising and dangerous by default, but if I add set -euo pipefail to the beginning of every script I write, I should be fine!”. You would be absolutely correct. If you stopped reading this right now and added that magic line to all of your scripts, I would be delighted — I would feel like I’ve done my job.

However, there are additional advantages to wrapping your shell interactions with a modern language.

• You and all of your teammates need to remember to add the magic string to all of your scripts from now until forever (do you add this to your team’s training for new workers?).
• It’s really hard to do a lot of things in Bash that are really easy to do in other languages.

More Bash Pitfalls

Basic language features like string manipulation are easy to get wrong

How would you extract the Nth word from a string in Bash? Unless you write Bash scripts daily, you probably had to look up one of the many solution available, and they probably require you to learn another tool like awk, sed or cut:

echo $STRING | awk -v N=$N '{print $N}'

If you already program daily in another language, wouldn’t you rather use that language’s features?

Basic language constructs like conditionals / loops are easy to get wrong

How do you check if a variable’s value is greater than 0 in Bash?

if [$VAR > 0]; # Syntax error, forgot spaces
if [ $VAR > 0 ]; # ">" Means stream redirection
if [[ $VAR > 0 ]]; # ">" Means comparison, but of strings (lexicographic comparison)
if [[ $VAR -gt 0 ]]; # Finally works!
if (( $VAR > 0 )); # Confusingly, this also works...

This also holds true to loops, iteration and other constructs.

No package manager

Bash’s “package-manager” is the OS’s package manager. If you want to use some great 3rd party package you better hope it has a CLI interface for your operating system. And if you want to write reusable modular code, you need to either resort to copy-pasting it, or using source to import functions — but be very careful to set your pwd as expected by the script, or source won’t find the script you’re importing!

Proposed Node.js Solution

We wrote a tiny module that solves all of the above for Node.js: @getvim/execute. The module wraps Node’s built-in child_process.exec function to create an API for shell interaction that’s simple, intuitive and predictable.

Usage

@getvim/execute is Promise based.

The Promise resolves to a string- the output of a shell command.

The module supports setting environment variables for the command.

Usage of the pipe function allows easy switching between Node.js and Bash context.

Pipes allow usage of modern language features / modules in your Bash script.

Error handling

All commands are wrapped with set -euo pipefail for safer script execution. This means that a command will fail (execute’s Promise will be rejected) if:

• It returns a non-zero exit code
• A mid-pipe command returns a non-zero exit code
• An undefined environment variable is used

The error object contains the command’s exit code, stderr and stdout so you can debug the origin of the failure. Using an undefined variable, for example:

Not only are you protected from accidentally deleting the entire filesystem, you also get a very useful error message: BUILD_FOLDER: unbound variable!

Disadvantages

@getvim/execute is extremely opinionated. It was designed to be inflexible so as to remain simple to use and predictable even for programmers who don’t write complex scripts daily. The design decisions bring some disadvantages with them:

• @getvim/execute is Bash only.
• Results are always a string. If you’re expecting multiline results, you must split the results with a newline character.
• The module is not stream-based, it’s Promise based. This means that long-running commands will execute sequentially and the memory foot-print will be large, since the output of one command is aggregated before passing it to the next command.
• There is no implementation for user input — if a command expects a user’s interaction, a Y/N confirmation for example, execute’s Promise will simply never resolve.
• There is no implementation for printing a command’s output while it’s running.
• Performance is slower than writing pure Bash scripts, as it takes a little while for Node.js to invoke a new shell using child_process.exec. However, once the command is running it is just as fast as if you ran it from a Bash script, so performance hits should not be noticeable unless you’re invoking a very large number of commands.

Conclusion

Bash is a fantastic language for gluing together parts of your workflow, but as your codebase grows, you start seeing its dangerous side. All of the disadvantages I’ve written above can be solved using Bash-only tools, but most of us programmers are not even aware of them.

I hope I managed to convince you that using pure Bash scripts can be dangerous, especially if done by people who don’t write Bash regularly. This can be solved by wrapping shell commands with a small module that protects its user from most Bash pitfalls, and on the way you gain other cool stuff like usage of more modern language features and of 3rd party modules. For Node.js users, you’re welcome to use our recently open sourced module — @getvim/execute.