Hacking vs. Prototyping vs. Production Code

How my personal command-line tool grew up to become a professional product

Not all code is built the same. The shelf life of your code typically depends on your motivations for building it:

• If you’re throwing something together at an all-night hackathon, then the code doesn’t need to survive past 24 hours — it is disposable.
• If you’re making demo code that others can use as guidance, then it may be written in a clear, but oversimplified, way. In this case, legibility is more important than performance or great architecture. The code is more than “disposable” — it exists as an educational aid, rather than an exemplar of best practice.
• If you’re building an open-source library that you’re expecting others to use and adopt, then you should have automated tests that exercise each line of code, to catch unexpected failures over time. This approach is for code that lasts the long haul.

Ideally, all of our code is built to the highest professional standards, but we often favour expediency when prototyping a new idea. Writing code to the highest standards takes a lot longer.

When some of your demo code is taken up and used by others for a serious purpose, then you get to see the difference between your “minimum viable product” hack and production-ready code.

This happened to me with couchbackup. It was time for my code to grow up.

Rugrats: All Grown Up is property of Viacom International Media Networks. Image credit: FANDOM TV Communities

The story of couchbackup

I like building command-line tools. As I’m a developer advocate for IBM, and Cloudant (an as-a-service Apache CouchDB database) falls under my purview, the tools are often for CouchDB: couchimport to import and export CSV data, couchshell to interact with your database like a command-line shell, and several others on npm. The couchbackup project is one such tool. It allows databases to be backed-up to text files and for the backup files to be restored to databases.

Here’s what a basic couchbackup interaction looks like:

# backup
couchbackup --db mydatabase > mybackup.txt
# restore
cat mybackup.txt | couchrestore --db myotherdatabase

Like most of my tools, it was built for me but published and open-sourced in case anyone else wanted to use it.

One day, IBM decided it would take couchbackup and make it production-ready. This work would be undertaken by the Cloudant libraries team, who build the officially-supported code that allows Cloudant to be accessed from Python, Java, Objective-C, Swift and Node.js.

What would they change? Would they take my ideas and rewrite the tool in Python or Java? Let’s find out.

Making code production-ready

Now that couchbackup v2 has been released, I can look back and see what code was changed and the work that the libraries team did. Here's a summary of how they took my demo code and made it production-ready:

• Instead of taking a shallow copy of the winning revisions of all the documents in the database, couchbackupnow keeps all undeleted document bodies.
• The code now features an automated test suite. If folks are going to rely on your tool to back up their precious data, it better do what it says it’s going to do! The codebase is still open-source, but pull requests must pass the test-suite and be reviewed by a contributor prior to acceptance.
• The API was tweaked: parameters were combined, naming is more consistent, and it is easier to invoke a programmatic backup.
• Events and error-handling are improved. The code behaves more predictably in error conditions. It fails fatally for permanent errors (such as authentication failure) but retries temporal errors (such as timeouts).
• Code style rules are in place. The entire codebase is “linted” to the same standards. Machine-readable comments are in place above each function, and a greater level of consistency is applied across source code files.
• Performance improvements: allowing extra pairs of eyes to see how an algorithm performs can lead to optimisations. Connection pooling of HTTP requests and memory usage reduction by controlling volume of data consumed from a stream are just two examples.
• A host of bugs were fixed and features added.

When your code is scrutinised by others, there will also be other optional changes that are a matter of taste:

• function(data) { } or (data) => { }
• callbacks, or Promises
• semicolons
• indentation
• function lengths
• module size
• build tools
• testing frameworks

Reworking someone else’s code can be like editing someone else’s essay. As well as fact-checking and improving the work, some changes will be matters of style, convention, and personal taste.

Is the code better than it was? Undoubtably.

Is it still open-source? Yes! Feel free to use, modify, and contribute changes to the project.

Using couchbackup

You can install couchbackup on your Node.js-enabled machine with:

npm install -g @cloudant/couchbackup

Backing up a database is as simple as:

couchbackup --url http://localhost:5984/mydb > mydata.txt

There are other options that allow a partial backup to be resumed and performance characteristics to be tuned. See the README for more details and examples.

Credits

Thanks to Mike Rhodes, Rich Ellis, Sam Smith, and Tom Blench from the Cloudant team for all the excellent work they have put it to make couchbackup workable.

If you enjoyed this article, please ♡ it to recommend it to other Medium readers.