Node Package Master

After this awful programming pun in the title hopefully caught your eye, I’m happy to explain, what this article is actually going to consist of: an ultimate node.js package development guide (trust me, it is ultimate) and my personal experience in the field. Let’s start with the latter (if you are not a fan of backstories or gathering arguments to convince your boss, you may very well skip ahead).

My personal experience

I have been working with node.js for the best part of the last 3 years and I have to admit that at the beginning, I had no idea I would end up all over the place. From basic REST API server development to various DB queries, swagger API implementations, WCF communication and a lot more things one could not predict to be working on, when asked to develop a node.js application. And don’t get me wrong, I loved it!

But all of us have been on this roller coaster ride sometime during our development cycle. When you are done with your feature, you are the happiest person on earth and you tend to forget that a mere two minutes ago, you were smashing the keyboard and wondering if the computer is considering today April’s Fool’s Day and it is just messing with you.

This is where node.js packages come into play. A much shorter roller coaster ride with the benefits of reusability, sharing and more moments of satisfaction. For instance, let’s say I was asked to implement a button for viewing a shopping cart content and supposedly it would take me 5 days to do so. I would have to create the button, make it send a request to the server upon pressing, query the DB, return the results and present them in a tasteful way. But wouldn’t it be better if I were to work for 8 days, and besides my requested feature, I would also create a DB querying package and a response parsing package for my company? That way, Bob, my colleague would thank me for shortening his feature development cycle and I would get triple the satisfaction for creating 3 products instead of 1.

I apologize if the last part sounded a little like a sales pitch, I’m sure most of you were already sold on the idea of packages. But I can’t stress enough the importance of insisting on developing them and convincing your boss it is the right thing to do. Investing a little more time now can save you and others much more time later.

Quickly grasping this concept, I began developing private packages for my workplace and sharing them with everybody. My colleagues did the same and needless to say, it was a huge success, because it saved a lot of dirty work for the whole company. It also improved our professionalism as it served as a stepping stone for introducing new technologies and methods, such as unit testing, linting, coverages and many more. It was the perfect platform for minor improvements. At some point I even had the privilege of developing a yeoman package generator specifically for our needs (which by the way I recommend to every development team to do).

But after doing so for two years, I have began questioning the idea of creating packages solely for ourselves. We were using so many public packages, from the npm ecosystem, but when we couldn’t find anything that would satisfy our needs, we developed something outside this ecosystem, even though it could help others in our position.

I decided it was time for me to step outside my comfort zone and create some packages for the public to use (unrelated to my daily work). At first it was fun creating them, but later it became fun looking at the downloads statistics (even if they were low, they still meant that someone, human or robot, was downloading my work). Feel free to take a peek yourself:

• JSDoctor — A smart jsdoc documentation generator.
• texting — Add additional styles to a block of text
• constring — Utility string constants
• semita — A visual hierarchy representation tool

Now, you’ll notice that some of the stuff I was and will be referring to is not implemented in those packages and this is where the most important takeaway from this section shines (somewhat borrowed from the LEAN startup methodologies):

Release your package as soon as its core functionality is done and build upon it in future releases!

Trust me, it’s the right way to go about it. Not because there are no alternatives, but because there would be less chances of you dropping out as a result of frustration, there would be more downloads for your package (each version adds downloads) and most importantly you would have more time to receive feedback and pivot accordingly. The most important features for the public are not necessarily what you envisioned, which is why it is a good idea to wait and hear their requests first.

With all of that in mind, I’d like to proceed to the guide itself (hopefully you will find somethings useful for yourself in there).

Ultimate Package guide

In this section I’ll address a lot of the aspects you should consider, when developing a node.js package. Keep in mind this is not gospel truth. The subsections below are merely advice and recommendations for your development experience. Feel free to incorporate your own techniques, methods and style into your development process. I would also gladly hear about it in the comments.

Note: I haven’t done 100% of the recommendations bellow. Besides my own experience, I have based this guide on heavy research I have done. And even though I intentionally shared imperfect projects on npm, with the intent to improve them over time, I learned new things, which I’ll prioritize over what I had in mind so far and hopefully include them in future releases. I encourage you to do the same: read more, learn more and gradually expand your knowledge and abilities. No shame in not knowing absolutely everything all the time.

package.json

Requirement: must.

According to the official npm docs (which also contain more details about the package.json structure):

This document is all you need to know about what’s required in your package.json file.

Basically, package.json is a file, which describes the current state of your node.js application/project: Basic info. How to use it? What are the usage conditions and requirements? Where to look for additional information? And so on and so forth.

I’ll address here the less obvious parts:

• version (must)— consists of 3 numbers, i.e. “1.2.3” in the format of “major.minor.patch”. Usually starts at “1.0.0”, but oftentimes versions below that are published in order to demonstrate development on the way to the initial “1.0.0” mark. The format is intended to show the level of change each version brings with it. “patch” is for minor bugfixes and changes. “minor” is for new features and changes that don’t break existing code. Lastly, “major” is for big features and changes, which are not backwards compatible and institute adaptations in order to upgrade to them. Please follow these rules when versioning, as they are a considered common convention. I also recommend using the following command in your terminal to perform the versioning:

npm version {patch/minor/major}

• bin (optional)— Contains paths to binary scripts within your project. Mainly for exposing a cli interface for running your module.
• scripts (optional) — An object, which contains scripts for performing certain command line actions. The key is the name of the script and the value is the script itself. For instance: “test”: “mocha --reporter spec”. Once you have written your scripts you can run them by typing in your terminal:

npm run {script name}

• main (must) — The entry point for your module. Whatever you export from that file is what will be required upon using the require keyword with your module name. Usually it is the index.js file, but in some occasions, you might want to export something else (like a dist file).
• bugs (optional) — A url/mail address for reporting bugs.
• keywords (optional) — An array of words, which best describe your project and help people in finding it.
• dependencies (optional) — An object with the key-value pairing of a module you are using and its version. While the dependencies object is used for modules you need for your package’s functionality to work, devDependencies object is used for modules required for your development process solely. Other dependency types also exist.
• license (must) — The license you are using for releasing your open-source code. The most common one is “MIT” as it allows full sharing and usage by the community without special strings attached (I am not a lawyer, so I recommend reading it yourself and not taking my word for it).
• engines (optional) — Similar to the dependencies object, only here you specify the node/npm versions needed to install and run your package.

For further reading, I recommend the following link:

package.json: an interactive guide - browsenpm.org

README.md

Requirement: must.

In my book, a README file is a must. After all it describes why and how users can use your product to their benefit. Without it, most developers may be lost and may decide to give up on your node.js package. You might lose lots of potential users and they may lose a potentially great package. In order to avoid all that, you should write a clear and descriptive README file, which would help everyone get started. It should contain:

• Description — A short one line description.
• Description — If necessary, a longer, more detailed description, in addition to the shorter one.
• Why — A statement explaining, why people should use it.
• Install — “How to install” section.
• Usage — “How to use” section (both programmatically and using the terminal if such an option exists).
• API — Detail here the different functions and options that your package exposes. Make sure you clearly explain their behavior, default values, accepted parameters, return values, aliases, etc. It may be a deciding factor in people’s choice of which package to use.
• Examples — Give usage examples through code excerpts, so it will be clear how exactly it is intended to be used.
• TODO — You may want to share future development plans. It is completely up to you.
• Credits — If your project is heavily based on another package it is considered a common courtesy to thank them. Also some licences require the users of the packages they are attached to, to do so.
• License— Talking about licenses, it is always a good idea to attach your license in your README file.
• Tags — You can add tags to your README file if you believe they hold a substantial value for your users (like amount of downloads or state of dependencies).
• There are many other sections you may want to include, like “About”, “Features Summary” and more. I’ll leave it up to your judgement.

On that note, I would like to emphasize the importance of updating you README file every time you make changes with implications your users should be notified about. I know it can be a drag, but it says a lot about your level of professionalism and can be of great significance.

CHANGELOG.md

Requirement: recommended.

I always recommend adding a CHANGELOG file as it is a great way for you and your users to keep track of the changes that occur in your package. Sometimes those changes can’t be reflected in the README, sometimes you forget about making the change and sometimes your users and potential contributors want to explore your package development even further. Either way, A CHANGELOG file is a great way to address all those issues.

In general it should contain each version as a title and under it every change as a bullet. Additional info like commit links and contributors credit is also welcome. You are free to check how I’ve done my CHANGELOG files in my projects (links in the beginning of the article).

CONTRIBUTING.md

Requirement: optional.

While there is no absolute need to write a CONTRIBUTING file or a contribution section in your README, it becomes a recommendation if you are working on a big project and/or aim to involve other contributors along the way.

I have to warn you, this is not an easy file to write. If you want to cover all your bases, you have to address a lot of different aspects. The upsides are, your project will remain organized, it will look professional and will help in avoiding uncomfortable issues in the future. Also once you write one, there is a lot of copy pasting 😄.

I’d like to give a shout out here to generate-contributing for helping me get started with writing my CONTRIBUTING file. It is very recommended as a basis for your version.

So as promised, here is a list of things you can mention in your CONTRIBUTING file:

• A “thank you” paragraph to everyone who takes their time to contribute. Imho it is very important and shows you value any help.
• A paragraph talking about the authenticity of the submitted content.
• Detailed outline of the different forms of contribution.
• Issues submission and treatment.
• Feature requests format.
• Pull requests submission, prerequisites and other related notes. May refer to tests, styleguides, documentation, work with git, etc.
• Helpful references, i.e. forums, docs and code which may help in deciding whether to contribute and how to do so.
• Other conditions, requests and guidelines you want your contributors to follow.

Code of Conduct

Requirement: optional.

Some projects also include a Code of Conduct file. While it is not an absolute must, the idea is quite nice actually. This file is mainly comprised of definitions of acceptable and unacceptable behavior within the community, as well as the view of the project’s maintainers and applicable enforcement.

While I haven’t written one of those myself, I came across several. I also found a package called conduct, which may prove useful in writing such a file.

LICENSE

Requirement: recommended.

This file should contain the text of the license you are publishing your code under (as mentioned in the package.json section of this article). My advice, generate it with GitHub’s help, when creating a new repository (it’s a built in functionality you can’t miss).

Ignores

Requirement: must.

As with the README, in my book this is a must. There are files you use for development only or you don’t want people to download because they don’t contribute to your package’s functionality in any way. Once you have a file like that, and trust me, you have, you should create an ignore file to prevent uploading it. The most common ignore files in our case are:

1. .gitignore — In here you would like to mention auto-generated reports, IDE generated files and any other file you would not like to see pushed into your repository.
2. .npmignore — Whatever is ignored here won’t be uploaded to npm. I assume you would want here everything ignored by the .gitignore file with several additions, such as the tests directory, README exclusive assets and so on. Basically you want to keep everything development related in your repository, but outside of npm.

I should mention here that upon creating a new repository, GitHub helps you in generating a default .gitignore file as well.

.editorconfig

Requirement: recommended.

This is something I sinfully haven’t used in one of my public projects yet, thought I very much plan on doing so in the near future. This file aims at maintaining consistency of coding style across the different editors and IDEs. The main pitfalls here are whitespace and encoding, which can dramatically influence development if diverse applications are submitted. I mean it in a negative sense. For future reference, you may want to look at this example:

.editorconfig

Project structure

Requirement: must.

Once again, the importance of this topic is heavily based on my own interpretation, but it is not unfounded. The directories and files structure is one of the most confusing subjects out there, so hopefully the following recommendations will help straighten it out:

• index.js — There are some exceptions, but in most cases this should be the entry point of your package. It should be inside your root directory.
• All previously discussed files — All the files we examined above are supposed to be inside your root directory as well.
• configurations — Most of your added tools will require extra configuration files (we will touch on that later). Your package might also require its own too. Commonly those configurations are also placed within your root directory, but if you feel like their amount is pilling up, it is quite okay to setup a config directory instead.
• assets — Often, you will have additional content you need your project to contain: images, audio files, videos and so forth. I usually create a separate folder for them and call it assets. It is also widespread to see a naming convention, such as “misc” or “media”, but I like “assets” more, because I feel it is less restricting and because Visual Studio Code recognizes it as a different type of directory. Once I have my “assets” directory set up, I create sub-directories for the specific assets I have: images, audio, video…
• docs — This one is similar to the assets directory, only it stores documentation. Once I have my docs folder setup, I create specific content oriented sub-directories here as well. For example: tutorials, about, guides…
• Code directories — Here is the juicy part. I have longly followed my own convictions in the matter only to stumble a few days ago onto an amazing project’s structure anatomical breakdown article. I implore you to read it and live by it: https://gist.github.com/tracker1/59f2c13044315f88bee9

The funny thing is, I wasn’t far of base.

Unit Tests

We have finally reached the part of the guide, where we talk about external tools. And what better subject to start with, other than testing?

There are many testing frameworks out there, so feel free to choose whatever you like. As a reference I’ll even throw a great comparison article in here:

JavaScript unit testing frameworks: Comparing Jasmine, Mocha, AVA, Tape and Jest

Personally, I prefer mocha over the rest of them. It’s lightweight, easy to configure, unrestricting and very popular. I have had a blast with it. As an assertion mechanism I use chai and chai plugins if I have a specific need. I also use sinon for stubbing and supertest for API tests. Once again those are my preferences at the time of writing, yours might be different, which is completely fine.

As unit tests development goes, I encourage you to write tests for your package’s functionality as soon as possible. TDD is a great way to go about it, but it isn’t easy and demands quite a lot of conviction and execution quality. The important part is the code you upload should be tested, so you’ll know that the core functionality is working and nothing was broken by you or your collaborators.

At this point, I feel the need to insert a serious advice/joke: Whatever you choose, be it TDD, BDD or something else, don’t do RDD (Resume Driven Development), unless you absolutely hate your colleagues!

Unit Tests Report

Now, here is where unit tests falter in my opinion. While it was fun writing them, it was hard to comprehend the results, as they felt as if they were all over the place. I have to admit that Webstorm does a pretty good job at exposing a dedicated interface to unit tests, but I still feel like it isn’t enough. If several tests were failing I had to hop and jump between the interface and different files and it always felt like an exhausting investigation. And maybe it’s just me and my affinity towards beautiful and clear html reports, but those are my impressions.

Enter mochawesome. A beautiful and slick html report, which is a joy to look at and share with your colleagues. Don’t take my word for it, go check it out for yourselves, I think you’ll reach the same conclusion.

Mochawesome is really awesome!

Coverage

This right here is the lost brother of unit tests. Coverage allows you to check, which parts of your code are untested, what are the specific cases that you haven’t covered yet and your current state of coverage in statistical format. The most popular coverage tool is istanbul and it is very understandable why:

Istanbul

An additional tool you might be interested in checking out is called coveralls. It has many coverage statistical features for your repository, so give it a look.

CI

Continuous integration and continuous deployment are something you definitely should consider. It acts both as an assurance of tested code submissions and as a stronger testing infrastructure for your codebase. In other words you can sleep at night with a CI setup.

While admittedly, I haven’t done CI outside of my workplace, yet, after a tedious (in a good sense) research, I am certain that Travis CI is the way to go. It is great in terms of popularity, presentability, comfort, etc. So I’m on team “Travis” now.

Linting

There are actually quite a lot of linting packages in npm, you would be surprised. Still, like in any other ecosystem, there are top dogs. You can read about them in here and see the comparison for yourself:

A Comparison of JavaScript Linting Tools - SitePoint

I guess it makes my choice very obvious, but was there any doubt to begin with? Eslint it is! It is very easy to use, greatly customizable and it has a built in html reporter. Life could not be any better.

Eslint

Code Quality

During my research, I have stumbled onto this article:

Useful tools for your Node.js projects

While it is a great read from beginning to end, the part that caught my eye the most was about code quality. More specifically it mentioned Code Climate and Codacy as somewhat of an analytical superset to linting and code analysis in general. I have to admit that after reading their websites I wasn’t sold on their necessity and added value, but they are free for open-source, so why not check them out for yourselves?

Documentation

We previously discussed documentation in the form of a README file or the docs folder, but this time I am referring to code documentation, which should be helpful to other contributors in your repository.

There are several ways to go about it:

• jsdoc comments — I believe that this is a great way to add additional data to your code. Whether it is types, examples or descriptions, jsdoc comments allow you to add documentation, without actually affecting your code and in my opinion it is a great documentation option. It is also widely supported by IDEs so I consider it an additional plus.
• flow — compatible with the previous point, flow adds the power of types to JavaScript. Nothing else! This is a huge compliment by the way. Though it adds a compilation layer to your development, the overhead isn’t big and you should definitely consider it.
• TypeScript — I use TypeScript at work and even in a personal project of mine so what I am about to say is not baseless: I am not a fan of TypeScript! I won’t elaborate too much (this guide is super detailed as it is so I’ll leave this topic for another article), but I’ll just say this: People are losing their head over TypeScript and their code remains as bad as it was with JavaScript, only this time with horribly written interfaces. If you are a solid developer you will write great code in TypeScript as well as in JavaScript. And the other way around. But it is a very common choice, so no judging on my part.

It might be clear already, but I usually go with jsdoc comments. Ironically, a whole section could be written on different jsdoc tools / generators alone. So I’ll leave it to this article to sum up:

Choosing a JavaScript Documentation Generator - JSDoc vs YUIDoc vs Doxx vs Docco

As for my choice, unsurprisingly it’s JSDoc. You might have noticed a pattern already, so it is as good time as any to come clean. I like customizable and extendable products a lot. Even if they have other flaws, they are very open for correction and from my perspective is great. I thought I should mention it for objectivity purposes.

Lastly there is the theming issue. The output of JSDoc is pretty ugly and thankfully you can tweek it with themes. Here are some better ones I found:

namis

smeijer/latodoc

toast-jsdoc

loke-jsdoc-theme

Out of those I only used the loke theme, yet I have to admit I am still a bit underwhelmed. I am searching for a better theme to use so I’ll gladly hear any suggestions you’ve got.

Another documentation option I just recently discovered is Raneto. It is a markdown based platform for sharing documentation. It might be of use to you, so check it out!

One thing I beg you not to do is: Don’t auto generate your README from comments or do anything of the sort! Sometimes lazy is good. This is not the case!

Loke JSDoc theme

Task Runners

If this topic seems like a deviation to you, than trust me, it is not. There are amazing things you can do with task runners in package development, like generating reports, cleaning them, exposing a documentation server and so on. But since it is a hefty topic in itself, I’ll leave it to this article (which greatly stresses out the main differences and some of the strong suits of each runner):

Task Runners, A comparison between Grunt, Gulp, Broccoli and Mimosa - Javascript Jedi Padawan

Predictably, I adore gulp and choose it over every other task runner. I know the article didn’t refer to Webpack and Browserify as they are bundlers and not task runners, but it is still a no-brainer for me. In my opinion mystical configuration files is not something that should be considered as positive development or any development for that matter. And gulp is an amazing tool for absolutely every task you want to accomplish.

Yeoman

If there is one section that could be considered a deviation it is this one, but I still have to talk about it.

If you or your team write a lot of private packages, do yourself a favor and create a custom Yeoman generator. Not only is it fun and a great experience, but everyone will thank you and claim it is the coolest time conserving tool they have ever seen.

Summary

It has been a long article, but I tried to deliver the “ultimate” package guide as I promised. Hopefully you found it useful. I would gladly hear any suggestions and comments you have, so feel free to share. Also, every clap, recommendation and social network share is appreciated. Thank you!

Jay Son! Get it? Comical development mind at its best. If you like our content, be sure to 👏 clap, 📱 share and 👣 follow us! Thank you!

Yasha Gootkin is a full-fledged developer, a part-time entrepreneur and part-time musician. Feel free to follow him on Facebook or connect with him on LinkedIn!