Lessons Learned from 809++ Days of Node.js in Microservices
Many of us at Springworks come from a long background of programming within Java and similar languages.
The main reason was that we wanted to use a language that enabled us to focus on delivering end-user value as fast as possible.
Also, it did seem a bit adventurous and challenging at the time, since Node was still a fairly young platform to work with. Let alone for large-scale, enterprise-grade systems. But who doesn’t like a good challenge?
In February of 2014, 2+ years ago, we added the first server-side building blocks for our “platform” for connected car eco systems.
Today, we’ve got more than 50 Github repositories of Node.js (quite a few public, but not necessarily usable for everyone) and are running 15+ microservice applications on Node.js.
These are some of the lessons learned so far, in no particular order of importance.
Using a linter is essential.
Use a shared and code style that makes sense to your team.
Using a linter also makes it easier to keep the code style consistent, e.g. with a shared linter config.
Prefer well-adopted standards to avoid spending time on lengthy, useless discussions about tabs vs spaces.
Have high test coverage.
We strive towards ~90% in bigger modules and even higher coverage in smaller ones. We use Mocha and Should.js with Istanbul for coverage.
Prefer unit tests over integration tests.
Unit tests are faster and cheaper to maintain. They also put pressure on your application design, since complicated code tend to require complicated unit tests.
Only test the public API of your files/modules.
Use mocking to isolate units under test.
Mocking is needed to test isolated parts of the code. We use Sinon.js to mock and spy dependencies (and `Date`) in unit tests.
Be careful when mocking.
Spend extra time in the code review to ensure mocks are correct.
There are lots of great modules on NPM, which you’ll want to use. Keeping them updated is tedious, but Greenkeeper is a real time-saver.
You might want to create something similar for your scoped NPM modules, since they are currently not supported by Greenkeeper.
Also, consider creating a bot that automatically merge Greenkeeper pull requests if all tests pass.
Use promises instead of callbacks.
Promises are easier to use and gives the bonus of catching any errors that might arise in your functions, avoiding uncaught exceptions.
The code tends to be easier to read and understand.
Avoid mixing promises with callbacks.
Mixing callback and promise code gets very messy very fast. It can also be the source of unexpected bugs, as well as weird stacktraces from assertion errors in your test suite.
If invoking a callback from within a Promise chain, use setImmediate().
Write codemods to migrate old code.
Code will get outdated and dirty. Instead of manually refreshing all projects, consider writing a code mod to do the work for you. Automation frees up time so that you can focus on building features for users instead.
Validate input params to all public APIs or use typing.
We use Joi a lot for validation and it’s worked out great for us.
Keep files small and respect SRP.
When the codebase grows, it might get more complicated to manage and modify. To keep it as simple and flexible as possible, keep each file small and respect the Single Responsibility Principle.
It is easier to manage many small files than few bigger ones.
We rarely create files larger than > 100 lines of code, preferably with a single exported function.
However, apply the same principles as with all modularization; don’t create a module if there’s only one consumer.
Either use full blown constructor dependency injection or don’t do it at all.
Mock globally in tests (they’re not run in parallel) and use static code analysis to avoid poor design (e.g. too many dependencies per file).
Keep code simple.
Don’t try to be too smart.
Write web server code agnostic to choice of server.
Web server code, e.g. route handlers, should only be a layer in top of your other services. Keep that layer thin to avoid tight coupling with the web server of choice, to make migration less complicated.
We started off using Restify but are moving over to Hapi, which is more suitable for our needs.
Keep project files organized and use names your peers can understand. As projects grow, it becomes increase important to find your way around it. Structure files in a way that makes sense to everyone in the team.
Don’t underestimate copy-pasting over making modules.
Extracting repeated code to an NPM module might be alluring. But sometimes it’s cheaper to just use the old ⌘+C, ⌘+V. Make a judgment and weigh in factors such as the potential module’s lifecycle, change frequency etc to avoid modularizing too early.
Evaluate and experiment in isolated but real parts of code.
In order to evolve, you need to try new things. Experiment with new patterns and structures, but always do an evaluation of the result to actually learn something (before adopting the change throughout the codebase). Do experiments in real use cases, but try to minimize impact, to make it a cheap decision to change your mind.
Never stop evolving the codebase.
You’ll keep improving how you write your code. Feed new learnings back into the codebase to make it easier to work with over time and to keep the amount of “legacy code” to a minimum. (Minimum does not mean zero.)
Use a highly capable IDE, e.g. Jetbrains IntelliJ.
It’s up to each developer to pick the tools in their toolbox. However, on our team, everyone’s realized that using a great IDE accelerates the workflow, makes it easier to refactor, test, format, lint etc etc. Well worth the money.
Even hardcore Sublime and Vim fanatics have realized this (there’s a Vim plug for IntelliJ that helps out too).
Use strict versions in `package.json`.
You’ll want to know which code that’s actually running in production. Pinning version numbers is a good strategy, paired with Greenkeeper.
Consider using the caret (`^1.2.42`) for dev dependencies if you want to avoid updating `package.json` when it’s not necessarily needed.
Also, the developments with ES6 and ES7 have also taken the language in a great direction.