My development/deployment stack for freelance projects
I’ve been a freelance developer since roughly 2012, and worked for various companies before then using a variety of languages, technologies, means of deploying and distributing software and so on. Since going freelance I’ve come to really appreciate certain principles that tools and technologies fulfil to varying extents.
Principles
Fast iteration
This becomes especially valuable when you’re the sole developer on a software project. If your platform slows you down, gets in your way, and you miss deadlines, it will all be on your shoulders alone. But if you beat deadlines and exceed expectations, (most) clients will really appreciate it and value your work (and way of working). Fast iteration doesn’t stop at implementing features, but also includes fast, effective deployment that meets the modern expectation of continuous delivery models and the “it ain’t finished until it’s in production” philosophy.
Guard rails
There are many tools and techniques that prevent you from making mistakes, introducing regressions and generally shooting yourself in the foot. I’ve worked on projects with no protections (classic PHP-style) and many (static languages with a full test pyramid of unit, integration and end-to-end test suites running in pre-commit hooks, pre-push hooks and CI pipelines).
There are definitely benefits, but as with everything, it’s about weighing tradeoffs. While more unit tests might make it safer, writing and maintaining them will reduce your agility and your ability to iterate fast. For small projects this can be an issue. If adding tests reduces your defects by 40% then it’s worth considering. If the nature of the code you’re writing (maybe it’s just very straightforward stuff, or maybe most of the real lifting is done by your framework or libraries) means more tests only prevents one or two bugs hitting production, you have a responsibility to challenge whether it’s worth it.
Fast feedback
Straight from the Lean Startup. In my experience, there are two types of feedback to consider: human feedback (from product owners, other developers, and of course users!) and technical feedback (from your language, platform, test suite, CI pipeline, crash reporter…). Decreasing the latency in these feedback loops will mean you can fix things faster, spend less time debugging, and have more chance of delivering features that are accepted first time instead of going through endless iterations.
Low cognitive overhead
Again related to the first item. It’s very simple: when you have less things to think about, you can concentrate on what matters. In my experience many people really are their own worst enemies in this area. Things like over-engineering, not using tried and tested tools they’re familiar with, feature creep, not following KISS and YAGNI when coding. YAGNI. Really, you aren’t!
Of course, this also has implicit tradeoffs. Maybe that mail sending component you’re writing that’s tightly coupled to AWS’s SES API actually will be replaced in 3 months. Maybe it’s even on a product roadmap somewhere? If you have a genuine reason to write a particular piece of code in a very generic way that will allow it to be extended or replace easily in the future, then maybe you should do that. But as a sole developer on a freelance project? Be careful. Be vigilant.
The stack
My technology stack won’t be the same as yours. It’s taken me years of experimentation, learning valuable lessons, dipping my toe in the bright and shiny river (hello Meteor and GraphQL) to get to this point. It isn’t suitable for every project, but having now developed several successful line-of-business webapps, I’ve decided I’d be very reluctant to change it now.
Backend API’s: SQL database, node.js, express, Sequelize, pm2
Why: I have a lot of experience with JavaScript. It speaks native JSON (agility!), has great automated testing libraries (guard rails - jest is awesome), is easy to configure and deploy. Sequelize is a mature, full featured ORM that works with pretty much every mainstream relational database.
What, RDBMS? Is that agile? It’s a tradeoff. I’ve used MongoDB and came back to relational databases. Maintaining migration scripts is not the most fun in the world but the advantages outweigh the downsides in my opinion: data integrity (foreign key constraints, transactions, ACID as standard), familiarity (most business analysts or vaguely technical people can use SQL to query their data). Also most projects I’ve worked with (always base your tech choices on project use cases and requirements) tend to have structured, relational data, not documents or ML models.
Frontend: React (TypeScript, create-react-app, redux, hooks, date-fns) or Angular
New Angular is really great, and all React developers should give it a serious try. Use the right tool for the job! For line-of-business apps with lots of forms, Angular really has an edge and its developer experience is much, much better than the old days of AngularJS.
For apps that have more “custom” functional requirements, like complex UI’s with more finer-grained components, React may let you iterate faster. Try to choose one set of packages you like and stick with it, even if something supposedly better pops up next year. I don’t use react-router or any of the form libraries because they’re a) quite complex and b) tend to have high API churn. Use redux for really central state and useState hooks for smaller, local state like form inputs. Date-fns gets a mention because what project doesn’t require some date manipulation? Its clean functional style is no-nonsense and lets you get stuff done quickly.
The common theme on the front end is my use of TypeScript, on every project I do. Again the trade off from agility to guard rails, but I find in some cases, having typings in your IDE actually lets you go faster because you don’t need to look things up which reduces your cognitive overhead. Adding static typing doesn’t sound like it reduces cognitive overhead, does it? Well, once you’ve done the groundwork (I find I get the most value from typing all of my models coming from outside of my system, as these are the hardest to look up, validate or reason about), it will make you more productive, and catch lots of bugs — especially those “Undefined is not an object” kinds, because every type that can be undefined will be marked as such so you know when you need to handle null cases and when not. Lastly, typings in your IDE give you very fast feedback — as you’re typing.
Development workflow
With create-react-app or Angular for the front end I get live reloading out of the box. I use nodemon for the API code to keep that reloading as I change code.
Automated testing, deployment, crashes: jest, bash, scp and mail
My deployment system is probably too simple for larger projects, but for those where I’m the sole developer, it gets out of my way and lets me focus on delivering.
The number of unit tests I write heavily depends on the nature of the code I’m working on: how complex and how business critical is it? TypeScript eliminates some classes of bugs, but for mitigating logic bugs, sometimes a solid unit test suite is exactly what you need. I use jest exclusively because it has all the out-of-the-box tooling I need for writing and running tests, and I love its fast feedback loop in watch mode.
For deploying, I use embarrassingly simple bash scripts with scp and ssh to deploy to DigitalOcean droplets. I don’t even use ansible or any provisioning tooling, though I probably should (if I’m claiming to learn from my experiences…). Again though, this system is simple, has low cognitive overhead, and lets me release things quickly and often so users can give me their feedback.
A basic build script for a node/React project looks something like this (they live in the root of the project):
rm ../AppName_*.tar.gz
cd app-name
npm run build
cd ..
tar — exclude “./.git” — exclude “./.vs” — exclude “./node_modules” -pczf ../AppName_`date -u +”%Y-%m-%dT%H%M%SZ”`.tar.gz .
Sometimes the build script also deploys the app, using scp to copy it to the server and ssh to run the commands to unextract the files, restart pm2, and perhaps backup the database.
Finally, I use pm2 logs to look at logs, and have an UnhandledPromiseRejection handler in the index.js of the API scripts that maps the exception and stacktrace to an email and shoots it into my GMail inbox. No matter what your logging level is, fix those crashes!
I’m very happy with this stack, but there is room for improvement. In the future, in roughly this order:
- Start using TypeScript on the server too
- Write end-to-end tests for business critical functionality and happy paths (probably with jest and puppeteer)
- Refactor those copy-pasted nginx conf files into an ansible project (though the benefits aren’t that great yet… reprovisioning the server in the event of catastrophe is still at the “oh crap, well this will take up some time but maybe I can use the event to finally put things in ansible” stage instead of the “oh crap I’ll never remember all those packages I installed, time to move everything to Heroku” stage
Footnote
This has been long, but I’d really like to give an honorable mention to Ruby on Rails. Even though I no longer use it for my projects, it has inspired so many tools and practices in our industry, showing us how to do things better. It’s still a great platform with amazing productivity for some use cases, and I wouldn’t hesitate to work on a Rails project again in 2019. Rails, you rock!
