What is a dev loop?
In this article, I’ll be talking about the dev loop prior to submitting code for review/submission, commonly referred to as the inner dev loop.
The inner dev loop is the iterative cycle that you go through, as a developer, to write and validate code.
- Edit code
- See results
- Repeat until code is good-to-go
The key to an effective inner dev loop: Minimizing the effort/time it takes to see results.
Mini case study: Writing an AWS Lambda function
I recently had to write a lambda function that would be scheduled to run once per day. The lambda would need to do the following set of work:
- Query data from Firestore and perform some grouping/calculating
- Send messages to Slack
- Send emails via SendGrid
My initial dev loop: Did the job, but brutally slow
The diagram below shows my initial dev loop:
- Write code in javascript
- Deploy code to AWS via terraform apply (~1 min)
- Go to AWS lambda console, press ‘Test’ button (~20 sec)
Problems
- Syntax issues were easy to make
- Syntax issues took about ~2 mins to catch
- Test button on AWS console doesn’t support easily setting different parameters (testing one case would take ~1m20s)
Consequences of a Slow Dev Loop
Laziness to regression test after code changes
It’s hard to test a lot of different cases. So you’ll tend to just assume you haven’t broke something.
Less fun doing the development
Being able to easily see your code working bulletproof is one of the primary joys of being a software developer. Having a slow dev loop robs you of that delicious instant gratification.
Dread about responding to new feature requests
Ever felt like you just want to be done with and never touch an area of code again?
→ This is a big smell that something is not right!
My revised dev loop: Rapid fire validation
I ended up doing some work to land at a much more effective dev loop.
Revised dev loop
- Write code in typescript
- Use ts-node to run code locally (~10 sec)
Problems: Solved!
- FIXED: Syntax issues were easy to make
→ IDE catches issues at code write time - FIXED: Syntax issues took about ~2 mins to catch
→ IDE catches issues at code write time - FIXED: Test button on AWS console doesn’t support easily setting different parameters (testing one case would take ~1m20s)
→ Can add different command line arguments and rapidly run different cases
Benefits of a Fast Dev Loop
Breaking down the solution into smaller parts
I was able to easily test specific chunks of code. This was particularly useful as I was integrating new third party libraries (e.g. Firebase, Slack, SendGrid).
For example, when I needed to add the functionality to send messages to Slack, I ended up adding a --test-slack command line argument to my ts-node app that would simply send a message to a test Slack channel. This allowed me to rapidly and repeatedly call my Slack code, and allowed me to fully geek out on making my Slack messages beautiful and optimal. No compromises.
Empowerment and desire to regression test
Being able to easily prove that your code is awesome is incredibly empowering, and brings a lot of delight.
Eagerness to handle feature requests
Taking on the challenge to see how your code can handle change → aim for this as much as possible!
By making your inner dev loop fast and enjoyable to use, you set your future self up for success, and eliminate the dread associated with touching code that you don’t want to touch again.
If you feel dread about responding to future feature requests in a particular part of the code, understand that you have full agency to reduce or eliminate that dread. The beauty of software engineering is how easy it is to make things better; you’re not moving heavy machinery around, or talking to a bunch of people to get permission (ideally). It’s just code that needs to be changed.
What it took to improve this dev loop
It must be noted that it does take work to improve your dev loop. This must be taken into account. In this scenario, the following describes what it took me to reach my revised dev loop.
Converting javascript project to typescript
This helped speed up code-writing and catch syntax errors quickly. I’ve done this a couple of times before in my life, so this wasn’t so bad for me, but there is a bit of a learning curve here.
My main advice here is understanding that javascript can be executed immediately, but typescript has to be compiled into javascript in order to run. There are many guides out there on how to do this, so use the developer’s best tool (Google) to figure this one out.
Refactoring
I had to split my monolithic code into three parts: 1) core logic, 2) lambda wrapper, and 3) ts-node wrapper.
This took a bit of time, but I personally love doing this type of refactoring. I wanted the core logic of my code to be written in such a way that it could be called from any calling context (in this case, from a lambda and from a ts-node app). I took comfort in knowing that this refactoring was healthy anyway, so it was a wise use of time. This also helped modularize my code with not-too-big of an effort. Modularization is a great thing to improve on as a developer.
Dealing with secrets/API keys
Our lambda deployments were already set up to read secrets from AWS Secrets Manager. So I needed a way for my ts-node wrapper also read such secrets. I ended up writing some minimal code to read secrets from environment variables. This was fairly straightforward.
TS-Node wrapper command line arg handling
I wrote code in my ts-node wrapper to read and parse command line args to be able to test multiple cases in rapid succession. I just parsed and read the process.argv object directly, which was good enough for what I needed. If I needed to do more complex parsing, I would have looked into a node library to help with that.
Important note: This isn’t the only way!
The solution I went with here was very much based on the current situation at hand, and the time constraints that I had. There are many other solutions for tightening inner dev loops. I list some examples below.
Unit testing
This would have been an even better solution to my problem, since unit tests allow you to keep a log of regression tests and run all of them with a single command line.
In my case, there was no unit testing set up for this part of the code. If I foresaw that our lambda code would need to handle much more complicated requirements, then I would have elected to set up unit testing infrastructure.
For front-end code (Angular, React): instant/hot-reloading
Angular and React both allow you to save your code and have it automatically redeploy to your localhost server. This is an incredible trait, as it achieves the goal of seeing the results of your code changes with minimal effort.
Find tools to run cloud services locally
AWS has support for running lambdas locally. This would have been a very cool thing to learn about and become familiar with.
The main reason I chose against this was that I have found these tools — for running cloud services locally — to have somewhat poor support and fewer people on StackOverflow asking questions. So, in order de-risk things, I went with the option I felt more comfortable with. The downside of this choice is I missed an opportunity to uncover a possibly incredibly powerful tool to add to my arsenal.
Main Takeaway: Always been thinking about how to make your development process more effective and enjoyable
We have a lot of autonomy and power as software engineers to make our working lives better. Strive to make your codebase as fun to work on as you can! It will make your life and your product much better.
