3 pitfalls of Serverless Development Ownership
Developing in a Serverless stack can be great. You can work in every layer of the stack to implement a feature from backend to frontend, which is precisely what I have been doing in the AWS Serverless arena this last year. There is no need to delegate a task to a dedicated Backend or DevOps team, and you can pick your programming language of choice (which you perhaps should not).
In other words: Full Development Ownership. But this freedom can come with a couple of longterm risks which might not be that apparent from the start. Let’s review some of them, plus some ways to ease the pain.
1. Reduced Knowledge Transfer
One of the main tasks of any Developer is to handle complexity. This complexity not only lies in the what we know to be complex, but also in what we can not foresee in the first place. After all, who knows what kind of code chaos can evolve in a void¹? That sounds a tad dramatic, but this is also where the combination of Full Development Ownership and working Serverless might start to rub a little bit.
Maintaining a Serverless stack basically comes down to braking up a software system, and reorganising its moving parts into logically independent blocks. The upside of this kind of distributed architecture is improved reliability and scalability of the system, but it also means increased overall complexity.
So if a single Developer can implement a feature from backend to frontend, that complexity is also owned by a single person. This knowledge is ideally shared within a team, either through Pull Requests, Requests for Change, or simply discussing things over lunch. But these routines sometimes lack quality and consistency. For example when starting a new project, exactly when founding architectural decisions are made. Or when there is a high turnover of team members, causing knowledge to ‘leak’ out and making onboarding of new developers more difficult.
2. Decreased Engineering Quality
Developing a new feature requires knowledge on two fronts. First off you need insight into the requirements. This mean knowing what functionality needs to be added (functional requirements), and within what bounds this functionality needs to be available (non-functional requirements). And secondly you need some technical know-how to get started with the actual implementation.
Ideally these requirements are clear when starting out, but the technical know-how can usually grow as you go. This kind of growth mindset, a.k.a. ‘Winging it’, is an often undervalued Engineering quality, which unfortunately can also backfire a bit in in a Serverless stack.
The problem is that decisions made by a single developer in this Serverless stack will often run across multiple infrastructural layers and directly impact the overall system architecture. As such sub-optimal decisions will most likely have a greater impact on future scalability and reliability of your software then a ‘nicely’ scoped issue in a single layer of the stack. Reverting this negative impact can often also be considerably more difficult since you have to deal with operational resources such as databases or user pools, which can be a hassle to migrate in a production environment (looking at you AWS Cognito).
3. Shifting/hidden Complexities
Developing software can be a complex affair. Typically new technical advancements tend to abstract away some of this complexity so you do not have to deal with it on a daily basis. Serverless is no different in this respect since it delegates infrastructure maintenance to a payed service provider which takes care of this for you.
The thing to keep in mind with Serverless however is that it not only hides complexity, but also exposes it again through various backend and computing services which you can cherry-pick to meet your software requirements. This way of working often requires a different mindset for a developer starting with a new feature. You are not only dealing with the code which implements some user interface or a piece of business logic, but also with underlying architectural decisions which impact future scalability and reliability. Not to mention oversights in security or the overal cost-effectiveness of your Serverless stack which you would want to minimise.
All this is not to say that working Serverless is something to be avoided, not at all. These pitfalls are not unique to Serverless alone, but you do run a higher risk stepping into them.
Assuming your project is suited for a distributed architecture, the benefits will certainly out way the risks. And as a Developer working in all layers of the stack you could actually call yourself a bonafide allround Software Engineer. For my part I am certainly developing software with an even bigger smile on my face these days. But before you get stared it might be beneficial to consider these pieces of common advice:
- Formalise your code review process, and get everyone involved from the start. This will help to retain crucial system knowledge within the team, and also speed up onboarding of new developers.
- Team up with experienced Serverless Engineers when starting a new project. A good groundwork for your distributed architecture will continue to instruct newly onboarded developers, and also help to maintain a decent level of Engineering quality on the long run.
- Periodically review the complete system design with the entire team and try to expose hidden complexities. Take advantage of the fact that Infrastructure as Code (IaC) acts as the single source of truth for your entire system.
: Interestingly enough the Greek definition of Chaos actually means
Emptiness. I have certainly seen some projects go from empty to exactly that.