Halloween Serverless Stories — Part 1
Most of us, who are working Serverless, find it irreplaceable. There are so many pros on which we are hooked nowadays, that it is hard to switch back. However, some people, on their quest for a successful Serverless application, have seen its dark side 🎃 👹 ☠️ ….
Building a Serverless application can be pretty neat and may give us a false impression of a very low learning curve. But remember, you are developing an application that is deployed on the most potent computational/storage platform that ever existed on the planet. Having in mind that you are paying for what you use, and if you are wasting it, it can be extremely expensive…
Let me tell you about a few stories that happened to me, my colleagues or I read it somewhere.
Some of them are very scary, so make sure you don’t read them before sleeping…
You may find yourself checking the account(s) in the middle of the night.
Story 1: Most expensive cost awareness
A friend of mine and his team were working on some Serverless applications for some startup. The client was so worried about the costs, as they are very tight with the budget.
They decided to create a reporting tool so that the client can be aware any second how much cost is created. The client was overwhelmed by the idea of having a report of the costs anytime he checks.
They know that AWS Cost Explorer is the service they need. It has the proper API and it seems like they have exactly what they need. Together with the client, they decided that slack reporting would be very easy to integrate and available to both stakeholders and the team.
So.. rather easy… anybody could do it!
The team started development…
Everything went so smoothly, the API was self-explanatory and worked as expected. Slack was also very simple to be integrated, with just one POST request. It is going to be done in a couple of hours so the team can go on a relaxed long weekend ☀️🏖
One developer had an amazing idea, to report only when there are changes to the costs, not to bother every few minutes.
They called Cost Explorer API very frequently (I am not saying they have put it in a loop 🤣) and then compared it with the previous state.
Nobody needed any documentation, everything was very easy.
They deployed it and left for a long weekend to relax and enjoy beautiful summer…
Nobody checked slack for a few days.
The client didn’t even install slack 🤣
When they got back in the office, the client’s a bit disturbed voice, gently notified them that the bill over the weekend was raised by 8000$ and their monthly bills were at an average of 50$ — 70$. That woke them up better than any horror movie before.
After some investigation, they have realized that they are using paid services!!! When you call a certain API on AWS it has its price.
The price of API Cost Explorer is 0.01$ per request.
Imagine if you call that service in a loop and leave it for a few days, to realize that that service is updated once or twice a day 🤣
Luckily AWS reduced significantly that cost so, in the end, it all went well. But it remained planted in the developer's brain to check the prices before they started using any AWS service carelessly.
Story 2: Malicious 3rd Party Library
Everyone wants to do their job as quickly as possible. So we, as developers, tend to use libraries in order to shorten our development time. However, apart from libraries being buggy, they might be malicious also.
This is not really a story but more heads-up notice. So let’s say that you use a library that wraps some APIs and creates an SDK or some algorithm that you would have to implement yourself otherwise. However, that library might do the job perfectly but on the other side, it might be malicious also. It might carry some code that for instance mines some cryptocurrency. That might be a very big issue in case your app scales a lot. You may provide some of your processing power for somebody's interest, but you might not be aware of that.
Here is an example of how easily this can happen. Let’s assume you use nodejs and have stated in your package.json
"dependencies": {
"some_dep": "^1.0.0",
"another_dep": "~2.2.0"
}So let’s say you checked utterly some_dep 1.0.0 and you are convinced it does exactly what you need it to do.
But…. Guys that developed some_dep 1.0.0 have malicious intentions. So they saw that they have so many downloads, and so many people are using the library. They decide to update the release with some_dep 1.0.1 stating that it is a bug fix in the change log. Next time you build a new version of your app, it will carry malicious code.
In the package.json you stated that you are ok with updating for patch release (^) or minor release (~). That may leave an open door for potential attacks.
In essence, you should be aware of the libraries that you find convenient for your app and make sure that they are developed by some trusted, well-known publisher.
To be continued…
There will be more stories coming up till Halloween, so stand by for some more scary stuff. 👻👻👻👻
