How many AWS services do you need to host a static webpage? (2/2)
4 AWS services are good for a static website hosting, but don’t forget we had a visitor counter to deal with. How many services are needed for this?
Storage
Counter’s state has to be stored somewhere. Amazon offers a NoSQL database, but I was afraid it was too much. I googled a bit (with duckduckgo.com 😛) and found something which calmed me:
After reading that, I’ve created a simple table in DynamoDB (5). It’s a key-value store, so you can have an entry like that:
Lambda (6)
The logic behind the counter: to get its value, increment it, store a new value, return it through HTTP.
It’s not much, but it’s too much for a simple database. I went for AWS Lambda, which allows executing a simple code snippet. Serverless.
The best way of creating Lambda with an access to DynamoDB, exposed to the outer world is to create it from a blueprint. There is an existing template “microservice-http-endpoint” which does everything we want. With NodeJS:
Behind the scenes, this blueprint configures also CloudWatch (7), API Gateway (8) and the corresponding IAM roles and policies (9).
Wow.
IAM
Let’s start from the end. IAM stands for Identity and Access Management. This is a formalized way of handling permissions. S3 Bucket Policy was just for S3 and here we have a central place for all the policies.
Blueprint adds us a single role containing 2 policies: for Lambda to read from and write to DynamoDB and for Lambda to write to CloudWatch.
Roles and polices defined in IAM can be reused all over the services.
CloudWatch
This is a place when you can monitor your services — check logs and metrics and see some nice charts, telling you e.g. what is the average execution time of your Lambda and so on.
API Gateway
This is your way to expose something like Lambda from AWS! The screenshot tells it all:
Each building block is a place where you can add something, like e.g. additional mapping to the flow.
In API Gateway you can also configure which HTTP methods you want to support, you can go for Actions and enable CORS (it was needed to make a call from the browser) and you can specify stages on which everything is available. Example stages would be dev, test and prod. Then, you can expose different Lambdas under different stages or have the same Lambda, but e.g. with different Gateway limits, depending on the stage.
Show me the code!
As mentioned, you can write your Lambda logic using NodeJS. I used the 8th version of the runtime, but you can also choose Node 10 recently.
What is nice in both versions you can use async/await and then, what is returned, will be a response body and what is thrown and not caught, will cause an error response.
const AWS = require('aws-sdk');
const doc = require('dynamodb-doc');
AWS.config.update({ region: 'eu-west-1' });
const dynamo = new doc.DynamoDB();const TableName = 'table';
const Key = { id: 1 };exports.bumpCounter = async(event, context) => {
const {Item: {counter}} = await dynamo.getItem({TableName, Key}).promise();
return {
statusCode: 200,
body: JSON.stringify(
(await dynamo.updateItem({
TableName,
Key,
UpdateExpression: 'set #C = :counter',
ExpressionAttributeNames: { '#C': 'counter' },
ExpressionAttributeValues: { ':counter': (counter + 1) },
ReturnValues: 'UPDATED_NEW'
}).promise()).Attributes
)
};
};
At the beginning of the snippet, there are imports to use AWS mechanisms. To be 100% sure everything happens in my region of choice, I set it at the beginning with AWS.config.update function.
What is exported is our lambda. It’s an async function, which consumes 2 parameters — an event which triggered Lambda (HTTP request and its details) and the Lambda context. In the context you can have e.g. variables, which can be defined when configuring Lambda.
Then, I use DynamoDB client, which can work with await thanks to its promise() method. And, personally, I found these Expression things from updateItem confusing. At the beginning I had an update expression like
`set counter = ${counter + 1}`but it was problematic as counter is a reserved DynamoDB keyword and there were some problems with the value passed like that. Therefore, I used template mechanisms from the client, where #alias is an alias for a table column and :alias — for its value.
It’s worth setting ReturnValue to 'UPDATED_NEW' to get a response with the changed database columns. Accessing Attributes from this response will give us exactly that.
In case of errors, you should rely on CloudWatch. I found it really helpful together with a plain ol’ console.log.
To be super safe when it comes to costs, you can set a number of possible parallel lambdas to 1.
What next?
Adding a simple visitor counter to our webpage, brought up another 5 services: DynamoDB, Lambda, CloudWatch, API Gateway and IAM.
But it’s not over.
Everything can be improved, e.g. by limiting CloudFront just to European countries and by limiting the access to S3 — it could be available only for CloudFront.
Also, what if our webpage becomes super popular and we will get close to DynamoDB/Lambda limits?
I encourage you to start #AwsVisitorCounterChallenge and to create a visitor counter with as many AWS services, as it makes sense. On your own!
Bonus: Chrzonstowski’s law
With my simple webpage I learned a lot about cloud computing. And reading about all the services and AWS users made me think about when should or shouldn’t you use a public cloud infrastructure.
As there is an empirical Conway’s law, I decided to make my own. If there is already something like this, just let me know. If no — enjoy a new law 😁
The larger the company the less, the more, the less and the more public cloud computing it needs.
[hypothesis] Until becoming a new public cloud provider.
It makes sense, doesn’t it? The higher on the y-axis, the more public cloud is used. Being on the “growing” part of the graph means the company is moving some of its work to the public cloud and — contrary — “falling down” part of the red line means the company is moving OUT of the public cloud.
Our lovely website as well as startups are almost 100% in the clouds.
Then, my company, allegro (which has like 17 mln users and 1 mln daily transactions) is using its own data centers to host 800+ microservices.
Then, Pokemon GO, which you could think was dead, appeared to get 147 mln active users in May, 2018. Number of users vary, depending on many conditions (e.g. the weather), so the scalability possibilities granted by the cloud are really important in this case. Pokemon GO relies on NoSQL database from Google Cloud and is the biggest deployed Kubernetes cluster. Ever.
Then, there is Dropbox with its outstanding number of 500 mln users. By moving out from AWS S3 to its own tech, Dropbox saved $75 mln in 2 years.
And, finally, we have Netflix. It is well-known AWS customer with 152 mln users, but almost twice as many people who actually use it. Netflix has heavy calculations per user and it uses its own CDN (The best way to express it is that everything you see on Netflix up until the play button is on AWS, the actual video is delivered through our CDN).
Will Netflix become the next public cloud provider? We will see… At some point in time, Amazon went this way to make use of its servers, which were not used up until Black Fridays and other Boxing Days.
