Mobile Website with AWS Lambda (BaaS)
Mobility today is a priority with Google’s new PageRank rules, therefore a new project came to my demand and I started working on this new project. Every time I start a project I always like to evaluate what’s new in the market and what’s best for my customers, I’ve been working with some demands in AWS Lambda in test drive mode and have always seen a lot of potential in this technology, on this project I have put a 100% drive on it, pushing costs down I saw a potential of using S3 only with Lambda, this means my customer is only going to pay for what their users see and use.
Meaning my customer will not pay for server being up 24/7 and will have an auto-scaling benefit if the site is really popular. This means driving cost effective solution with high output performance and happy customer in short and long-term relationship with the product.
Objective
I want to help others who will or who’re using AWS Lambda for their projects, other technologies involved: Cognito, CloudFront, Lambda, S3, CodeCommit.
Why? Thanks to the open technologies that are out today in the market, I’ve completly changed the way I used to work, before with Java, EJB, web.xml, Hibernate, Spring, JSF, C++, PHP, etc. Now with NodeJS and NPM. I’m thankful to Java as well, due to what I did with Java I was able to rapidly implement great practices in NodeJS, specially with Gulp which is now my swiss knife for NodeJS.
Thankful to all who worked or works on NPM projects, people behind NodeJS, all the community within Github, and all behind Github! So many people releasing great things, companies taking so long to adopt great technologies.
AWS
I didn’t really see Google or Azure as more compeling options, I’m also used to AWS and it’s very stable so I see this choice of being more strategic, if we look into prices it’s probably not the cheapest “today”, but it has the biggest and best toolset so it was an important choice to choose Amazon.
Support
Another clear winner for Amazon is a great support team they have, they try to help you in a fast manner, if you’re a paying or a not paying customer. This is very important when you’ve hard doubts about what you need in the toolset that’s provided by AWS JS API GitHub and AWS Lambda Forum, big thanks for both teams, they’re great! It’s great because it helps to keep you within the AWS ecosystem. I think this should always be one of their main strategies as it helps all the developers to keep on pushing further with AWS based projects.
Know your Cloud provider’s toolset
Getting to know more about the toolset from the cloud provider in every new project is always good, because I’ve learned that serving from CDN (Cloudfront in AWS) is cheaper than S3 directly on this one, so I rather serve it faster and cheaper for my customer through CDN. I like to revisit old topics (tools you think you know) because we can always learn something new and maybe the company upgraded the functionality or pricing so it’s always a good idea to revisit and rethink the solution.
Requirements
Do a mobile website, use my current database data, no overhead for my administration, I don’t want to keep supporting this solution, give me something fast, without support, easy hand-over for my IT.
Programming Language
Since the decision was to move on with Lambda and a full static front-end, the strategy became easy: Javascript on front-end and back-end.
IDE
I’ve tested most out in the market, my requirements are: Has to be open source and free, the good options I see are LightTable and Atom, after a while I picked Atom for obvious reasons, there’re so many plugins coming up it will be hard to not have Atom in the future, it’s light, fast and reliable with a huge amount of plugins in such a short lifetime.
Front-end Framework
Looking into what’s new around is always good, now there’re a lot of new frameworks with the static concept. Bootstrap became a natural choice due to JQuery and I really like the maturity and capabilities of JQuery and Boostrap is also mature.
A lot of people complain about the boot size of bootstrap, I got around that with the build process in which I remove the CSS I’m not using. There’s a problem with dynamic content which is not on the page and I got around this as inputting as exceptions to not exclude for specific classes.
Security
My biggest concern when working with client information is all about security, learning that Amazon has strict security approvals for governmental services has led me to give them security credit regarding handling customer information.
Flexibility in security credentials change, how will the IT staff be able to do fast password change in case needed. How will Lambda, security credentials work with non authenticated situation. The answer that came to light was Cognito.
SQL Injections, even dough the function call is safe, someone can still call it from a “console” from a Chrome Browser and try to do some SQL Injections into your function. Make sure you are secure against this, it can harm your database, your data. Always parse your data, delete and insert must be totally isolated from your regular selects user. This is valid not only for NodeJS or Lambda, but for any other programming language in the backend having interface with database.
Build & Deploy
Even working with Javascript only, it’s important to think about performance, code build and best practices, I worked with grunt before but it’s path length being too big sometime in Windows that got me scared a bit a lot of time, so I decided to test out Gulp for this project, this is the best piece I found for the project. Besides having smaller packages for the plugins, it’s easier to work with and it’s just a great tool.
Gulp is the most important part of the project, it’s how I build the AWS Lambda packages and it’s how I build the files, even with the HTML I do templating in Gulp. It’s important to have templating for the HTML in order not to repeat code everywhere, it’s great because I don’t have to have another framework like ReactJS just for templating, I like the ReactJS idea, but since it’s a mobile-website it’s important to be small, fast, less is better, therefore JQuery only.
Build
I use Gulp for building optimized HTML, JS and CSS. Using lazy loading best practices for Images, JS and Gzipping the files.
Deploy
Deploy for S3, the Gzipped and Cache header is all done with Gulp process. I have built a full deploy process and a partial deploy process, to consume less time and faster fix/deploy.
Architecture
Communications with the Mobile Website
PageSpeed
Pagespeed is something to always take into account, it changed how I worked on the project, everything on the site became action based. Lazy loading images, JS. It really became a lazy loading experience, remembered me of the AJAX, before AJAX, javascript with the get. This is a event-driven architecture model.
The fun part is that you’re controlling the state of the experience and responses, Lambda is just processing so the control of speed and performance is all based on the work in the frontend (Javascript, HTML and CSS).
Image Optimization Tools
Performance
CloudFront
It’s important to note that if you’re using S3, you’ll need to use CloudFront, it’s not an option it’s a requirement. You’re not able to control browser caching from S3, you’re able to control CloudFront’s caching and invalidations.If you get cache for a long time while testing, invalidate your cache through the “invalidate” option in your cloudfront configuration.
Another thing is that if you’re not using CloudFront, you’ll get less performance from AWS Lambda from the AWS Javascript SDK, therefore you’ll spend more money on processing where the problem is file location / internet connection / geo-location and not your Lambda function. Save money on Lambda and S3 file serving, get more control over your frontend and get more performance, Cloudfront is a must.
Lambda
if your AWS Lambda function is accessing external database it’s important to set the timeout for 3x the normal processing time it’s taking. If your query is taking over 3 seconds, change your query / process, create more filters within your query and analyse your indexes in order to verify if it’s a database performance issue or if your query is just iterating through too many rows.
Bring only the rows you’re going to utilize in that state specifically, if you’re thinking about bringing more rows for “further pages” in a “pagination mode” I suggest to bring the minimum you can, that’s how Lambda is supposed to work, so bring the least and focus on performance always, the faster your query is the better value your users will have with a fast website or mobile-site.
In my case where I was accessing external DB, I noticed a big difference going from 128MB to 640MB of memory, if your function is taking too long I suggest you analyse this before going public, getting a function from 7 to 3 seconds can get you more usability and engagement, so test and see if in your case you get the performance to see if it’s worth it or not. I also tried upgrading from 640MB to other higher values, but the timing stayed the same so I didn’t opt to move to those higher memory values. I don’t think there’s a exact math due to the function you’re creating can be from an image processing to a database connection, so each case is very different and specific, you should analyse every case in a separate manner, don’t keep testing the same type of case “database connection” because you know that already.
So if you’re executing a function within AWS Javascript SDK + AWS Lambda here’s the sum of the time consumption for the user:
Frontend
- The objective is to always minimize the render blocking portion of the site in order to maximize loading and browser parsing time. CSS is a big portion of this optimization, that’s why Gulp is very important.
- On item 2 is where I loaded async the other JS files, that’s where I loaded AWS SDK JS file locally from the S3 bucket (better speed).
- After the AWS SDK JS is loaded I need to Login to Cognito in order to identify the non-auth user to use the Lambda function safely.
- Then after we’ve authenticated with non-auth I call up the lambda function with the parameters, all these steps with Cloudfront is one thing, directly from S3 is a another, Cloudfront makes a big difference here.
Backend
- I have noted that just the require libraries within AWS Lambda can take from 1 to 1.3 seconds, it’s important to note this when your function takes 3–4 seconds, you can optimize this portion loading less libraries, smaller libraries. Smaller index.js file, minimize the rendering path for Lambda
- Process the index.js file, make a faster exit, shorten functionalities into smaller pieces. Think of a Lambda function as a WS function, don’t acumulate too many functionalities into one, cut down the functionalities.
- Exit with the result to context.succeed function.
Thread Safe but not Exception Safe
Even dough your function is thread safe it doesn’t mean you’ll never have problems, that’s a misconception one might have about Lambda. There’re two scenarios where you’ll always have to work against which are:
- Timeout of your functions
- Error or exception handling
There’s no such thing as a perfect error prone function, so you’ll have to work against the expections which you’re not expecting.
Doing a test scenario is good to validate if your function is still valid, but it’s not the expection inspector, be wise to test all possible scenarios so you catch the error before your users.
Debug/Logs
All the logging is done in CloudWatch, I think this is so awesome, the dashboard. Sometimes I don’t get the logging dough if I’m not looking into the logs, that gets me a little upset but it’s reasonable since the service has been around a little while, it’s ok for them to have their “little” expections scenarios as well, it’s not affecting my functions, everytime I’m looking I alwayse get the Logs, it’s very important por debugging.
Timeout
Timeout is really the masterpiece that AWS putted on Lambda, it serves for limiting processing, serves for you to limit costs per processing, serves for AWS to keep your functions running in a healthy manner even if there’s an exception. It’s the best thing they thinked about and it’s truly amazing how it works, you might initially not give it a lot of value, but in the long run it’s the most important piece of configuration together with memory. It’s like if you’re in Formula1 and you want to choose which tire the driver should use, the driver is your wallet, the tire is the timeout, the track is the memory you configured.
Minimize Timeout Risks
Minimize the timeout risks with CloudFront, this is very important. You might think your function is not performing but the problem is really on the otherside, on the delivery or network latency. It’s the only way to guarantee the performance on the customer’s side independent of geographic location or the route that the internet service provider is taking.
External Database
This was certainly my biggest problem, it’s located in a provider in which the connection speed doesn’t seem great, the queries vary from 1.5 seconds to 5 seconds, being the same query. Maximizing connection speed on my end was vital, so the lack of performance stays on the otherside of the fence.
Following a closure concept I solved this matter, I also used closure with callbacks to ensure speed and asynch calls within my front-end code.
Another important aspect is to maintain DB logic within SQL. Others might go back to the programming language to filter out some other logic, this usually is very time consuming, having experience with this with other technologies I know that following this best practice was also vital to the success of the DB calls being within seconds.
I cannot really tell my client how bad his DB performance both in terms of network speed and in terms of performance but the mobile site now is faster than the website which is within the External provider. Even dough there’s a geographic location challenge (DB in Brazil and AWS in Virginia) and a disavantage of not being within the same local network, the mobile is faster due to other great pieces of technology that made it a masterpiece.
Upgradability and Zero Maintanence
The fun part about having all this work is that you’re future proof. No need to look into security patches, fixes, plugins, you’re not managing the server(s), AWS is, so the security in Operating System level is their responsability. That’s very important.
Another important aspect is that there’s no need for maintanence, so you’ll never need to have a support 24 hrs in case your server goes down or slows down, you’re under a PaaS, they take care of that for you.
The Result
PageSpeed Insights
My objective is always to keep the 100% on Google PageSpeed Insights, if we reach the 100% it’s good, if not we have to work on it :). Thanks to the hardwork and dedication I was able to get this result, I would like to thank Google for all the knowledge they provide to us for free about how to get there. It’s amazing how these big companies provide so much material for free because they really want to provide value to the internet and they’re not only here for the money, that’s very important if we truly want a better place for the web and for our internet users. A better mobile web is a very important driver, thanks to this driver from Google that I got this opportunity as well :).
The Mobile Website
I didn’t not get formal approval yet from my customer in order to reference his url here, as soon as I get it will make it public.
If you want to checkout my blog feel free to visit me at: http://ramonck.github.io
