Answering All Your Questions on Serverless Javascript

Bustle serves over 50 million unique readers per month through a “serverless” architecture based on AWS Lambda and Node.js. There are still servers, but Bustle doesn’t manage them. This shift has allowed Bustle to develop products faster and decreased the cost of its infrastructure.

Steve Faulkner, Director of Engineering at Bustle, recently held a Sideway conversation to answer questions around Bustle’s transition to serverless and and how it has worked out. He also discussed some of the tools and best practices the company is using, including its open source framework shep.

Image from Pixabay.

What are the security implications of this architecture? Do you have user accounts and if so, how does authentication work?

- Eran Hammer

Steve Faulkner: Security is definitely something we think about. At Bustle all of our staff and contributors have user accounts and authentication is done via a node.js port of our previous devise/rails setup. We use some custom lambda functions as well as API gateway custom authorizers to do authentication and authorization. AWS provides some other products that handle security as well. You can setup Lambdas to run inside VPC. As we made the transition to lambda this is what allowed us to talk to legacy infrastructure. We also use IAM in several places to handle things that are not in VPC such as dynamo. Ben Kehoe of iRobot just wrote a great piece on serverless security: https://serverless.zone/serverless-code-security-d592a3010699#.70f8b9ool

When would you NOT use serverless architecture?
- Eran Hammer

Steve Faulkner: AWS Lambda functions are stateless and have a 5 minute max execution time. This is no problem for our use case, but these constraints can be an issue. Serverless can also be expensive if you have very uniform predictable load. In Bustle’s case this is where serverless shines. We are able to scale with our spikey traffic quite easily.

I can imagine performance critical situations where serverless wouldn’t be ideal. We have done a lot of work to achieve good response times with API gateway and Lambda, but if you are trying to tune for ultra low response times it might now be ideal. I’ll say we are quite happy and regularly achieve < 200ms response times for our server side rendered javascript applications.

It seams that this architecture is very dependent on AWS, is possible to simulate it locally for development?
- Adilson Schmitt Jr

Steve Faulkner: That is where our tool shep comes in. It is able to run the functions locally. There are several other frameworks that will do the same thing. Running a whole set of interacting functions is still hard to do and one of the current pain points with serverless. The flip side is that provisioning new environments is not too difficult and you only pay for usage, so testing and deving in the cloud is not a bad option. Using tools like cloudformation or terraform it is easy to spin up these new environments for each developer.

Steve Faulkner: I will note that there is a difference between “serverless” and AWS Lambda. Right now they are very often used together, but that will be short lived. “Serverless” is the concept that you can rely entirely on services to build your application. Lambda was really the missing piece of that picture. Where does your business logic live in a world with only third party services? There is nothing about “serverless” that requires you to use AWS, but I think they just happen to be pretty far ahead out of every other provider in execution. Google and Azure are catching up quickly, which is great. Competition in this space is great for users.

Have you looked at Joyent’s Manta and how they do lambda-like distributed compute of tasks?
- Eran Hammer

Steve Faulkner: I’ve seen Manta, but I am not super familiar with it. It looks a little similar to some of the other open source lambda like project. IBM OpenWhisk is one: https://github.com/openwhisk/openwhisk Kubeless is another that uses kubernetes: https://github.com/skippbox/kubeless

Steve Faulkner: I always poke around these projects to see how they do things. But one of the biggest benefits for us is not having to worry about how to run or deploy these projects in production. Amazon does that for us :) Now obviously that comes with its own challenges but I think we are still better off. We’re pretty AWS centric at Bustle and 100% OK with that. I don’t really agree with most arguments about lock in. I struggle to find a business case for why lock in is a huge risk for our business. That said, others may feel differently. If I was in a business that was very concerned about lock in I would be constantly trying out all the open source lambda competitors.

Is there anything specific to node about this architecture? Would it be as easy to use other languages with the same setup?
- Eran Hammer

Steve Faulkner: Definitely possible to use other languages. You can use any language that will compile to an executable binary. Apex, which is written in Go, has great support for this https://github.com/apex/apex I have seen people using Go, Haskell, Rust. Our language of choice is node. We love having the same language on the frontend and the backend. Shep specifically only works with node and I don’t really see that changing

Steve Faulkner: Lambda has first class support for node, java, python and c#. Any other language will need a small shim that calls out to the binary executable. This does have some overhead. Depends a lot on your load and function size, but it can be 50–100ms on cold starts

Steve Faulkner: One thing on the horizon is that Rust has very early experimental support for WASM as a build target: https://twitter.com/BrendanEich/status/812698705753083904 I would LOVE to try and deploy that to Lambda. Not sure yet if it would be possible, but seems like it will happen at some point.

what is the differnce with Azure functions, and why AWS over Azure?
- nikosv

Steve Faulkner: I have only briefly played with Azure functions. This was a few months ago and you had to manually choose a scale for them to operate at. Maybe they called it a consumption plan? It reminded me a lot of heroku and dynos. Serverless isn’t only about simple scaling, but it is a pretty important part. That said, they worked ok. If I was on azure I would consider using them. The bigger reason for Bustle is we were already heavy AWS users. It made much more sense to stick with Lambda as it integrated nicely with all the other AWS products we are using.

Steve Faulkner: So much attention gets focused on the function part, but I think the more interesting one is actually the router — AWSAPI gateway. I don’t believe Azure offers this product yet. Google just recently got it. I have not tried it. Lambda itself existed for quite some time before we started using it. API gateway is really the key that unlocked an end to end serverless for our application. It allows you to route user calls to functions. It is super critical to running a user facing serverless app and I think Google and Microsoft are both pretty far behind with that product.

Besides the 5 minute execution time, were there any other road bumps you faced? Was there anything that could have been a potential deal breaker from utilizing Serverless?
- Peter Piekarczyk

Steve Faulkner: Plenty! When we started experimenting with serverless (about 1.5 years ago) it was very difficult to setup and deploy these apps. It is a big reason we wrote shep. Getting all the environments setup properly and having a fast repeatable deploy process was a huge pain. I’m not sure we had any deal breakers, but there were definitely moments of “ugh why are we doing this?” But things have gotten much better. API gateway is a much better product. They added many features and it is much easier to use now. Same with Lambda. I think you still need some kind of framework or deployment tool though. Manually doing everything with a few functions or APIs is not super fun.

Steve Faulkner: The payoff has been great. We are definitely happy we made this move and if I was starting from scratch today on an app I would probably default to the same setup.

Steve Faulkner: I will say there are still some issues. Logging and exceptions are still hard. Basically everything goes to cloudwatch. Which is a bit like a needle in a haystack. There are some third party services that are trying to fill the gap. Check out IOPipe https://www.iopipe.com/

what is the differnce between Lambda Function Integration and HTTP Endpoint Integration? which one do you use and why?
- nikosv

Steve Faulkner: HTTP endpoint integration lets you call through to any endpoint. You could do something like wrap twitter’s API with API Gateway if you wanted. Or wrap your own legacy APIs. We have done this in several cases. Lambda function integration can take a few forms, but we heavily prefer where lambda functions return objects of the form { statusCode: XXX, headers: {}, body: 'stuff' }. API Gateway can do much more complex things and you can write templates to transform data as it comes in or goes out. We really try not to do this anymore as they are hard to maintain and written in a template language (VTL) we are not huge fans of.

Does the lambda function executing get a new IP everytime?
- Michael L. Benin

Steve Faulkner: Not always. Under the hood AWS is using some kind of container system and it is possible for your functions to get reused. It depends on how often you are calling the function. This can bite you. I can think of one incident where we ran out of IPs in one of our VPCs because of too many lambda functions.

do you think that the process of starting out with Lambda and API gateway on AWS is cumbersome? I mean too much too setup, logs,permissions,etc but most of all figure out the relation between the components and how one affects the other?
- nikosv

Steve Faulkner: It is getting better. shep makes it super easy. You can get from nothing to a deployed public API with shep in just a few commands. But at scale it is still hard. We have hundreds of functions and keeping track of everything is still difficult. We're thinking of adding features to shep that help document and visualize some of the relationships you are talking about. Generally I think getting anything truly "production ready" is hard. Even architectures with really well established tooling and practices are hard. AWS and Serverless is the same, it is new and different and will feel unfamiliar to many people.

Why did you decide to create shep rather than using an existing framework such as Serverless/Apex/etc? Were they not mature enough when you started?
- Tyrone Tudehope

Steve Faulkner: When we started Serverless Framework was the only thing that existed. At the time it was still called “JAWS”. We decided not to use it mostly because of CloudFormation. We wanted a tool that used the APIs directly. CloudFormation is slow :/ If I was starting today I would be tempted to use Apex. It is nice and I think TJ has a great sense for what makes good tooling and developer experience. That said there are things we can do with shep that can’t be done with a multi language tool. So who knows, we might have built something like shep anyways. I do dream of the day where AWS services all play together a little nicer and we don’t need these things at all. But I’m not sure when that will happen :)

isn’t IAM athentication enough? why also go for tokens?
- nikosv

Steve Faulkner: Having all of our users in IAM would be amazing. Unfortunately due to various legacy systems it would be a hard thing for us to do. We’ve looked at is a few times and just can’t see a great path. Now that cognito user pools are more mature we should look again and see if it is possible.

regarding Lambdas, is there a notion of state, i.e in subsequent calls or whan one function calls another?
- nikosv

Steve Faulkner: Lambdas can call other lambdas and pass data. But functions themselves are stateless. Once the function is done executing anything done in that execution frame is gone. It is possible to hang onto some things, like database connections, between calls, but don’t rely on it. We use clients or code that always check for existing connections before talking to external services.

would you say that in essence Lambdas are a substitute for a RESTful API that lives on a VPS and calls your functions writen in your backend framework of choice?
- nikosv

Steve Faulkner: That is definitely one way to use them, and the primary one for us. But Lambda can be triggered by SNS, Kinesis, DynamoDB, S3, and other things on AWS. We use them to do indexing in elasticsearch, data migrations, analysis. Not just for responding to end user HTTP calls.

You mentioned earlier that keeping track of all these functions is still quite difficult. How do you manage them currently?
- Tyrone Tudehope

Steve Faulkner: We use shep and have multiple projects/repos. That setup is decently manageable for us. We kind of group them by "microservices" although I'm not sure that is best word to use. We also have been moving many of our APIs to a single graphQL endpoint which helps decrease the number of functions. Our front end applications are all served via one shep project

for debugging you rely only on the logs or are there other ways too?
- nikosv

Steve Faulkner: console.log and cloudwatch most of the time. I wish I had a better answer :/ We're actively experimenting sending our logs to various third party services so we can get a better handle on this. Kibana and ElasticSearch are also easy to setup inside AWS if that is your preference.

Steve Faulkner: https://aws.amazon.com/xray/ is very exciting for helping solve this problem. Not available for Lambda yet, but hoping soon.

When deploying changes, is there any downtime involved? What kind of strategy do you use for deployments?
- Tyrone Tudehope

Steve Faulkner: As best we can tell there is no downtime and the change to new code happens nearly instantly. Obviously it can’t be actually instant, but we haven’t noticed any issues as a result of deploys. We deploy many times a day, sometimes from CI and sometimes manually. One of the reasons for shep was that we wanted to make deployments as simple as shep deploy production. It is very important to me that anyone on the team can deploy any of our applications quickly.

In shep’s readme on github you mention “excessive amount of manual work such as zipping files”.is that an alternative way of creating your Lambdas?
- nikosv

Steve Faulkner: Simple lambdas can be written directly in the AWS web UI. But for anything with even minimal complexity you’ll need to zip up the file along with any dependencies and upload it to AWS. This can also happen via S3. Shep handles all the zipping for you.

Steve Faulkner: Were coming up on 2 hours, so I am going to have to step away. Thank you to everyone who participated! This was great.

Steve Faulkner: I am always happy to answer more questions. Feel free to email me: steve at bustle dot com.

great chat and shep looks great.do you respond through twitter?
- nikosv

Steve Faulkner: Twitter works too! This is me https://twitter.com/southpolesteve

Interested in hearing more? Steve presented on this topic at Node.js Interactive North America — watch the video here.