Using AWS Lambda in Meteor
Ever thought that your Meteor app is not able to take processing load?
Meteor is one of the fastest growing JavaScript framework, written in Node.js, is very much suited for fast app development. But there are developers who have gone through cases, where Meteor failed to meet their performance expectation when doing some heavy processing.
Well, I am one of those developers who went through this and today I’m gonna share that use-case and how I solved it (well, as the name of my blog might suggest, I used AWS Lambda).
USE CASE:
I was working on a module which would generate a set amount of JSON objects, where a user can pass fields of Object and get random values. For these random fields, I was using an awesome Javascript library, ChanceJS.
I was aiming to create 1 Million objects under few minutes but when using simple Meteor app, time was going to 30–40 minutes for simple objects with 4–5 fields. That was not something I wanted.
So, someone suggested me to use AWS Lambda for 2 reasons:
- Auto-scaling and
- configurable RAM (I can set it from 128 to 1536 MB to be exact)
Before I tell you how I used Lambda, here is what it is exactly:
- A computer service where I can deploy my code and don’t have to think about server,
- I will only have to pay for time that my Lambda function runs, now for the time it was up (pay-per-user),
- I can write code in 4 languages: C#, Python, Java and of course Node.js (I will be covering the only node.js).
Basically, you can write a micro-service for your project by just uploading your code in a specified format.
Setting up Lambda
I said “specified format”, because by default lambda runs index.js in your function directory and runs exports.handler function in it.
For bundling the app into a zip file, I would suggest you use node-lambda package as it is great for testing your lambda function in the local environment and will also help you in setting a default structure of the app.
Note: if the size of this zip exceeds 10 MB then you have to upload it to S3 bucket and then give its URL to the lambda dashboard.
Once you have your lambda function ready and have tested it on the console, you can use it by either of the two following methods:
- Build an API gateway using AWS CloudFront
- Using AWS SDK for JavaScript
There are other methods but I have tried only these so far, so I’m not going to discuss the other ways.
First of all, both of these solved it. By using a bigger amount of RAM for Lambda function, my app’s processing power increased hugely. I was able to generate 1M objects under 10 mins (tested from console).
But there was an issue when I was using API gateway. But let’s see first how to implement both of these.
CLOUDFRONT API GATEWAY
Cloudfront can expose your function to a gateway and can create a REST API using very simple steps. AWS has already covered it nicely, so I won’t go in depth of it, you can just follow this guide to get the REST API for your Lambda function and can test it in Postman.
AWS SDK
For this, use aws-sdk package. Once set up, you can call any official AWS Lambda-class method. To run your function, just call invoke the method of SDK. You can find all these methods here.
Now, I first tried CloudFront and tested it for 10–20 K objects and response was overwhelming. But when tried for 200–500K objects (as I was aiming for at least 1M), I kept getting the timeout error. After doing some research (googling) online, I found out that CloudFront let its APIs run for only 5 mins. And for generating my data, my function was taking more than 5 mins. There is no way to configure this 5-minute constraint.
After this roadblock, I tried using above-mentioned SDK package, and as Meteor (< 1.3) allows devs to use npm packages just like in typical node app, I realised that to use any AWS feature (BeanStalk, s3, Lambda, etc) this package is more than enough. As it gives access to every method of their respective class.
Once I integrated it in my Meteor app and invoked the function, it was giving me 1M objects in approx. 10 minutes.
That day I finally got it, that I don’t need to do all the heavy lifting in Meteor as I can isolate my node.js code as modules and use them as microservices using AWS Lambda.
I hope you found this post useful and please do let me know in case of any question or query. If you like this post, you can clap for it or follow me on Medium for my latest posts or on Twitter.
