Chasing the infinite scalability dream…
I am a developer & I have an idea for a web app. What should I care about? Right, The idea!
Once I am sure about the idea, I have a set of problems to solve. I need to slog my ass off to make sure the web app:
- Delivers functionality
- Is easy & intuitive to use
- Isn’t buggy
- Doesn’t have performance issues
Then there are other things I need to take care of like:
- Share it among family & friends
- Share it in the right forums
- Generate traction & belief in people that, we can deliver
- Make it sustainable
Problem:
There is nothing more important than going into the market well prepared in terms of:
- Handling fluctuating network traffic — you go viral on hackernews (like we did)! / an eminent guy tweets about your app / get featured in techcrunch and suddenly a herd is coming your way! worst thing it won’t be there tomorrow. So what do you do?
- Data security — you cannot afford a world class system admin! You don’t want to get up in the morning and get to know that your server got compromised!
- Easy scale out — because you don’t know at what pace your business is going to grow. The next thing you know is you need a server cluster & a database cluster & a load balancer! wait! what, really?
- Well written code that is maintainable — a new guy joins your team and you don’t know how to explain what you were smoking when you wrote that bit of code optimization!
Now I am tired! My competitors are four times the strength (in terms of manpower, money, customers etc.) & growing at double the pace and here I’m thinking of scalability, security and maintainability, but all this while I don’t even know if the app will survive to see the light of the day!
I have enough problems already, I don’t want to buy a server or a VM on the cloud, install linux, apache, postgresql and all that is required for managing & monitoring a server on it while I code in Java/RoR/PHP/Python or hire a system admin to manage all this for me!
Solution:
You can now use Google’s platform — “Google App Engine” (GAE), finely tuned for speed and massive scaling, instead of trying to build it themselves.
There is no machine setup or administration as GAE takes care of all the logistics!
Features: auto-scaling, NoSQL database, global memcache, task queues (including scheduled tasks), zero downtime deployments, awesome development experience
Handle fluctuating traffic with the auto-scaling feature of GAE is awesome. It will create server instances as required and shut them down when the herd is gone!
Need a distributed database? Google provides a scalable NoSQL datastore. You can map models like Hibernate/JPA (more about this in coming posts!)
Need MemCache? Google has a global Memcache API.
These 2 things by themselves hide a lot of problems from the developer & allows the developer to concentrate on developing functionality. A carefully designed app will scale auto-magically! (more about this later!)
Security is gravely important. Developers (typically) would not be interested in not letting other applications/users get any kind of access to their application code or data. To ensure this, all App Engine applications run in a restricted sandbox.
The development-to-release cycle is minimized because high-level services, that developers would normally have to build, are already available via an App Engine API.
Zero downtime deployments is the icing on the cake :)
Wait. If its sooo goood why isn’t the world already there?
Gotchas:
You cannot:
- Open a local file for writing
- Open a socket connection (which they are working on currently)
- Make operating system calls
- Cannot have custom database running
- Cannot execute commands on the server (and hence cannot install / use custom software)
Resources:
