A mostly static site with S3, Lambda, Jekyll, and Flask

TLDR

Creating a simple mostly static website on AWS is surprisingly complicated, but great once you get it all knitted together. I’ve created a bootstrap repo that could get you up and running in an hour or two.

Here is an example website

Here is a Github repo to get you started

Background

I needed to create a website for the new product I’m building. I tried a static site before, but it turns out that you almost always need at least a little back-end functionality — even if it is just for processing a web form. So static is out. I’ll use my goto — Flask.

But the Flask installation on my everything server is python 2.7. And it turns out the Apache mod_wsgi can only run ONE python distribution, and I don’t want to do any more 2.7 work. But to get 3.6 to work, I’d have to install another server (like gunicorn) and then figure out how to link that up to my apache. Or install another linux server, and deal with either manually installing stuff, or go through the hassle of updating all my build scripts (ansible). From experience, that would take a while.

The alternative was something I wanted to try: a static site on S3, with a backend server on Lambda. I figured this would also be a lot of work, but not much more than my other options. I decided to go for it.

So now here it is: http://FileSimple.info

GitHub Repo

Here is a link to a “Hello, World” repo that could help you get started. Note that it is neither a push-button, best practices scaffolding maker like Yeoman nor a super clean and minimal “Hello, World.” Instead this is a slightly cleaned up version of my own simple site, with all the bad or idiosyncratic decisions I made along the way embedded.

Architecture

Super simple in requirements, but LOTS of parts in implementation:

Static pages

• Built in Jekyll
• Served on S3

Backend

• Flask (3.6) (using Boto3)
• Served on Lambda
• Behind API Gateway

Backend functionality

• Save feedback in DynamoDB
• Send email to administrator (me) via SES

Deployment

• zappa.io
• Plus some scripts

Other AWS stuff

• IAM — identity management
• CloudWatch — logs
• CloudFormation — deployment scripts created by Zappa
• Config — track config changes
• AWS Budgets
• Alerts with SNS

That’s a lot for a tiny site! But my hope is that given what I learned, doing it next time will be MUCH easier. And now I don’t have to spin up new servers (and worry about security). Also, once you have the basic infrastructure working front-to-back, implementing a new service (like email, messaging, database, etc.) is pretty easy — easier than getting it running on your own linux server.

Disclaimer

I am a self-taught programmer. The amount I know is massively outweighed by the amount I don’t know. I’m sure my ignorance comes shining through. But my goal was to get my first AWS site up without taking too much time. The next time I’ll dig deeper and start filling in my ignorance.

The Hardest Parts

• Permissions — ugh! I tried to manage the permissions myself, which was difficult. In my Hello, Lambda repo I let Zappa manage it, which is a lot easier (use the `extra_permissions` setting in zappa_settings.json). Unfortunately Zappa is quite permissive with its default permissions, which is a bit troubling.
• Logging — something is wrong with logging (I think) and it doesn’t always log properly, which makes fixing things a pain.
• CORS — what a *** hassle!
• API Gateway — works, but I don’t really get exactly what’s going on, so hard to fix things.
• Zappa — really helpful in getting it started, but it is opaque. I wish it were more transparent.

Development Notes

• Python — I like python so I use it wherever possible. If you prefer JS, your stack will differ.
• Static site generator — I used Jekyll because I already know it. Another good choice is probably Hugo, based on its popularity and description. I tried Pelican, since it is in python, but it really didn’t work well. Should you even use a SSG? I think yes — the templating and dev server are worth the complication.
• Logging / Cloudwatch — Invest time early into getting your logs working and tested. DO NOT DELETE any. I think there is a bug whereby if you delete a log for one of the services (I forget which one), it will not create a new one. I had to redeploy several times to figure out what was going on.
• Zappa.io — This was incredibly helpful in getting started. But it also hides what is going on. In the background it creates a CloudFormation template that is used to deploy the site. I would recommend using Zappa.io to start, and then taking ownership of the CloudFormation template and making changes through it. You’ll have a much better idea what is going on.
• Stages — right now, I only have one API stage and deployed static pages. Next time I’ll implement a dev and production stage, backend and frontend.

Helpful Feature #1: Local & Remote Interactive Testing

After a bunch of iterations I created an interactive test page that works locally or remotely. It was very helpful in debugging. Example

Convenience Features

• Static pages hosted locally with LiveReload (`jekyll serve`) or on S3
• API endpoint either localhost (so you can step through with your favorite python debugger) or on AWS API Gateway & Lambda
• Auto-guesses default endpoint and turns background yellow when locally served — just because I’m lazy!

Helpful Feature #2: Rate Limiter

One thing that concerns me about AWS is it’s unbounded liability. If someone hacks or slams my Digital Ocean droplet, I have some downtime, but my cost is still $10/month. If someone attacks my AWS installation, who knows how much that could cost me?

I’m no expert, but start with:

1. Be CAREFUL with your AWS credentials. Make sure they are NEVER in source control. (Add `.aws/credentials` to your .gitignore)
2. Use IAMS to limit the access your service uses
3. Set up an AWS budget and alert

Still, I felt nervous. In order to make progress with my projects, I accept a lot of ignorance. That’s usually ok, but not if it could wind me up with a $20,000 AWS bill. I’m assuming that’s pretty unlikely, but I wanted to make that discomfort go away.

My top-of-mind security risk is that someone would maliciously slam my form API, just to hurt me. So I created a little rate limiter decorator to protect my DynamoDB and SES calls.

The code is here: limiter.py

Use it this way:

Conclusion

I’m glad I did it. I learned a lot and I can see this as a good way forward.

I’m eager to try site #2!