Baking Static Sites with Python and Jinja

Matt Harzewski
May 30 · 6 min read

I have long been a proponent of static site generation, even before the (relatively) recent wave of tools like Jekyll, Hugo and Gatsby. In the pre-WordPress blogging world, having an engine such as Movable Type or the original Blogger.com churn through your content and produce plain old HTML files was the norm. It made sense; after all, the basic principle of a content-driven web site is an author pens an article once in awhile, and users do little beyond reading that content. Why waste time building the page on every request, slowing everything down, when your web server can throw back thousands of static files per second?

The advantages of generating blogs and other sites that change infrequently (marketing sites, documentation, etc.) are well known and often talked about. You can shed complexity and hosting expenses, and it becomes easier to handle backups or archival. There is a multitude of tools available to solve this sort of problem.

However…what about sites that aren’t a collection of content written by humans? Perhaps you have some sort of semi-dynamic content that is collected and presented by an algorithm without human intervention. However, the content doesn’t change very often, so it theoretically would be advantageous to write the page once and let the web server do the heavy lifting. One example might be an aggregator like Alltop or Dave Winer’s “River of News” concept. You only check feeds every so often, and then could generate out a static web page. Or maybe you want to write a bot that watches messages in a Slack or Discord channel and periodically updates a public log page.

It’s actually fairly easy to roll your own purpose-built static site generator with a little bit of file I/O and a templating library. I will be using Jinja2, the template engine used by the popular Flask framework for Python. (I chose Jinja for that reason in particular, as I’ve found it to be pleasant to use, and the familiarity that Flask users will have is a plus.)

As an example, we’re going to build a quick and simple feed aggregator. First, create a Python virtual environment and install Jinja2 and the Universal Feed Parser.

pip3 install jinja2
pip3 install feedparser

You’ll want a project structure similar to the following screenshot, with a main Python file, a directory to hold the template and CSS/image files, as well as a public directory for the site to be generated into.

Image for post
Image for post
Project directory structure

Starting off, let’s import the libraries we’ll be using and define a list of the feed URLs that the aggregator will fetch. There really aren’t many dependencies. We need Python’s internal os and shutil libraries in order to copy files around, and a handful of classes from Jinja. Beyond that, this example uses the Universal Feed Parser so we have something to do, but of course your hypothetical application may do something completely different.

import os, shutil
import feedparser
from jinja2 import Template, Environment, FileSystemLoader
URLS = (
'http://feeds.arstechnica.com/arstechnica/index/',
'https://www.reddit.com/r/programming/.rss',
'https://www.theatlantic.com/feed/all/',
'https://css-tricks.com/feed/',
'https://www.theverge.com/rss/index.xml',
'https://lobste.rs/rss'
)

Now we want to define a class for our static site builder and fetch the feeds. Fairly trivial stuff with the feedparser library. When the file is run, we should have two key components: a list of parsed feeds in self.feeds and a Jinja Environment object in self.env.

Environment is a Jinja construct that holds settings used by the template engine, such as where to look for files, whether to autoescape HTML by default, or even the syntax used to delineate variables or content blocks in the template. We only really need to worry about telling Jinja where to find our template files, which are (creatively) in the “template” directory.

class SiteGenerator(object):
def __init__(self):
self.feeds = []
self.env = Environment(loader=FileSystemLoader('template'))
self.fetch_feeds()
def fetch_feeds(self):
""" Request and parse all of the feeds"""
for url in URLS:
print(f"Fetching {url}")
self.feeds.append(feedparser.parse(url))

if __name__ == "__main__":
SiteGenerator()

As we progress, our goal is to expand __init__ with method calls to carry out the following tasks:

def __init__(self):
self.feeds = []
self.env = Environment(loader=FileSystemLoader('template'))
self.fetch_feeds()
self.empty_public()
self.copy_static()
self.render_page()

With fetch_feeds() taken care of, we can move on to the two methods that prep the output directory and copy the stylesheet/images/etc. over. They’re fairly similar in nature, just basic Python file I/O.

To ensure we have a pristine public/ directory to write our files into, the simplest approach is to delete it and recreate it. Then we can use the recursive file copy function in shutil to move all files in the template/static directory to public.

def empty_public(self):
""" Ensure the public directory is empty before generating. """
try:
shutil.rmtree('./public')
os.mkdir('./public')
except:
print("Error cleaning up old files.")
def copy_static(self):
""" Copy static assets to the public directory """
try:
shutil.copytree('template/static', 'public/static')
except:
print("Error copying static files.")

Finally, we can render our template out to a static HTML file.

def render_page(self):
print("Rendering page to static file.")
template = self.env.get_template('_layout.html')
with open('public/index.html', 'w+') as file:
html = template.render(
title = "Spiffy Feeds",
feeds = self.feeds
)
file.write(html)

Breaking it down, we get a Jinja Template object by its filename and then open our output file for writing. Where the magic happens is when template.render(...) is called. This method returns the final HTML, which we can write to the file, and accepts as many parameters as we desire to expose to the template.

In this example, the two parameters being passed are the page title (to illustrate variable substitution in the template) and the list of parsed feeds. This could be expanded as much as necessary.

The parameters that are passed to template.render(...) all become available to us when editing the template file. So the title parameter can be accessed by simply putting something akin to<h1>{{ title }}</h1> in the template. Since we’re passing a list of full feed objects from the feedparser library, we can get at all of those objects’ methods in the template.

The following is an example _layout.html file:

<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>{{ title }}</title>
<link rel="stylesheet" href="static/style.css">
</head>
<body>
<div class="container">
{% for feed in feeds %}
<div class="feed">
<h3>{{ feed.channel.title }}</h3>
<ul>
{% for entry in feed.entries[0:4] %}
<li><a href="{{ entry.link }}">{{ entry.title }}</a></li>
{% endfor %}
</ul>
</div>
{% endfor %}
</div></body>
</html>

One feature that I particularly like about Jinja is how it uses Python’s slice syntax, making it easy to cleanly limit the number of headlines displayed from the template side of things. Instead of just having{% for entry in feed.entries %} to iterate the entries in the feed object, all you have to do is add [0:4] to only display items 0–4. Jinja has many control structures, expressions and filters available, and is very extensible as well.

Notably, any file can be a Jinja template, not just HTML. If it’s text, you can template it.

Image for post
Image for post
A static web page generated by the script.

Not bad. Now with a little bit of quick and dirty CSS, it can be shiny and stand out.

body {
background-color: #DBDBDB;
font-family: sans-serif;
}
.container {
display: flex;
flex-flow: row wrap;
justify-content: space-around;
width: 70%;
margin: 5em auto;
}
.feed {
width: 300px;
background-color: #FFFFFF;
border: 1px solid #989898;
border-top: 10px solid #E75131;
margin: 1em;
}
.feed h3 {
border-bottom: 1px solid #989898;
margin: 0;
padding: 1em;
font-size: 1.3em;
}
.feed ul {
padding: 0 1em;
list-style: none;
}
.feed li {
margin: 1em 0;
border-bottom: 1px dashed #989898;
padding-bottom: 1em;
}
.feed li a {
color: #333333;
}
Image for post
Image for post
The same web page, but with some fancy styling added in the CSS file.

That should give you an idea of how simple it is to design an application around static site generation, rather than leaning on a dynamic web framework. In many cases, it’s simpler to deploy and can save you a lot of idle RAM wastage. Something this simple could easily run on a cron job, instead of tying up a precious megabytes of RAM even at idle, and is ideal for small hobby projects.

You can find the full source as a GitHub gist here.

The Startup

Medium's largest active publication, followed by +733K people. Follow to join our community.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store