Introduction
1. In this post you will learn
2. Why we need a template system
3. Basic syntax of Jinja Template (Python Jinja2)
4. Generate a simple HTML report with Jinja and Google Chart (Google Chart)
Why we need a template system
While string formatting in Python is pretty straight forward `e.g. print(“Hello {name}”)`, it could be quite troublesome when we are writing some long messages with many variables, or sometimes we want to use Python function like `if` or `for` loop in the paragraph.
A template system in Python, or any programming language, is a tool that allows you to separate the presentation logic (Template) from business logic (Code) in your applications. It provides a way to define and generate content, such as HTML, XML, or plain text, by combining static template files with dynamic data. In this post, we will demonstrate how to generate some simple HTML report in the following section.
Basic Syntax of Jinja Template
1. Render your first jinja template
It looks very similar to string formatting at the first glance. We will first **define** a template `t = Template(…)` with variable `{{ var_name }}` , and then **render** it with the python variables `t.render(var_name=”sth”)`.
```python
from jinja2 import Template
t = Template(‘Hello, {{ name }}!’)
print(t.render(name=’John Doe’))
# Output:
“Hello, John Doe!”
```Basic syntax
The syntax is almost the same as python with a little bit of syntactic sugar, we can use some conditional block via `{% %}`, and reference to the variable with `{{ }}`. And another useful thing to note is, we can add a dash `-` to the operator, so Jinja knows we dont want an additional line break, e.g. `{%- -%}` (No line break) vs `{% %}` (With line break).
#### a) Variables
Syntax: `{{ foo }}`, `{{ foo.bar }}`, `{{ foo[“bar”] }}`
You can use a `dot (.)` to access attributes of a variable in addition to the standard Python `__getitem__` “subscript” syntax (`[]`).
from jinja2 import Template
baz = "a"
foo = {}
foo["bar"] = "b"
t = Template("""
{{ baz }}
{{ foo.bar }}
{{ foo["bar"]}}
""")
print(t.render(
foo=foo,
baz=baz,
))
# Output
a
b
bb) For loop
Syntax: `{% for i in some_list %} {{ i }} {% endfor %}` with line break and `{%- for i in some_list -%} {{ i }} {% endfor %}` without line break
from jinja2 import Template
some_students = ["john", "terry", "ken"]
t = Template("""
{%- for student in students -%}
<li> {{ student }} </li>
{% endfor %}
""")
print(t.render(students=some_students))
# output
<li> john </li>
<li> terry </li>
<li> ken </li>c) If.. elif.. else
Syntax: `{% if x = “a” %} {% elif x = “b” %} {% else %} {% endif %}`
from jinja2 import Template
x = 40
# Additional spacing is added for readability
t = Template("""
{% if x >= 0 and x < 30 %}
x is larger than 0
{% elif x >= 30 %}
x is larger than 30
{% else %}
x is smaller than 30
{% endif %}
"""
# output
x is larger than 30d) Comments
Syntax: `{# #}` (with new line) or `{#- -#}` (without new line)
from jinja2 import Template
t = Template("""
{#- This is a comment. And it is not displayed in the output. -#}
Hello, {{ name }}!
""")
print(t.render(name='John Doe'))
# Output
Hello, John Doe!e) Read template from a file
For better file management, we usually put the files into a `./templates` folder. It helps to separate the presentation layer (template) and logical layer (code). The file structure would be something like this.
.
├── main.py
└── templates
└── simple_report.htmlFirst we put the template file under `./templates/simple<sub>report.html`
{#- <! — templates/simple_report.html → -#}
<!DOCTYPE html>
<html>
<body>
<h1>Hi {{ name }}!</h1>
</body>
</html>
Then we can read the templates with `open(“some_file.html”, “r”) as f: … Template(f.read())` in **main.py**
# May mess up Jinja Template Hierarchy tho. In our simple case, it wont matter much
from jinja2 import Template
name = "John"
with open("templates/simple_report.html", "r") as f:
template = Template(f.read())
rendered = template.render(name=name)
print(rendered)Simple Reporting Templates with Google Chart
After the long syntax introduction, we will jump straight to the use cases.
Sometimes when the buisness users are asking for some advance/deep-dive adhoc reports, there are not much report choices for a quick study, you can use
- Excel file
- Commercial products (e.g. Tableau, PowerBI, etc..)
- Self host a Python server (e.g. matplotlib, seaborn, etc..)
- **A simple HTML file** generated with Template and some Javascript Library (Below example)
Each of the option has it’s pros and cons, a static file (like HTML) provides you an other options, if other commercial tools or hosting a python server is not available.
Google Chart Library
Google chart tools are powerful, simple to use, and free. And usually these kinds of Javascript chart library provides a richer (and customizable) analytic gallery for you to use, which sometimes it is quite difficult to do it in excel, like [Sankey Diagram](https://developers.google.com/chart/interactive/docs/gallery/sankey), [Interval Plots](https://developers-dot-devsite-v2-prod.appspot.com/chart/interactive/docs/gallery/intervals) and [Tree Map](https://developers-dot-devsite-v2-prod.appspot.com/chart/interactive/docs/gallery/treemap).
Here I will use a simple **bar chart** as an example. And you will see how we can create a simple HTML file with bar chart (or other graphs) easily if you know how to use Jinja Template.
This is the outline of the steps to create a standalone HTML file with Google Chart
- Copy the **minimal example** of Stacked bar chart ([Google Charts](https://developers.google.com/chart/interactive/docs/gallery/barchart#chart_description))
- Replace the `data` part with Jinja Template variable
- Try it with dummy data
- Replace it with real data. Here we will use the **Daily Passenger Traffic** for different Control Points from HK Government Open Data ([data.gov.hk](https://data.gov.hk/en-data/dataset/hk-immd-set5-statistics-daily-passenger-traffic))
- Render the Template, and export as HTML file
#### a) Stacked Bar Chart
Let’s copy the minimal example of stack bar chart. I rename some variables name from the original example here (e.g. materialChart to chart, etc..)
The code is pretty easy to read. We import some chart library with `<script type=”text/javascript” src=”https://www.gstatic.com/charts/loader.js"></script>` and initialize the google chart object and callback function with with `google.charts.setOnLoadCallback(drawChart);`. It’s quite straight forward even if you do not have much experience with Javascript.
<html>
<head>
<script type="text/javascript" src="https://www.gstatic.com/charts/loader.js"></script>
<script type="text/javascript">
google.charts.load('current', {packages: ['corechart', 'bar']});
google.charts.setOnLoadCallback(drawChart);
function drawChart() {
var data = google.visualization.arrayToDataTable([
['City', '2010 Population', '2000 Population'],
['New York City, NY', 8175000, 8008000],
['Los Angeles, CA', 3792000, 3694000],
['Chicago, IL', 2695000, 2896000],
['Houston, TX', 2099000, 1953000],
['Philadelphia, PA', 1526000, 1517000]
]);
var options = {
chart: {
title: 'Population of Largest U.S. Cities'
},
hAxis: {
title: 'Total Population',
minValue: 0,
},
vAxis: {
title: 'City'
},
bars: 'horizontal'
};
var chart = new google.charts.Bar(document.getElementById('chart_div'));
chart.draw(data, options);
}
</script>
</head>
<body>
<div id="chart_div"></div>
</body>
</html>b) Replace with Jinja Template variables
Then we can replace some of the values with Template variable like `{{ data }}`, `{{ title }}`, `{{ h_axis }}`, etc..
from jinja2 import Template
t = Template(
"""
<html>
<head>
<script type="text/javascript" src="https://www.gstatic.com/charts/loader.js"></script>
<script type="text/javascript">
google.charts.load('current', {packages: ['corechart', 'bar']});
google.charts.setOnLoadCallback(drawChart);
function drawChart() {
var data = google.visualization.arrayToDataTable(
{{ data }}
);
var options = {
chart: {
title: '{{ title }}'
},
hAxis: {
title: '{{ h_axis }}',
minValue: 0,
},
vAxis: {
title: '{{ v_axis }}'
},
bars: 'horizontal'
};
var chart = new google.charts.Bar(document.getElementById('chart_div'));
chart.draw(data, options);
}
</script>
</head>
<body>
<div id="chart_div"></div>
</body>
</html>
"""
)c) Test with Dummy Data
If we go back and study the embeded data in the json, we will see the data we need is a **list of list**, with 3 columns (One categorical and two numerics). So we we create the dataframe accordingly, with `[“Control Point”, “Arrival”, “Departure”]` as our column for our traffic data.
# The embeded data in the original json. A list of list with each record as an element.
[
['City', '2010 Population', '2000 Population'],
['New York City, NY', 8175000, 8008000],
['Los Angeles, CA', 3792000, 3694000],
['Chicago, IL', 2695000, 2896000],
['Houston, TX', 2099000, 1953000],
['Philadelphia, PA', 1526000, 1517000]
]Let’s create a dummy dataframe, and convert it to list of list object.
import pandas as pd
df = pd.DataFrame({
"Control Point": ["Airport", "Lok Ma Chau", "Lo Wu"],
"Arrival": [154, 120, 40],
"Departure": [21, 40, 32],
})
print(df.head())
# Convert dataframe to List of list, and insert to column header as the first element.
data = df.values.tolist()
data.insert(0, df.columns.tolist())
print(data)
# Output
# df
Control Point Arrival Departure
0 Airport 154 21
1 Lok Ma Chau 120 40
2 Lo Wu 40 32
# data
[
['Control Point', 'Arrival', 'Departure'],
['Airport', 154, 21],
['Lok Ma Chau', 120, 40],
['Lo Wu', 40, 32]
]d) Replace with real data
Here we will get the 2022 passenger traffic data from HK Government Open Data ([data.gov.hk](https://data.gov.hk/en-data/dataset/hk-immd-set5-statistics-daily-passenger-traffic)). And aggregate the data from daily records to yearly records. For the df operation, you can check out my previous post on [Common Pandas Functions](https://data-gulu.com/2023/10/18/18-useful-pandas-functions-for-data-science/).
import pandas as pd
# Get the 2022 arrival/departure data
url = "https://www.immd.gov.hk/opendata/eng/transport/immigration_clearance/statistics_on_daily_passenger_traffic.csv"
df = pd.read_csv(url)
df = df[["Date", "Control Point", "Arrival / Departure", "Total"]]
df['Date'] = pd.to_datetime(df['Date'], format="%d-%m-%Y")
df_subset = df[(df['Date'] >= '2022-01-01')
& (df['Date'] <= '2022-12-31')].reset_index(drop=True) # 2022 data only, for simplicity
print(df_subset)
# Group by Control Point. We can safely ignore date, as we are only using 2022 data
df_agg = df_subset.groupby(["Control Point", "Arrival / Departure"]).agg(Total=("Total", "sum"),).reset_index()
df_agg = df_agg[df_agg.Total > 0].reset_index(drop=True)
print(df_agg)
# Long to wide. pivot_table is used here, just for the fill_value argument
df_wide = df_agg.pivot_table(
index=["Control Point"],
columns="Arrival / Departure",
values=["Total"],
aggfunc="sum",
fill_value=0,
)
df_wide_flattened = df_wide.copy()
df_wide_flattened.columns = ["_".join(x) for x in df_wide_flattened.columns.to_flat_index()]
df_wide_flattened.reset_index(inplace=True)
print(df_wide_flattened)
data = df_wide_flattened.values.tolist()
data.insert(0, df_wide_flattened.columns.tolist()) # Insert back the column header
print(data)# df_subset (2022 Data)
Date Control Point Arrival / Departure Total
0 2022-01-01 Airport Arrival 628
1 2022-01-01 Airport Departure 778
2 2022-01-01 Express Rail Link West Kowloon Arrival 0
3 2022-01-01 Express Rail Link West Kowloon Departure 0
...
# df_agg (Aggregate to Year level)
Control Point Arrival / Departure Total
0 Airport Arrival 1957891
1 Airport Departure 2175626
8 Heung Yuen Wai Arrival 301
10 Hong Kong-Zhuhai-Macao Bridge Arrival 77075
...
# df_wide_flattened (Long to wide table)
Control Point Total_Arrival Total_Departure
0 Airport 1957891 2175626
1 Heung Yuen Wai 301 0
2 Hong Kong-Zhuhai-Macao Bridge 77075 114504
3 Kai Tak Cruise Terminal 8233 3610
4 Shenzhen Bay 485248 439669
# data (Final List of List object for input of the graph)
[['Control Point', 'Total_Arrival', 'Total_Departure'], ['Airport', 1957891, 2175626], ['China Ferry Terminal', 0, 0], ..]e) Render with Jinja Template, and export to html file
Finally, we can render the variables with Jinja, and export to the HTML file.
from jinja2 import Template
import pandas as pd
# Get the 2022 arrival/departure data, hopefully the URL will not break
url = "https://www.immd.gov.hk/opendata/eng/transport/immigration_clearance/statistics_on_daily_passenger_traffic.csv"
df = pd.read_csv(url)
df = df[["Date", "Control Point", "Arrival / Departure", "Total"]]
df['Date'] = pd.to_datetime(df['Date'], format="%d-%m-%Y")
df_subset = df[(df['Date'] >= '2022-01-01')
& (df['Date'] <= '2022-12-31')].reset_index(drop=True) # 2022 data only, for simplicity
# Group by Control Point. We can safely ignore date, as we are only using 2022 data
df_agg = df_subset.groupby(["Control Point", "Arrival / Departure"]).agg(Total=("Total", "sum"),).reset_index()
df_agg = df_agg[df_agg.Total > 0].reset_index(drop=True)
# Long to wide
df_wide = df_agg.pivot_table(
index=["Control Point"],
columns="Arrival / Departure",
values=["Total"],
aggfunc="sum",
fill_value=0,
)
df_wide_flattened = df_wide.copy()
df_wide_flattened.columns = ["_".join(x) for x in df_wide_flattened.columns.to_flat_index()]
df_wide_flattened.reset_index(inplace=True)
# List of List
data = df_wide_flattened.values.tolist()
data.insert(0, df_wide_flattened.columns.tolist()) # Insert back the column header
# Create template
t = Template("""
<html>
<head>
<script type="text/javascript" src="https://www.gstatic.com/charts/loader.js"></script>
<script type="text/javascript">
google.charts.load('current', {packages: ['corechart', 'bar']});
google.charts.setOnLoadCallback(drawChart);
function drawChart() {
var data = google.visualization.arrayToDataTable(
{{ data }}
);
var options = {
chart: {
title: '{{ title }}'
},
hAxis: {
title: '{{ h_axis }}',
minValue: 0,
},
vAxis: {
title: '{{ v_axis }}'
},
bars: 'horizontal'
};
var chart = new google.charts.Bar(document.getElementById('chart_div'));
chart.draw(data, options);
}
</script>
</head>
<body>
<div id="chart_div"></div>
</body>
</html>
""")
# Render template, and save to a html file
output = t.render(
data=data,
title="Inbound and Outbound Passenger at different Control Points (Year 2022)",
h_axis="Passengers",
v_axis="Control Point",
)
with open('output.html', 'w') as f:
f.write(output)**Output** — [output.html](https://storage.googleapis.com/billylkc-blog-image/images/006_traffic_output.html)
— -
### Final Thoughts
There is a lot of other Javascript chart packages (e.g. [Vega Chart](https://vega.github.io/vega/), [HighCharts](https://www.highcharts.com/), etc..), you can easily create some pretty charts with the techniques here with Jinja Template. As it is in HTML format, you can easily include some Insight session as well for some quick comments for your report.
**Remarks:** Just be careful for not leaking any sensitive data, as a static report is difficult to do access control or audit logging.
I also write on my own blog ([https://data-gulu.com](https://data-gulu.com/2023/10/29/introduction-to-jinja-template-and-google-chart/)). You can find more articles about python and machine learning there.
Happy Coding!