How I Deployed a Crime Heat Map Web Application using Python, Flask, and Heroku

Police officers in the city of Berkeley, California deal with approximately 77,000 calls for service every year, according to berkeleyside.org. As a junior studying Data Science at UC Berkeley (2022 update: I graduated!), I understand that one must always be alert when living in a city and make smart decisions regarding where one goes, who one mingles with, and the actions one takes. With this in mind, I decided to use some free time this summer to create something that could help Berkeley residents stay safe: a crime heat map that allows users to interactively explore the locations and densities of crimes in Berkeley, which includes an information panel with latitude, longitude, and crime type.

Check it out here: https://berkeley-crime-heatmap-app.herokuapp.com/

2023 Update: Free Heroku dynos are unfortunately no longer available, meaning the site above is currently not operational. You will need to subscribe to Heroku’s Eco plan in order to proceed with this tutorial.

I’m writing this article to not only document my work for future reference but to provide a guide for people to create their own Python applications if they are interested. If you’re the type of person that would rather follow along with the GitHub repository, click here.

Downloading and Cleaning the Data

For the Python code, we will be working in Pandas, a Python data manipulation and analysis library. This article assumes the reader has some knowledge of Pandas, although explanations will be provided.

To begin with, let’s first obtain the data from the City of Berkeley public database. Create a file named main.py to store all your code for the data download and cleaning. While there are multiple ways to export the data, I decided to export via The Socrata Open Data API (SODA). Since SODA has a default record limit of 1000, I needed to change the $limit parameter to the maximum $limit of 50,000.

crime = requests.get(“https://data.cityofberkeley.info/resource/k2nh-s5h5.json?$limit=50000")
crime = crime.json()
crime = pd.DataFrame.from_dict(crime)

Next, we must extract the latitude and longitude from the “block_location” column, check for NA values, and drop unnecessary columns. This process is pretty straightforward, although I’ve included the code below for reference:

# add coordinates column
crime[‘Latitude’] = crime[‘block_location’].apply(lambda x: x.get(‘latitude’))
crime[‘Longitude’] = crime[‘block_location’].apply(lambda x: x.get(‘longitude’))
crime[‘Longitude’] = crime[‘Longitude’].replace(r’\s+’, np.nan, regex=True)
crime[‘Longitude’] = crime[‘Longitude’].replace(r’^$’, np.nan, regex=True)
crime[‘Longitude’] = crime[‘Longitude’].fillna(-0.99999)
crime[‘Longitude’] = pd.to_numeric(crime[‘Longitude’])
crime[‘Latitude’] = crime[‘Latitude’].replace(r’\s+’, np.nan, regex=True)
crime[‘Latitude’] = crime[‘Latitude’].replace(r’^$’, np.nan, regex=True)
crime[‘Latitude’] = crime[‘Latitude’].fillna(-0.99999)
crime[‘Latitude’] = pd.to_numeric(crime[‘Latitude’])
# check for any NA/null values
crime.isnull().sum()
crime = crime.drop([‘:@computed_region_b3wi_w8ix’, 
 ‘:@computed_region_fhmw_rucx’,
 ‘:@computed_region_u3y2_d2ws’,
 ‘:@computed_region_5s6d_2f32’,
 ‘:@computed_region_3ini_iehf’,
 ‘:@computed_region_5bih_7r3y’,
 ‘:@computed_region_x3q3_gi3e’,], axis=1)
# drop null values
crime = crime.dropna()

Building the Heat Map

To code the heat map, we will use Folium, a Python library for visualizing geospatial data. While this was my first time using Folium, I was thoroughly impressed at how simple and easy it makes plotting interactive maps. First, let’s initialize the map object, setting the location to the UC Berkeley campus and the zoom to 14.5.

map = folium.Map(
 location=[37.871853, -122.258423],
 zoom_start=14.5
)

All the above line of code does is create a blank map; we need to add the crime data points for it to be useful! First, create a MarkerCluster object, which we will think of as a layer of the map that contains the crime data points. We will then be placing this layer on the blank map, therefore populating it with crime markers.

marker_cluster = MarkerCluster(
 name=”Crimes by Marker”,
 overlay=True,
 control=True
)

Now that we have a MarkerCluster object, we must add each individual crime record to the object. While there are multiple ways to do this, I chose to use iteration due to the relatively small dataset size. Let’s convert the “Latitude” and “Longitude” columns in our crime dataframe to lists, then add each one to marker_cluster.

lat = crime[‘Latitude’].tolist()
lng = crime[‘Longitude’].tolist()
offense = crime[‘offense’].tolist()
locations = list(zip(lat, lng))
offense = crime[‘offense’].tolist()
for i in range(len(lat)):
 location = lat[i], lng[i]
 crime_type = offense[i]
 html = ‘’’<b> Type of Crime: </b> {}<br>
 Latitude: {}<br>
 Longitude:{}<br>’’’.format(crime_type, location[0], location[1])
 iframe = folium.IFrame(html, width=200, height=200)
 popup = folium.Popup(iframe, max_width=200)
 marker = folium.Marker(location= location,popup=popup)
 marker_cluster.add_child(marker)
marker_cluster.add_to(map)

Notice how markers in Folium have a popup parameter. This allowed me to make a simple HTML-formatted popup for each marker that includes the crime type, latitude, and longitude.

Funny enough, the actual heat map component is pretty easy and only requires two line of code.

HeatMap(
 data=list(zip(lat, lng)),
 name=”Crimes by Heatmap”).add_to(map)
folium.LayerControl().add_to(map)

And, voilà! That’s it!

Setting Up Project Folder for Deployment

This was my first ever deployment of a Python application to the web and I quickly learned that translating a raw Python script file to a web-friendly HTML page was trickier than I anticipated.

We spent the previous part working on our main.py file. For this next part we will use Flask, a micro web framework for Python, and Heroku, a cloud platform as a service that allows for Python web app deployment. We will now create a project folder with the following file arrangements:

Blue nodes are folders; red nodes are files

1. main.py contains the Python code for the Folium heat map, created in the previous sections above.
2. wsgi.py contains the following code:

from app.main import app
if __name__ == “__main__”:
 app.run()

3. requirements.txt lists all of the application’s dependencies that allow it to work correctly. In this case, we only need a few lines:

numpy==1.17.2
pandas==0.25.1
Flask==1.1.1
requests==2.22.0
folium==0.12.1

4. runtime.txt specifies the Python Runtime that will be used by Heroku. A bug I ran into was that Heroku only supports specific Python runtimes:

python-3.7.11

5. Procfile tells Heroku the commands that will be ran by the application at the very beginning:

web: gunicorn wsgi:app

6. index.html is a bit trickier. Depending on how you want your HTML layout to look, this file will vary. For my application, I ended up using Bootstrap, a free CSS framework that makes HTML web development easy. Bootstrap’s website has a ton of cool templates to choose from for web designs.

While I won’t get into the specifics of Bootstrap (feel free to checkout my GitHub repository to see exactly how I used it), I will go over how to insert the Folium heat map into the HTML webpage, which requires two steps:

A) Go back to main.py and add the follow code at the bottom of the file:

html_map = map._repr_html_()
app = Flask(__name__)
@app.route(“/”)
def index():
 return render_template(“index.html”, map = html_map)
if __name__ == “__main__”:
 app.run(host=”127.0.0.1", port=8080, debug=True)

This converts our Folium map into an HTML object that we can then pass into our render_template function to allow it to be used in our index.html file.

B) Open index.html. Choose where in the page you want to insert your Folium map and then use the follow code to insert:

<div class=”col-md-8" style= “margin:0 auto;”>
 {{map|safe}}
 </div>

It’s a good idea to test your app via Flask to ensure nothing is breaking. Refer to this article by Real Python for the specifics of testing the application locally using Flask.

Deployment to the Web

Congrats on making it this far! The last step: web deployment. There are many ways to deploy your Python App, including Google Cloud Platform and Heroku. I used Heroku in my project but feel free to use any of your choice.

In your terminal, navigate to your crime-app folder and run the following command:

pipenv install flask gunicorn

What this does is create a virtual environment that allows for the Python libraries and dependencies to be kept separate from other projects.

Next, type this command to run the virtual environment:

pipenv shell

Then initialize a git repository, add the files, and commit:

git init 
git add .
git commit -m "Our first commit"

Now, assuming you have a Heroku account set up already, run the following commands in sequence:

heroku login
heroku create this-is-my-crime-app
git push heroku master

Done! Your application will be deployed to http://this-is-my-crime-app.herokuapp.com (at the moment, there is nothing here since this is an example).

Again, if you want to check out my working version of the crime application, go here: https://berkeley-crime-heatmap-app.herokuapp.com/

Conclusion

While my crime application is pretty simplistic, I wanted to challenge myself this summer by deploying a Python application for the first time. The process was definitely worthwhile and I look forward to working more with Heroku and Flask in the future. Moreover, I would love to expand upon the current site; ideas that I had included making an AI-based navigation app that recommends routes that avoid areas of high density crime and a dashboard that contains general crime statistic summaries for various regions. Ultimately, I want to continue using my coding knowledge to provide the public with something that will be of use in everyday city lives.