Create a free, easy-to-implement and customizable (and a bit slow :D) Distance Matrix API using Flask with a Selenium Bot deployed on Heroku
If you are interested in articles related to Data Science for Supply Chain feel free to have a look at my portfolio: https://samirsaci.com
My first project using GPS routing was 4 years ago. I wanted to optimize a transport plan for 1,200 trucks deliveries/month covering 50 stores from a Cross-Docking platform.
I have built a Transportation Route Optimization tool using Excel-VBA — mainly for transport plan design using distance collected from Google Maps API.
This was my first experience using an API; Google Maps API was free with a limit of 10,000 requests per day.
What about now?
A few years later, Google has changed its billing policy so you have to pay from the first request.
If you have never subscribed to the Google Cloud Platform (GCP) service you can have 200$ free credits after setting up a credit card.
But, what if
- you need to get several thousand distances?
- you don’t care if it takes a long time?
- you don’t feel confident about using your personal card (or cannot get a company credit card) for non-personal projects?
This article will show you a solution built with a Flask API using a selenium bot connected to Google Map WebPage.
How does it work?
Before starting to read this part, please forget everything you know about how to put in production a fast, efficient and stable code ensuring quick response with limited resources.
This will be simple, quick and dirty, with no intention to be a scalable solution. The performance will be way lower than if you directly query the official Google API — but here it’s free :)
Build your API
Let us do it in three steps
- Build a Selenium Bot that will query the distance from City A to City B in Google Maps Website
- Set up your Flask API that will receive the request and return a distance
- Deploy your code on Heroku
Set up your Selenium Bot
- Set ChromeDriver options to ensure the highest speed of execution
- Input Environment Variables that will be created in your Heroku instance
Write your distance scrapper
Google Maps link to get distance from "Paris,France" to "Marseille, France"https://www.google.fr/maps/dir/Paris,France/Marseille, France/data=!4m2!4m1!3e0"/data=!4m2!4m1!3e0" is added to ensure that you take the road transportation distance
Set up your Flask API
Your API link to get distance from "Paris,France" to "Marseille, France"http://xxx-xxx.herokuapp.com/distance/Paris,France/Marseille,France
(replace xxx-xxx by your heroku app name)<fr> = Paris,France
<to> = Marseille, France
Deploy your API
I will skip details on how to create and deploy an app on Heroku. You can find links to Medium articles explaining detailed steps to create your Heroku instance at the end of the article.
Prepare files for deployment on Heroku
Prepare requirements.txt file with a listing of libraries needed with pip freeze
(env) C:\Users\yourprojectfolder> pip freeze > requirements.txt
Create ProcFile to launch your web app
(env) C:\Users\yourprojectfolder> echo web: gunicorn -t 120 -b :$PORT app:app > ProcfileP.S: Please make sure that your app name is "app" and your python script is named "app.py"
Download Buildpacks on Heroku to use Selenium + ChromeDriver
Go to settings > Add Buildpack
Enter Two Links
Set up Environment Variables
Test your API
Test your API to calculate the distance
From: Paris, France
To: Marseille, France
(replace xxx-xxx by your Heroku app name)
What can we get in Google Maps?
It’s matching :)
Conclusion and next steps
I deployed this solution on a free Heroku instance and tested it using a Google Sheet querying my API to get 40 road distances.
Next Step 1: Find a way to ensure that your sheets send queries once at a time
If you not, you can quickly exceed your memory quota
Step 2: Errors management and extract all distances
You can see in the example above, the first result showed is the shortest travel time and may not be the shortest distance. Route time can change if you query at a different time of the day, so you’d better take the three distances.
Many details were skipped in this article to make it concise and easy to read. You can find detailed instructions in the excellent articles listed below.