Linkovation: Designing a Python-Powered URL Shortener System

In this article, we will look into how we can implement a URL shortener system using Python.

First let’s look at what is meant by URL shortener. You all must have sometimes received a short URL starting with, for example, https://tinyurl/r/abcde, and when you click on this URL, it will redirect you to the original URL.

Here the suffix ‘abcde’ represents a short code for the original URL and there are many systems/companies like tinyurl, bitly which gives us these short URLs.

But why do we require these short URLs?

Many companies or businesses use these short urls because they are easy to dictate to any one. Many companies may be having very long urls which may contain different characters and look weird to represent. Instead, using these short urls and then redirecting users to their main urls after clicking these is much easier and simpler. Basically short urls make sharing easier.

Implementing URL shortener using Python

Here we will use flask as the main backbone to implement this system.

High level overview:

1. We will create a page which will contain an input box where the company user who wants to create a short url will enter the Original URL/long URL and click on submit.
2. We have an app route “/create-short-url” which accepts the input and check in a table if short url already exists for that. If it exists it will return it other wise will proceed further to create a short url for it.
3. The function will call a database table — URLRange to retrieve a number for eg. 10000 which is atleast greater than 3 digits from a random range(primary key of the table). These ranges will be preloaded at the startup.
4. The number that is received will be converted into binary format.
5. We would store a map of binary numbers as a key and unique alphabets from a to z and A to Z and numbers from 0–9 in an in memory DB like redis such that each character or number from 0 to 9 is mapped to a 6 digit binary number.
6. We will then split the converted binary number into 6 digit each such that the length of the whole binary number is a multiple of 6. If the binay number is not a multiple of 6 we will add extra zeroes on the left such that it becomes.
7. For each splitted 6 digit number we will lookup in redis to get the corresponding alphabet or a number from 0 to 9 such that we will receive a code eg. ‘rtef ‘
8. This code along with the original url will be stored in one of the table and a short url like http://localhost:5000/rtef will be returned back to the user.
9. Whenever any user clicks on such short urls we will have a function in our class which will lookup for the original url in the db depending on this short code and will retrieve the original url and redirect to it.

Low level implementation details

a. Create corresponding tables:

1. url_range: It holds the ranges along with other columns

2. mapping_code: It holds 6 digit binary codes and mapped to a random alphabets and numbers. We will hold these values as key value pairs in Redis on startup.

3. url_details: This will hold the created short url codes, the original urls and other metadata.

b. In Flask code start with importing required libraries and in app context initialize the required app variables as below:

import redis, random, datetime, pandas as pd
from flask import Flask, render_template, flash, redirect
from flask_bootstrap import Bootstrap
from forms import URLForm
from models import db, Mapping, URLRange, URLDetails

app = Flask(__name__)
short_url_prefix = "http://127.0.0.1:5000/"

with app.app_context():
    app.config['SECRET_KEY'] = 'Youarethesecretkey23456'
    app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///url_shortener.db'
    r = redis.StrictRedis(host='localhost', port=6379, db=0)
    db.init_app(app)
    db.create_all()
    Bootstrap(app)
    url_range_id = []
    #Get mapping codes
    mapping_codes = Mapping.query.all()
    for row in mapping_codes:
        number = row.mapping_number
        alphabet = row.mapping_character
        if not r.exists(number):
            r.set(number,alphabet)
    
    #Get ranges
    url_ranges = URLRange.query.filter_by(id=URLRange.id).all()
    for range_id in url_ranges:
        url_range_id.append(range_id.id)

c. Create a models.py file and create model classes in it which will create the above tables on load.

from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()


class URLRange(db.Model):
    __tablename__ = 'url_range'
    id = db.Column(db.Integer, primary_key=True)
    start_range = db.Column(db.Integer, nullable=False)
    end_range = db.Column(db.Integer, nullable=False)
    current_number = db.Column(db.Integer, nullable=False)


class URLDetails(db.Model):
    __tablename__ = 'url_details'
    id = db.Column(db.Integer, primary_key=True)
    short_code = db.Column(db.String(20), nullable=False)
    original_url = db.Column(db.String(1000), nullable=False)
    no_of_clicks = db.Column(db.Integer, nullable=False)
    date = db.Column(db.String(250), nullable=False)


class Mapping(db.Model):
    __tablename__ = 'mapping_code'
    mapping_number = db.Column(db.String(6), primary_key = True)
    mapping_character = db.Column(db.String(1), nullable = False)

d. Create short urls code:

#App route for creating short urls
@app.route("/create-short-url", methods=['GET','POST'])
def get_url():
    url_form = URLForm()
    #Code to get orginal url from UI
    if url_form.validate_on_submit():
        org_url = url_form.url.data
        org_url_list = URLDetails.query.all()
        for org in org_url_list:
            if org.original_url == org_url:
                flash('Short URL already exists for your business URL...')
                return render_template("add.html", short_url=f"{short_url_prefix}{org.short_code}", short_url_form=url_form)
        url_range = random.choice(url_range_id)
        range_details = URLRange.query.filter_by(id=url_range).first()
        if range_details.current_number <= range_details.end_range:
            binary_code = convert_binary(range_details.current_number)
            range_details.current_number = range_details.current_number + 1
            db.session.commit()
            if len(binary_code) > 0:
                short_url = save_short_url(org_url, binary_code)
                save_short_url_in_file(org_url, short_url)
                return render_template("add.html", short_url=short_url, short_url_form=url_form)
    return render_template("add.html", short_url_form=url_form)


#Save short urls
def save_short_url(org_url, binary_code):
    print(f"Org url: {org_url} and binary code: {binary_code}, {len(binary_code)}")
    group_length = 6
    groups = [binary_code[i:i+group_length] for i in range(0, len(binary_code), group_length)]
    code_list = []
    code_list_details = [Mapping.query.filter_by(mapping_number=group).first() for group in groups]
    for code in code_list_details:
        code_list.append(code.mapping_character)

    final_code_list = ''.join(code_list)
    url_details = URLDetails(short_code=final_code_list, original_url=org_url, no_of_clicks=0,date=datetime.date.today().strftime("%Y-%m-%d"))
    db.session.add(url_details)
    db.session.commit()
    return f"{short_url_prefix}{final_code_list}"


#Convert the number to binary
def convert_binary(number):
    #Convert the umber to binary
    binary_code = bin(int(number))[2:]
    count_pad = 1
    while len(binary_code) % 6 != 0:
        binary_code = str(binary_code.zfill(len(binary_code) + count_pad))
    return binary_code

#Save short urls in file
def save_short_url_in_file(org_url, short_url):
    data = {
        'Original-URL': [org_url],
        'Short-URL': [short_url]
    }

    # Make data frame of above data
    df = pd.DataFrame(data)
    # append data frame to CSV file
    df.to_csv('short_url.csv', mode='a', index=False, header=False)

e. Code for redirecting the short url to the original url when clicked on it.

#App route to redirect from short url to original url
@app.route("/<short_url_code>")
def get_original_url(short_url_code):
    org_url_details = URLDetails.query.filter_by(short_code=short_url_code).first()
    if org_url_details:
        org_url_details.no_of_clicks = org_url_details.no_of_clicks + 1
        db.session.commit()
        return redirect(org_url_details.original_url)
    else:
        return render_template("index.html", message="Original URL not found")

f. add.html file which will showcase the UI where the user enters the original url for creating short urls.

{% extends 'bootstrap/base.html' %}
{% block styles %}
  {{super()}}
  <link rel="stylesheet" href="{{ url_for('static', filename='css/styles.css') }}">
{% endblock %}
{% import "bootstrap/wtf.html" as wtf %}
<html>
<head>
{% block title %}Create Short URL's {% endblock %}
</head>
{% block content %}
<body>
<div class="jumbotron">
    <div class="container">
        <h1>Create Short URL's for your business</h1>
        {{wtf.quick_form(short_url_form)}}
        {% with messages = get_flashed_messages() %}
            {% if messages %}
                <ul class="flash-messages">
                    {% for message in messages %}
                    <h3>{{ message }}</h3>
                    {% endfor %}
                </ul>
            {% endif %}
        {% endwith %}
       {% if short_url %}
        <h2>Short URL is: {{short_url}}</h2>
        {% endif %}
    </div>
    <div class="container">

    </div>
</div>
</body>
{% endblock %}
</html>

The whole code along with a video that I have recorded showing the creation of short url and then its redirection to the original url is uploaded to github at https://github.com/sagar160589/python-url-shortener

That’s all folks for this article. If you like it please follow me for more such articles!!