My Wifi was Slow… My ISP Didn’t Believe Me

Proving it with a historical network speed dashboard built with Python, Plotly, DuckDB, and Streamlit.

4 min readAug 26, 2024

In today’s hyper-connected world, the speed and reliability of your internet connection can dramatically impact both your productivity and your sanity. Imagine being in the middle of a crucial video call, only for your connection to stutter or drop entirely. Frustrating, right? This was the predicament I found myself in, but with a twist: my Internet Service Provider (ISP) insisted that everything was working perfectly on their end.

The Challenge: My ISP Didn’t Believe Me

For weeks, I grappled with intermittent yet persistent slow-downs. Calls to my ISP resulted in the same script: “We’ve checked your connection. Everything seems fine from our side.” It was clear that if I wanted this resolved, I’d have to provide incontrovertible evidence that the problem wasn’t just a figment of my imagination.

The Solution: A DIY Network Speed Dashboard

As a machine learning scientist and someone deeply immersed in technology, I decided to tackle this issue with the best tools at my disposal: Python, Plotly, DuckDB, and Streamlit. My goal was to create a comprehensive network speed dashboard that would log, visualize, and analyze my internet speed over time. This would allow me to present my ISP with hard data on the inconsistencies in my internet service.

Step 1: Collecting the Data with Python

I started by writing a Python script that would measure my internet speed at regular intervals using the `speedtest-cli` library. This library interfaces with Speedtest.net’s infrastructure to measure ping, download speed, and upload speed. I set the script to run every 30 minutes, logging the results into a local database created with DuckDB, a fast, SQL-standard, and embeddable database that’s perfect for lightweight applications.

import speedtest
import duckdb
import pandas as pd
from datetime import datetime

# Setup Speedtest
s = speedtest.Speedtest()

# Measure speeds
ping = s.ping()
download = s.download()
upload = s.upload()

# Timestamp for logging
now = datetime.now().strftime('%Y-%m-%d %H:%M:%S')

# Insert into DuckDB
conn = duckdb.connect('internet_speed.db')
create = """
CREATE TABLE IF NOT EXISTS 
  speed_tests (
    date TIMESTAMP
    , ping REAL
    , download REAL
    , upload REAL
)
"""
conn.execute(create)
insert_stmt = """
INSERT INTO 
  speed_tests VALUES (?, ?, ?, ?)
"""
vals = (now, ping, download, upload)
conn.execute(insert_stmt, vals)
conn.close()

Step 2: Visualizing the Data with Plotly

With a growing dataset, my next step was to visualize the data to identify patterns or inconsistencies. Plotly, a powerful graphing library, allowed me to create interactive charts that showed the fluctuations in ping, download, and upload speeds over time.

import plotly.express as px

# Load data from DuckDB
conn = duckdb.connect('internet_speed.db')
df = conn.execute("SELECT * FROM speed_tests").df()
# Create interactive time series plot
fig = px.line(df, x='date', y=['ping', 'download', 'upload'], title='Internet Speed Over Time')
fig.show()

Step 3: Sharing the Dashboard with Streamlit

Finally, to make my findings shareable and interactive, I used Streamlit to create a web dashboard. Streamlit is an open-source app framework for Machine Learning and Data Science projects. With minimal setup, I was able to turn my scripts and plots into a live dashboard, showcasing my internet speed tests over time.

import streamlit as st

# Load data
conn = duckdb.connect(‘internet_speed.db’)
df = conn.execute("SELECT * FROM speed_tests").fetchdf()
# Streamlit dashboard
st.title('My Internet Speed Dashboard')
st.plotly_chart(fig)

The Outcome: Evidence Unveiled

Armed with weeks of data and a comprehensive dashboard, I contacted my ISP once more. This time, I wasn’t just a frustrated customer; I was a customer with evidence. The detailed logs and visualizations made my case undeniable.

Conclusion

This experience reinforced a valuable lesson: data speaks louder than words. By leveraging Python, Plotly, DuckDB, and Streamlit, I turned a personal frustration into a problem-solving project. Not only did this approach solve my immediate issue, but it also provided me with a versatile toolkit for future data analysis tasks. For anyone facing similar challenges, remember that the tools and techniques to uncover the truth are at your fingertips.

Please follow me for more on software engineering, data science, machine learning, and artificial intelligence.

If you enjoyed reading, please consider supporting my work.