Extending Global Internet Access without building anything
How we can extend Internet bandwidth with smart network optimization
Slow Internet sucks. It limits our ability to communicate with people, entertain ourselves, and get work done. For many of you reading this article, fast internet seems more of a necessity or a right than a luxury at this point.
But in the United States, seen as one of the most developed countries in the world, 37% of people living outside of cities lack access to high-speed broadband connection.
Zooming out a little bit to the world, we find that 37% of the world (2.9B people) have never used the internet. This statistic is very different to the first one about the United States showing that a large portion of the countries has a “spotty” connection. Almost 3B people have never even looked something up on the Internet.
And 96% of these people live in developing countries, often in rural areas, where access to the Internet could change their lives. In fact, only 36% of Least Developed Countries (LDCs) use the Internet.
Think about the doors a stable Internet connection could open for a young student in Burundi (where only 6% of people use the Internet). Using Kahn Academy to pass school, having Zoom calls with people around the world, and watching YouTube videos about anything that interests them.
Further, we as a society are building technology to provide opportunities to people in these LDCs, but without a stable Internet connection, how can these technologies even get used? For example, in a hackathon, I built a consortium blockchain platform for drug approval boards to keep a ledger of official drugs in their local marketplace to eliminate the counterfeit drug market. Let’s say a technology like this was approved in a region in an LDC. How would the technology impact anyone who can’t access the internet? SMS-based technology can only take individuals so far.
Why is rural Internet connection so bad?
There are three main reasons that can explain a poor Internet connection in rural areas.
- Poor Infrastructure
Although you usually don’t have any wires that connect to your phone or laptop for Internet connectivity, a vast interconnected network of cables, routers, cell towers, switches, and servers are needed to provide a connection to the web. In areas with lower population densities, it is more difficult to provide the infrastructure for every single person because of how spread out people are. A lot of the infrastructure needs depend on what type of internet connection is being used, but overall, the rollout of Internet infrastructure in rural areas has been much slower than in densely populated urban areas
2. Hard-to-reach areas
Many rural areas are isolated from the telephone exchanges that deliver broadband connection and it is therefore much more difficult for broadband signals to reach the device. It is also incredibly difficult to provide the infrastructure to regions where the land is vastly different from the nearby urban areas.
3. Expensive to Install
While Internet connection is emerging as a human right as technology gets more advanced, the industry of providing Internet services is still a profit-maximizing industry. That is, if a service provider maximizes profit by providing Internet to a region it will, otherwise it will overlook that region. Governments of more developed countries have begun to subsidize the services in rural areas to increase the spread of these services, but this spread is still very slow. Further, starting up a new ISP (Internet Service Provider) company is not easy because of the high barriers to entry due to government regulation. This regulation is why you see oligopolies with telecommunications companies around the world.
What Biden is doing for rural communities in the United States
In the United States, a major cause of 39% of the people in rural areas lacking access to sufficient Internet is related to infrastructure limitations. But including this number, almost half the country doesn’t use high-speed broadband because they either can’t afford it or because they don’t have the skills or knowledge to use it.
Biden’s office has recognized the extreme digital divide that is being developed every day rural communities lack high-speed broadband. In fact, his office has acknowledged that fast Internet can quite literally save lives.
During the Pandemic, it was found that a 1% increase in broadband access across the US is associated with a reduction in COVID-19 mortality by 0.1% for every 100,000 people.
As a result, Biden’s office has launched the Internet For All Initiative which is allocating $65B from the Infrastructual Investment and Jobs Act to closing the digital divide.
The main component of this program is to direct $42.5B through individual states to build infrastructure on the unserved and the underserved. The government defines the unserved as those with download speeds of under 25mbps and upload speeds under 2mbps and the underserved as those with download speeds of under 100mbps and upload speeds under 20mbps.
The implicit goal of this program is to incentivize and subsidize the transition to high-speed broadband for black communities who have access to high-speed infrastructure but cannot afford high-speed internet. This goal comes from the statistic that exactly 25% (16.5/66 million) of the people who have access to high-speed internet but cannot afford it are black.
Although this is a major step in the right direction, like many large government spending budgets, this money along will not solve the problem due to the inevitability of a misallocation of resources and inefficiencies in coordination and implementation.
There is a large subset of people in the world who live within proximity to existing Internet infrastructure but cannot access it for two reasons:
- Slow speeds due to high Internet traffic and bandwidth misallocation
- High speed broadband plans are too expensive
Targetting these two problem areas is a means to optimize the existing infrastructure before pouring more billions into new government-funded infrastructure (especially in a macro environment where government spending is being cut)
Before diving deeper into the possible solutions, let’s understand fundamentally how the Internet works…
How the Internet works
For you to watch a movie, load a webpage, or do anything that requires an internet connection, the data you load is broken down into little data packets and transmitted across networks. These data packets turn complex information like webpages into little pieces of binary information for the computer to interpret. Then, routers and switches direct these packets to their intended destination.
Bandwidth
The amount of data transferred per second in a packet is called the bandwidth of the network. To understand this concept more completely, think of water flowing through a pipe. In this case, the water is the data and the pipe is the network. If the pipe is wider, more water can flow through per second, which indicates a higher bandwidth.
Bandwidth is “how fast” the internet connection is and is measured in megabytes per second (mbps). Bandwidth is super crucial for streaming videos or movies as data packets are continuously being send to your device to display the media. Obviously the higher the bandwidth the better. Fiber Optic connections (we will get into this later) are deemed as the fastest type of internet connection as they have the higher bandwidth.
Latency
Bandwidth is not to be conflated with latency. The latency of a network measures how long it takes for a signal to be sent back and forth from a device to the network. Latency measures “how reactive” the connection to a network is. For example, the time it takes between when you send a request for some data (clicking a link) and when you receive that data (the page opening) would be the latency of the network. For gamers, you may better understand latency by calling it ping.
Moving forward with the water analogy, the latency of the network would be how long the pipe is. For better reaction times, you’d want a shorter pipe as it takes less time for water (data packets) to flow through. Therefore, you want the lowest latency (measured in milliseconds. A low latency is crucial for gamers as new requests are made to the server every millisecond. For streaming videos, latency is only relevant when first turning the video on, and less so for watching the entire video/movie.
A use case that would need both high bandwidth and low latency would be a video call as you are both streaming AND sending requests to the server.
Wifi vs Cellular data
As end users, our perception of how these two systems work under the hood is limited as the end goal is pretty much the same. That is, we can access the Internet through both mediums. However, they have key differences to consider when looking at network optimization.
Wifi networks are relayed by routers to people. Data is transmitted to the router by an ethernet cable which runs underground in a complex network to the ISP. As a result, there is no extra cost for consumers to transmit extra data to their Wifi networks as the data flows through cables.
Cellular data, on the other hand, connects to the Internet directly (instead of going through a router) by accepting and relaying signals to antennas on cell towers. Because cellular network providers incur a cost when more data is sent through their towers, there are limits for customers that would cause them to pay extra if they use more data. Because of this design, however, 90% of the Internet population (NOT global population) have used a mobile device connected to the Internet.
What is Internet Infrastructure?
Physical Infrastructure for Wi-Fi
To understand how you can watch YouTube videos at home, there are three main components you are using: the router, the switch, and the servers.
The router is the Batman-looking box that acts as the communication bridge for different computer networks It also determines the best path for data traffic to take by using the IP addresses of different nodes (computers) on the network. Routers act as the point guard of your internet connection by directing the information to the person who requested it.
When you look up a website, say Medium.com, your computer sends a data packet to request the file. That packet travels through your router and is then routed to the modem. The modem then sends the request via the network to the router of the server of Medium.com. This router then finds the specific file or webpage you requested and sends it back to your local router, which routes it to your computer.
The modem is what takes the data packets from your network and transfers them through the Ethernet cable to electric currents so they can be transferred quickly around the internet.
At your home, you may also have a switch that is used to connect devices on a network for communication. The switch acts as the controller directing data between devices for efficient and reliable communication.
Finally, you have a physical server which is a computer that provides access to other shared devices on the network.
What are the different types of Internet connections?
You now understand from a high level what components of hardware are used to provide you with the Wi-Fi to read this article. However there is a lot of variance in the hardware between different technologies of providing Internet connectivity. Let’s dive into the main four.
- Fiber Optics
You may have seen the advertisements from the top telecoms companies about this word, but never really understand the hype. Fiber Optic Internet is the fastest means of transferring data in the world with a max download speed of 2000mbps. That being said, it is also the most expensive and least widely available around the world. As a result, Biden’s Infrastructure Bill I discussed earlier is dedicated to expanding the network of fiber optic cables around the country. Why? Because 43% of the US has access to a fiber optic connection, but only 20% use it as it is expensive.
What makes Fiber Optic internet so much more advanced is that it uses glass (or plastic) filaments that transmit data packets in light pulses through a network of cables. In this way, data packets are literally transmitted at the speed of light and put other methods of transferring data to shame. Another key feature to its popularity is that download and upload speeds are the exact same which is unheard of for other means of transferring data where upload speeds were at least 5x slower than download speeds.
2. Cable Internet
Cable Internet has been the status quo for year before fiber optics stole the spotlight. This design is known for being fast and widely available, but is limited in rural areas. The design of these coaxial cables is the same as the cables for cable TV: it is an electrical cable wrapped in a copper mesh to protect it from signal interference. These cables transmit data in electrical signals making them subject to electrical resistance, thus being less efficient and slightly slower than fiber optics. Furthermore, cable internet bandwidth is shared amongst other users, leading to network congestion at times of high usage in an area. However, because these cables have been around for so long, they are much more widely available around the world, thus making Internet connection via these cables much cheaper.
3. Digital Subscriber Line (DSL) Internet
DSL Internet is a modification of the OG form of the Internet where data is transmitted through telephone networks. As a result, no major infrastructure needs to be built as it operates of existing telephone networks already incredibly robust in rural areas. This system makes DSL the most widely availible Internet type in the world. Data is transmitted through copper wires that send data as electricity through landline phone networks. The data travels through unused frequencies in phone wires allowing it to bypass voice connection (unlike how dial-up Internet worked in the true OG days).
his design ensures data is unaffected by network congestion due to the different frequencies it operates upon so the network will always be the same speed. Unfortunately, due to its design, DSL can only reach a max network speed of 100mbps which is not enough to operate in our complex Internet today.
4. Satelite connection
Satellite is the most scalable form of Internet connection we have right now, but unfortunately, the technology is not quite ready for ubiquitous usage. Let’s take Elon Musk’s Starlink for instance. The goal of Starlink is to achieve network speeds of 10gbps (they recently 10x’d their goal from 1gbps) which would revolutionize the world by providing everyone with Internet access, regardless of their local infrastructure. But that goal is set to come when Starlink also reaches its goal of 30,000 satellites around the world (compared to 2000 right now).
What sets Starlink apart from other satellite companies is that it is building LEOs or Low Earth Orbit Satellites that are about 300 miles from the earth’s surface, compared to the normal distance of 22,000 miles from the earth’s surface. Comparatively, its download speed of 220mbps is much faster than other satellite companies, but is not comparable to the speeds of fiber optics or even coaxial cables that provide Internet today.
Ok so you understand how Internet works, and why it is slow in rural areas with little infrastructure. Let’s now explore how we can leverage Machine Learning to optimize the current infrastructure we have.
How can AI help rural Internet Access?
Artificial Intelligence can be applied to literally any industry because of how many functions it has.
We can use Machine Learning in many different ways to focus on the variety of problems native to how the Internet is spread today.
One problem we can focus on is the misplacement of telecommunication towers around different countries. By training a Machine Learning model on geospatial data from sources like arcGIS, our model can identify patterns or heat maps that reveal high densities of people with a poor network connection. This model would not only tell us where towers need to be built today, but also predict where towers need to be placed for future population trends. This software is one that can be build and used in tandem with governments and ISPs looking to spend money (like Biden’s Infra Bill) on network infrastructure.
Through the BEAD (Broadband Equity Access and Deployment) program from Biden, data is being collected on a per-block level to figure out where to place cell towers. AI is assisting this data-collection process to make informed decisions in the US.
But how can we optimize our existing infrastructure without needing billions more dollars?
This is where we use Machine learning to predict and improve network traffic patterns, thus limiting network congestion that slows speeds for many communities. By equipping a machine learning model with statistics like network logs, user behavior, and weather conditions, these models can dynamically predict future traffic patterns and reroute data packets to avoid network congestion. In this way, pipes don’t get clogged and water can continue to flow freely. With this information, algorithms can make real-time decisions as to where to send data and in what areas to allocate more bandwidth to based on higher demand. They can also use predictive demand data to identify where to provide bandwidth to in the future, creating a proactive instead of reactive network.
These same algorithms can also be used to detect anomalies like Cyber attacks (Distributed Denial of Service attacks) on certain networks.
Up until today, the monitoring of broadband networks has been limited to identifying physical cuts in fiber optic cables to send technicians to fix the cables. These are reactive instead of proactive measures. But now, these networks would predict and mitigate any anticipated issues and essentially be a “self-healing broadband network.”
With this architecture, both latency and data transfer rates can be decreased, as well as the creation of a more secure network. Furthermore, less infrastructure is needed to be built by the government to have the same impact which can save the government billions. As a result, ISPs can maximize the throughput of their existing infrastructure, thus maximizing their ROI and customer satisfaction.
A solution leveraging these technologies at scale in the United States has the potential to increase user experience by 15% and increase network reach by 20% across the nation.
Why is high-speed internet needed now more than ever?
Well, it’s pretty clear that compute power is one of the hottest commodities in the world right now to train convoluted AI models. However, right behind compute infrastructure, Internet speed is the next highest in-demand asset for the industry.
How are Cellular Companies using AI today?
As mobile traffic has consistently grown year over year, global mobile data traffic is expected to grow at a CAGR of 25–30% between now and 2027. However, although the number of subscriptions has gone up, the revenue per user has gone down each year, prompting Cellular Companies to turn to AI to help decrease their costs.
They are looking to deploy AI in a variety of ways to cut costs from maximizing ROI from existing infrastructure to reducing energy consumption by 30%, to accelerating their time to market. Vodafone, for example, has launched their Zero Touch Operation Strategy which aims to prevent 50% of their network faults through having AI “self-heal” the network.
To understand the desire for automation within the industry, we have to understand the levels defined:
- L2 — Partially autonomous networks
- L3 — Conditional autonomous networks
- L4 — High autonomous networks
- L5 — Full autonomous network
Huawei, for example, is planning to reach L4 by 2025, whereas Vodafone wants L5 by 2025.
Clearly, the desire for companies to cut their costs and increase efficiencies with AI is strong, which is why network optimization is a crucial field of research for the future of the Internet.
How can we optimize these networks?
Complex computer networks have a few ways in which they already optimize the flow of data on their networks. Here are a few of them:
- Traffic shaping: controlling the flow of data packets in a network to limit congestion and ensure prioritized data can bypass heavy network traffic. The prioritization of data on computer networks is called Quality of Service (QoS).
- Load balancing: the even distribution of traffic across multiple servers to avoid overloading and network availability
- Payload compression: the compression of data packets with an algorithm before transmission to reduce the quantity of data being transmitted
- Using an SD-WAN: A software product that gives centralized control to one entity to support the management and optimization of dispersed networks.
The main difference with this solution would be to build upon a method listed above like traffic shaping and apply an Internet demand prediction model to inform the decisions of the optimization technique.
Random Forest Regression
One way of predicting Customer bandwidth needs is to pull data from existing users of ISPs and use a Random Forest Regression model to predict their future bandwidth usage.
Randon Forest Regression builds many decision trees to make a prediction and then combines the predictions for each data point. It works well in this case as it can predict numerical data points from a large dataset. We can then plot the importance of each feature to bandwidth usage and find the top 5 metrics with the highest correlation. With this model, we can then predict the next 12 months of customer data for bandwidth demand. Other steps that can be explored are gradient boosting or deep learning.
Ways to develop the prediction model
I will probably end up using some type of neural network to forecast demand based on a variety of data points.
One simple type could be a Multilayer Perceptron (MLP) which consists of at least three layers (including one hidden layer) by which to process input signals (data points). A Perceptron, or layer, is an algorithm for supervised learning for binary classification. It works as an artificial “neuron” by which to perform computations on input data. They are coined the basic building blocks of a neural network. MLPs are good for large datasets and can handle non-linear relationships, but it may be prone to overfitting as it require many training examples.
Overfitting is when a machine learning model is too accurate for the data it is given, and not very accurate in predicting future data. Obviously, for this project, overfitting would be very undesirable.
MLPs are feedforward neural networks meaning the neural network does not form a loop as information is only passed forward. Data goes in through input nodes, through hidden layers, and out the output nodes, with no link to send information back through from the output node.
Another means to build the demand prediction model would be to use Recurrent Neural Networks (RNNs) which are designed to handle sequential data, like time series data for Natural Language Processing (NLP). This design would be very beneficial for demand prediction over a period of time as it uses patterns to predict the next likely scenario as does the neuron activity in the human brain. The concern is that they are difficult to train and require large datasets for reliable performance.
A type of RNN that could be considered is Long Short-Term Memory (LSTM) which is designed for long-term dependencies (whereas RNNs are for short-term) in sequential data. For example, you’d use RNNs for Twitter sentiment analysis and LSTMs for sentiment analysis in a long-form article. LSTM is best used with time series data.
Another consideration would be Convolutional Neural Networks (CNNs) which are designed to handle image data. CNNs are very widely used across computer vision applications but may be useful here if any geospatial data is used to detect anomalies or patterns in the correlation between certain geospatial factors and Internet demand in a region. However, CNNs are not great with time series data, so their demand predictions may not be the best, but they will be great for detecting patterns in Internet demand. CNNs are best used with spatial data like grids, satellite maps, etc.
The final algorithm to consider is an Autoencoder which is an unsupervised learning algorithm that learns to reconstruct input data. It is usually used for feature extraction, dimensionality reduction, and anomaly detection. In this case, it can be used to extract features in sequential data and use them to forecast future demand. Autoencoders can be used with both time series and spatial data as well.
Demand Prediction components needed
The key to success for this project is finding high-quality data on which to train the model. If I can gather labeled data that points me in the right direction of understanding the demand for a region, I will be able to use some deep learning models to predict the location of future demand and use that information to guide the network optimization algorithm I employ.
The next steps of this project
SEARCH FOR DATA!
As I mentioned, the data I find will dictate where I take this project next. To learn the inner workings of how this prediction model is going to work, I am going to build some projects in the smart grid space with the intention of building transferrable skills. I am choosing this project as the next step because energy grids leverage demand prediction models to inform the allocation of energy resources, and there are many parallels between the demand prediction models used in that scenario vs the model I am looking to train for this specific scenario.
Thanks for giving this a read. I hope you learned something new about the beautifully complex network (and its problems) that is allowing you to read this article right now. To stay updated with my project progress, follow this account and subscribe to my monthly newsletter where I will display project updates, unique experiences, and my reflections. Finally, feel free to reach out to me on Linkedin or Twitter (X.com) and check out my full portfolio for other projects I have worked on in the blockchain, climate tech, and now computer networking space.