Self-Perfecting Networks
By: Vinayak Sharma, Praveen Mada, Pierce Lovesee, Matthew Reed, Lauren Mieczkowski, Jingda Xu, David Cherney, Aakash Agrawal, Members of Scientific staff, DISH Wireless
In our previous blog post Outsourcing Understanding of Telco to AI, we unveiled the concept of FONPR (First Open Network Pattern Reactor) and underscored its vital role in addressing immediate challenges within telecom networks. In this post, we’ll demonstrate how the FONPR agent acts as the brain of our self-perfecting network and examine what such a network is capable of achieving in the telecommunications industry. Before delving into the architecture, let’s take a moment to revisit what differentiates DISH’ s 5G network and the concept of self-perfecting networks — setting the stage for a closer examination of its inner workings.
The DISH 5G Network
DISH has spent three years meticulously architecting, building and now expanding its greenfield 5G network. But it’s not just any network; it’s an ecosystem built on cloud-native principles, designed for and by developers. Powered by Kubernetes, an extensible open-source platform for managing containerized workloads and services, this system reflects DISH’s commitment to cloud-native principles. Notably, all underlying network functions seamlessly run in the cloud, signifying a substantial departure from the traditional telco approach.
DISH’s Guiding Principles: Open, Secure, Flexible
DISH’s network is anchored in three key principles: openness, security, and flexibility. These principles are not just buzzwords but are embedded in the very architecture of its network. This approach ensures that the network is not only robust and resilient but also adaptable to the ever-evolving landscape of digital connectivity.
Challenge — How to Scale Intelligently ?
Despite having a revolutionary approach, we encountered two major challenges: scaling the network to support a wide array of use cases, and maintaining this ever-evolving network with limited resources.
Solution — The Concept of Self-Perfecting Networks
A self-perfecting network is a network that learns, adapts, and perfects itself over time. By seamlessly integrating AI into the framework of the 5G network, our team is establishing a new benchmark for what telecommunications networks can achieve.
Machine Learning and Decision Making
To tackle the challenges mentioned, our team employed reinforcement learning to train machines to autonomously react to the patterns identified by the autoencoders. This involves creating an agent (FONPR — First Open Network Pattern Reactor) that learns the optimal behavior to maximize rewards over time. This agent navigates through the network environment, refining its understanding through trial and error, and is steered by a reward mechanism.
FONPR and the Reward Mechanism — The Brains Behind The Operation
FONPR’s training involves understanding the environment, taking action, and receiving rewards based on performed actions. The reward mechanism is a guiding compass, ensuring that FONPR’s responses align with key business metrics: service continuity and profitability.
Introducing NAPP — Network as an APP
A key component in enabling self-perfecting networks involves the deployment of the entire 5G core network as an application (NAPP) for simulations and testing. Our implementation utilizes Open5GS, an open-source framework for the 5G core network, enabling flexibility and adaptability. To simulate user equipment (UE) and gNodeB interactions and test the network under various scenarios, we employed UERANSIM. This comprehensive approach ensures a dynamic testing environment, allowing us to fine-tune our FONPR agent for real-world efficiency and adaptability.
Reinforcement Learning agent based closed-loop control system
In its simplest form, a closed-loop control system looks like the image below. We have an environment and an agent that is continuously observing the state of the environment and takes actions based on the policies defined. Now in our case the environment is NAPP, which is the simulated 5g core environment and FONPR is the Reinforcement Learning agent.
Self-Perfecting Network Architecture
Adhering to cloud-native principles, our architecture embraces a microservices approach, seamlessly orchestrated by Kubernetes, housing the NAPP and FONPR components. The entirety of our codebase resides within open-source GitHub repositories.
First comes the simulated environment, NAPP (Network as an APP), which is organized into two namespaces: openverso and m-and-m. Openverso hosts the 5G core (Open5GS) and UERANSIM, while M-and-M oversees management and monitoring. Management tasks are handled by Flux, which facilitates Kubernetes cluster reconciliation based on configuration changes to helm charts in the Git repository. Simultaneously, the monitoring component encompasses our logging and metric infrastructure, capturing near real-time observations and events from the network functions running in the cluster. Our commitment to modularity facilitates reusability, customization, and a lightweight footprint.
To tailor our metrics precisely and maintain cloud-agnosticism, we leverage the Prometheus and Loki stack (Fluentbit, Loki, and Grafana). This choice grants us the flexibility to adapt metrics as needed.
With the logs collected, our DQN (Deep Q Network) Reinforcement Learning (RL) agent deployed in the FONPR namespace queries this data, executing actions based on the predefined policies. This modular and responsive architecture ensures adaptability, ease of customization, and efficient use of resources.
Advantages of the RL-based closed-loop system
With in-the-loop DRL agents, networks have the opportunity to be constantly self assessing and self optimizing. This enables higher efficiency in resource utilization and cost, more consistent QoS delivery, and greater differentiation among customers, bringing the grand promises of 5G within reach. Best of all, these systems naturally learn from and explore their environments themselves, always seeking the best way to effectively serve both customers and the business.
What’s Next
FONPR is constantly observing the network and its state currently comprises only three network related metrics. Our next step is to increase the number of metrics which FONPR can observe and make smart actions keeping the network healthy and always available.
The Vision of Self-Perfecting Networks
The ultimate dream in this space is a network of FONPR agents, each learning, communicating and reacting to patterns within the network. This vision of a self-perfecting network is not just about maintaining connectivity but pushing the boundaries of what’s possible in telecommunications. As we look to the future, our approach to integrating AI with 5G is not just a step forward; it’s a leap into a new realm of possibilities in connectivity. Stay tuned to see how this journey unfolds and reshapes our digital world.
