Observe and report

Global Technology
McDonald’s Technical Blog
3 min readSep 26, 2023

McDonald’s has been on a journey to revolutionize its network infrastructure through network observability and AIOps.

by Saif Ahmad, Director, Global Network Platforms

In this era of digital transformation, understanding and controlling network infrastructure have become key contributors to the success of businesses.

As emerging technologies, like AI, continue to advance, there’s been a growing need for McDonald’s to develop a comprehensive visibility strategy for its network infrastructure.

Recognizing the benefits of merging network monitoring with artificial intelligence, the Global Technology Infrastructure and Operations team set out to reshape observability within our network structure to streamline and improve our monitoring capabilities — a challenge given our company’s vast size and reach.

The approach
The initial phase of this transformation entailed recognizing several challenges at hand: Using outdated, on-site solutions that are lacking intelligence and are unscalable; using systems without multivendor compatibilities prohibiting us from expanding and resulting in system-focused tools that lack a centralized view; and finally working in a reactive rather than proactive fashion.

Acknowledging the hurdles set the stage for developing a strategic approach, centered on network observability and the power of AI.

Subsequent efforts focused on creating a robust, tiered architecture for network observability, designed to provide detailed insights into every layer of the network. From the foundational network elements to the most complex, every facet of the network was brought under this observability umbrella.

With the help of industry-leading experts, we put the architectural model and its underlying strategy to the test. This affirmed that our model was on the right track, reinforcing our belief in its potential to help revolutionize McDonald’s network infrastructure.

Next, we undertook a five-month long exploration of various observability and AIOps tools. The evaluation and subsequent ongoing implementation of the best-fit tools fine-tuned the architecture, tailoring it to meet McDonald’s unique needs while ensuring robust network observability.

Benefit realization
The goal of the finalized strategy and architecture is for it to enable seamless real-time transmission of telemetry across the backend network fabric, spanning from endpoints to the observability platform. Employing robust and well-established technologies for constructing event pipelines, these capabilities not only grant a comprehensive overview of the entire network, but also function as a health gauge, offering insights into operational facets necessitating attention.

Moreover, the telemetry can be harnessed to establish benchmarks for operational performance. Any deviations from these benchmarks are pinpointed through the application of anomaly detection algorithms, allowing us to proactively address potential issues prior to their impact.

When integrated with AIOps, the data becomes actionable, yielding invaluable insights. This empowers us to identify and intercept emerging incidents in real time, thwarting their escalation into outages.

Furthermore, these platforms enable us to trace an incident back to its root cause, furnishing visual representations of incident progression and highlighting alterations in services and infrastructure that may have contributed to the incident’s occurrence. Notably, the system can trigger self-healing automation to tackle problems before they fully manifest, thus enhancing the overall customer experience.

In conclusion, the coupling of network observability and AIOps marks a significant stride in the evolution of network operations. It not only helps bring unknown aspects of the network to light, but also provides a robust foundation for self-healing and automation. This achievement is a testament to technology’s potential to drive operational efficiency and deliver superior customer experiences.

--

--