Soaring in Data: Exploring the Airline Market in Colombia
First Flight: A Personal Journey into Aviation Data
Do you remember your first time flying on a plane? For me, I was about eight and I felt a mix of adrenaline and fear, seeing those metal birds up close was a total different thing that seeing them soaring in the skies, and being inside one was a total different story. I also recall the delightful snacks they used to serve onboard, I loved those.
Ever since the Wright brothers decided wings were better than legs for travel, airplanes have become the primary means of long-distance transportation. They’re fast, comfortable, and offer breathtaking views from high up in the sky. Being naturally curious, I decided to dive deep — or soar high — into how this market operates in Colombia, my homeland.
Where is the Data?
Like the “new oil” of this century, the project had to start by sourcing data. I web-scraped traffic data from the last ten years from Colombia’s Special Administrative Unit of Civil Aeronautics (Aerocivil), which contains monthly aggregated air traffic records for the entire country.
The Cleanup Crew: Polishing Ten Years of Data
Since my undergraduate years, I often heard that data scientists spend only 10% of their time building models, another 10% talking to stakeholders while the other 80% is spent cleaning data. With each project, I find this to be increasingly accurate. Working with ten years of data and several administrative changes meant that data formats varied widely. We had to tackle these issues with powerful tools, cleansing the data of null values and standardizing. Issues like schema differences and datatype inconsistencies were carefully addressed using the polars library, resulting in a tidy dataset.
All clean now what?
After the cleaning process, I wanted to host the data in the cloud. Even though 1 million rows may not sound like much in the grand scheme of things, it could slow down analytics workflows. I chose to host my data on AWS, providing secure and convenient access wherever needed. With my data securely stored in the cloud, it was time to initiate my analysis.
Sigma Takes Flight: Analyzing the Skies
As I was choosing which tools to use from the plethora of options available today, I came across Sigma, a web-based solution for data visualization that appears to be a strong contender against more established tools like Tableau and Power BI. What stood out to me about Sigma is that it meets three crucial needs:
- Data Uniformity: Your data is stored in a single, centralized location, making it accessible and consistent.
- Data Security: Centralizing data ensures uniform security protocols, benefiting from the layers of protection offered by your data warehouse.
- Simplicity: Sigma’s spreadsheet-like user interface ensures that if you’re familiar with Excel, you’ll find Sigma intuitive and easy to use.
What makes Sigma particularly powerful is its accessibility. If I previously spent around 10% of my time building dashboards after cleaning data, with Sigma, it felt like just 5%. The streamlined and familiar UI meant I could connect to my data source, edit tables, aggregate data using formulas, and create visualizations quickly — all with an experience reminiscent of working in Excel.
Here are some of the analyses I performed, all done simply by connecting to my data source and editing tables, aggregating data using formulas, and adding visualizations, just like in Excel:
In 2018, the market saw sustained growth, signaling a stable travel industry. However, as expected, the drastic drop in flights due to the COVID-19 pandemic is starkly evident. The most pronounced peak occurred just before airport closures, as people scrambled to get home or reach safer destinations amidst a global crisis. Looking into 2023, we observe a steady recovery, hinting at gradual growth and renewed optimism in the Colombian airline market.
This analysis confirms that approximately 75% of the market share is concentrated within five companies, with Avianca reigning as the undisputed leader. This finding raises critical questions about the market’s competitive structure. Is it verging on a monopoly? Are the regulatory frameworks robust enough to protect the interests of consumers? Such a high concentration in few hands may stifle competition, potentially leading to higher prices and less innovation.
Interestingly, while most air traffic is domestic, a significant 33% of flights are international, underscoring Colombia’s appeal and connectivity on the global stage. This balance between domestic and international routes highlights the strategic importance of Colombian airports as hubs in Latin America.
It’s no surprise that Bogotá is both the most common origin and destination for flights in Colombia. Home to Aeropuerto El Dorado, the country’s main airport and one of the busiest by passenger volume in South America, Bogotá serves as a crucial node in the network. This centralization impacts not only the distribution of air traffic but also economic activities in the region.
Final Approach: Navigating Through Clouds and Data
As we gently descend from this data-driven flight over Colombia’s skies, it’s clear that the landscape of air travel is as intricate as it is fascinating. From the nostalgic crunch of in-flight snacks to the sleek dashboards of modern data analysis, our journey through the airline industry reveals not just numbers, but stories. Stories of rapid ascents, unexpected turbulence, and hopeful recoveries.
Our exploration has shown how vital a clear view from above — the kind you get from a well-analyzed dataset — can be in navigating the complexities of market dynamics and regulatory environments. Whether you’re a seasoned data pilot or a curious onlooker with your tray table down, there’s a wealth of insights to be gleaned from looking closer at the numbers that crisscross the skies.
So, as we touch down, I invite you to unbuckle your seatbelt and explore the data yourself through the dashboard linked below. Dive into the project details on my GitHub, and don’t forget to follow me for more high-flying data stories, brought to you from the city of Eternal Spring.
Until our next takeoff, keep your eyes on the skies — and the data!
