Hacking up a reporting pipeline using Python and Excel

When I started as an Analytics Manager in an E-com company, the reporting process was manual, prone to errors and was taking over a full day to consolidate. We were looking to scale up and continuing the way it was wouldn’t allow us to do that effectively. We needed a solution that would be provide a higher degree of automation.

Data Sources

Our reporting process involved extracting order files out of 7 different shops, getting multiple extracts from google analytics for each of these shop, extracting information related to our marketplaces activities on Amazon, Groupon and bol.com, getting spend metrics from more than 8 different advertising channels.

For some of the datasources, we had access to the API, other only a reporting interface that could export to CSV or excel. This was due to some of our operations being outsourced to different agencies at that moment, but we were looking to insource as much of these operations as possible.

Selecting Python & Excel

Given that our business was quickly evolving, we needed a solution that would cater to integrate a…