Decommissioning your LMS: the art of data extraction

Reading Time: 7 minutes

The final project at my last role involved extracting seven years of learner data for 80,000 learners from an LMS that was being decommissioned. The shock of our vendor’s £150,000 quote to both extract and host 3TB of learner data led me to explore a robotic process automation (RPA) solution using a low cost browser plugin, UIVision. This article explores how I crafted and fine-tuned a suite of RPA scripts to automate the data extraction and reflects on the challenges faced and lessons learned.

Extracting 7 years of learner data

I joined a training provider during the final year of the three year IT rationalisation and LMS migration project. The final step of this was to extract and archive seven years of learner data from the old SaaS LMS platform, in line with the government data retention policy for apprenticeship learner data. The SaaS vendor we were moving away from had a data export function, but it was limited to formal learning data (achievements and progress) and not more informal data such as learning journals and notes. Given that our tutors and coaches had been storing critical data in these areas too, we needed to export from all parts of the system. The upshot of the conversation that followed with the vendor was a request for £100,000 for them…