My challenge of porting a production distributed agent software from Python 2.x to Python 3.x

Base story

The challenge was to fully convert the code base of an autonomous agent software which runs on Ubuntu OS. This agent is initially written in Python 2.7 and runs in over 10,000+ remote devices which runs on either Ubuntu 14.04 or 18.04.

Why we need this ???

Since Python 2 is getting deprecated and community support will eventually be dropped and as the production codebases should not contain deprecated technologies we took the decision of this migration.

Challenges

1. Find the best suitable way to convert the whole codebase to Python 3
2. Thoroughly test the agent in all aspects and make sure this migration doesn’t introduce bugs and anomalous behaviors. (Criticality is that if I mess this, all these 10,000+ devices will be unreachable. (Think it like you sent an autonomous spaceship and you lost all the communications and lost the connection to Earth))
3. Finding how to upgrade these agents in 10,000+ remote devices smoothly without hindering customers' production work.

Since this is a bit longer I’ll divide this into 2 series.

Series 1: How to convert a codebase from Python 2 to Python 3

Series 2: How the upgrade happened remotely to all those Ubuntu devices