Test Management for multi-vendor, multi-program Digital Transformation
If you’re a Test Manager with a task of managing testing vendors — not just one or two, but a multi-vendor, multi-program digital transformation initiative, here’s some advice.
- Draft a winning contract: If property is about Location, Location and Location, then outsourcing is about Contracts, Contracts and Contracts…
2. Do not rush finalizing contracts. Spend enough time thoroughly reading through the drafts provided by vendors and revising them until you have specified non-negotiables. Make sure that the testing aspect is taken into consideration in the contract. A fixed term contract works out to a lower price but a time and materials contract will give you more control over the project.
3. Ensure that you specify the Test Reporting tool, Test Deliverables & Metrics that you would like to see. This has to be specified clearly so every vendor in the program knows what to expect. Failing to do this, is going to cost you later on.
4. For a large scale testing program, it is recommended that the System Integrator takes ownership System Integration Testing. This will improve the chances of the system being in a better shape when it is ready for User Acceptance Testing.
5. Unit Tests: Automated Unit tests for each component — are a non-negotiable. Ensure that coding standards are followed and the developers follow templates to create their unit tests and also for their code documentation as well.
6. API Tests: This is an absolute must. If you can get these developed, it would be a great start to assuring quality of the micro-services even if the user interface is not yet ready.
7. Testing Tools: Talk to the different vendors about Test Automation tools that they will be using. If each vendor uses their own tool, you will be left with the additional overhead of converting these scripts. So deciding this upfront will save you time and you can use the automation scripts for regression tests.
8. Performance Testing: Ensure that the performance tests done by vendors takes advantage of the existing functional tests. Using a performance test tool that plugs into your functional testing tool and API test tool will save you time & money!
9. Test Automation Framework: Even if the vendors use the same automation tool, the test automation framework they use could make a world of difference. Getting all the 20 vendors to work on a common test automation framework will help churn out code that your test center of excellence team will be grateful for!
10. Legacy Systems: If there’s data migration involved, make sure you include that also in the contract. Also, you need to make sure that there is ownership assigned to make sure old data is available for access if required for audit purposes.
Set up the communication plan so everyone’s aware of what’s expected in status meetings. Create focus group across vendors to work together to solve problems. For instance, Test Reporting Focus group will have one member from all vendors and representation from the client side so if anyone has questions or concerns, they don’t have to escalate it to the Vendor Test Program Manager and bring it up to the committee. Such trivia can be handled better collaboratively. So, the Test Program Committee can focus on more weightier concerns.
So, what if all this wasn’t done and you’re just thrown into an ongoing Test Program to clean up the mess? I’ll cover that in my next article.
