Can Teams Successfully Automate Testing of Third-Party Applications?
While this may sound daunting, I am here to tell you it’s worth the effort. At ITHAKA, we took the life-changing step of improving our QA for our product and app/services teams through automated testing and continuous improvement over the past two years, but more recently we turned our eye to our business applications as well, including third-party software. We were able to automate successfully, and you can too! With this post, I chronicle at a high-level our work to improve the QA effort with the Customer Relationship Management (CRM) Services team and highlight our approaches to improvement and better software.
The Dreaded Manual Process
In late 2014, our organization completed a migration from a CRM application called Talisma to a new application called SugarCRM. We chose SugarCRM because it allows us to meet the needs of sales, marketing, and other teams through customized control while still giving us the ability to leverage upgrades and new features as they’re released by the SugarCRM product team.
After the initial migration to SugarCRM where QA was completed by an implementation partner, our in-house CRM Services team took over development and customization. Being able to start delivering user-requested updates on a weekly basis was great, but the lack of dedicated testing on these changes was apparent in the number of bugs being found by users.
Post-Release Challenges
After months of being “live” on SugarCRM, the team planned its first major application upgrade, upgrading our SugarCRM base software version from 7.2 to 7.6. This would mean a full testing cycle for existing functionality, customizations and various 3rd party and custom-built integrations to the CRM application.
It became obvious that with this implementation and its many customizations, a manual regression cycle would be very slow and cumbersome. The version upgrade ultimately required weeks of testing in the test environment and a weekend of testing after releasing to the production environment.
Because of this frustration, the team put their heads together and began building an automated regression suite—which would reduce the time taken to test for regression, as well as propel the team toward embracing continuous integration. Creating automated test suites for an application that is already complete and in use is a time-consuming and challenging task as is, but doing so while maintaining regular release cycles makes it even more so.
Therefore, a plan was needed…
Our Plan
SugarCRM is an on-demand software application, centrally hosted and managed in the Amazon Web Services (AWS) cloud by ITHAKA. This meant the testing approach would need to focus on the customizations built into this implementation and ensure that it could be tested in the AWS environment. There were no other SugarCRM Implementations using this environment at the time, and we didn’t want to repeat the testing effort already done by the SugarCRM product team on the stock application.
The fact that the application was already deployed and in active use had the potential to complicate the automation process. Therefore the approach had to be simple:
- The team made a list of the modules that included customizations.
- We prioritized those modules based on usage.
- Automate, automate, automate!
These initial steps provided the CRM team with a springboard for an effort that would take just a few months, ultimately resulting in the creation of automated regression and smoke test suites in Cucumber to cover the core functionality of the application. This effort has greatly helped the team find defects in the test environment instead of pushing them to production for our users to find. Additionally, test cycle time has been cut by more than half.
The team is on track to fully leverage ITHAKA’s continuous integration pipeline in the months ahead, which will allow them to deploy individual user-requested changes — perhaps even the same day! It has been exciting to see our team make such fantastic progress due to our full commitment to continuous improvement.
I can now say that yes — it is most definitely possible to automate testing of third-party application customizations. Of course there were challenges along the way, but together we worked through them and have multiple suites in place that run without error. Our lives are much easier now, as are the lives of the applications users; and we implemented the automation without negatively impacting our other commitments.
It is all possible when you have a plan and a group of people that want to make things better! For more, read our post on how to engage your entire team ensuring quality.
