Continuous Integration and Test Automation: Let’s Get Started
Several of us from ITHAKA recently attended Agile & Beyond, a conference all about agile principles and practices. I gave a talk about how we transformed our QA process from a manual test shop to fully automated test coverage, focusing on the lessons we learned throughout the journey. If you want to step into the world of Continuous Testing, here’s a guide to jump start your efforts.
Define and share your quality strategy: Where do you want to go and why? Define it and share it with the leadership and the teams. Create partnership!
Select tools that fit your requirement: What does your application look like? What would automation for it look like? Pick your tools accordingly. Don’t be afraid to try some out before finalizing it. Behavior-driven Development was our approach to aid continuous testing and Cucumber(Gherkin)/Ruby perfectly supported that.
Create a dedicated task force: Engage early adopters! Try out the tools you pick. Build the testing pipeline. See if it works for you in practical/live environment. Fail fast, learn faster!
Set coding standards: Developers would vouch for this! So should we! After all, that’s where we’re headed. Partner with developers to understand the best practices.
Write smaller user stories: Smaller the stories, faster the process. Small and quick deployments are the key to continuous delivery, and smaller user stories do the same for shorter testing cycles. Quick and focused!
Yes, you can have too much test coverage: Should every single thing in your application be automated? Should every possible workflow be automated? We’re learned that the answer to that is, no. We need to constantly ask ourselves while automating. Every story, every functionality. Is this workflow an edge case? Can I combine multiple tests? Is this just a data variation? Most importantly, should this be a unit test? Engage with your developers. Coverage is also important when we talk about total run time of a test suite. Optimize your tests, refactor tests/code when you learn more. Once your test suites are so fast, you get tempted to use them for monitoring your applications. Don’t fall for it. There are some very good tools like AppDynamics and New Relic developed specifically for monitoring.
Develop team culture: Share your goals, share your responsibilities, share what you are learning. Don’t reinvent the wheel. Lessons are the best when someone learns from them. Develop a shared learning environment, support each other to stay focused on the goal.
Retrospect: Not only after, but during the transition. What did we learn? Can we share what we learned with others? What should we keep doing? Which tools worked? Which site has helpful documentation for reference? Share, learn, iterate, march forward!
From 3–6 weeks of testing cycle time to 20–60 mins is what our journey from manual test shop to fully automated test coverage looked like. Was it easy? No. But I’m not here to say it’s easy, I’m here to say that it’s worth it. We went from being able to deploy one release per month to 250+ deployments per week. From 2:1 developer/tester ratio to 4:1.
In today’s world of continuous-everything, has there ever been a better time to get started? Maybe this post is the stimulus you need. We love to learn, and we learn faster when we engage others. If you have questions or lessons from your own work, drop a comment below!
