The Changing Face of Software Testing
Automation and the Place of Manual Testing
The software testing industry has changed enormously over the last ten years. One of the most obvious changes is the growth in test automation. Whether we call them Developers in Test, Software Development Engineers in Test (SDET) or Automation Engineers, the people who write automated tests are becoming more and more important.
So what does an SDET do?
In my experience, the skill set required of an SDET is much wider than many people imagine. It goes without saying that they must be a first-class developer. Whether they’re working in Java, Ruby, C# or some other language, they must have a detailed knowledge of the language itself, its paradigms and best practices. Historically, many developers have seen SDETs as second-class coders. This is beginning to change. The ever-increasing complexity of the systems under test requires the development of highly maintainable test frameworks, that in many cases are themselves complex software systems.
The other side of the SDETs role is rooted in testing. The tester’s mindset, or the desire to ‘break stuff’, is crucial to successful automation. Not only must an SDET be a skilled software engineer, they must also have a detailed understanding of testing principles and practices, and be able to make difficult decisions about what (and how) to automate. Where teams are using Behaviour Driven Development (BDD) or Specification by Example, the SDET must also work closely with Business Analysts to define user stories, acceptance criteria and examples, and with production developers to automate those examples as the story is delivered. To be effective in these teams, SDETs must have a solid understanding of agile methods, and the people skills to work closely with business.
How does all this automation affect the role of the Manual Tester?
With the rise of automation engineering, we’ve seen a lot of manual testers move towards automation — many see it as fundamental to their future career. This has led an awful lot of testers to learn how to code, and many companies to retrain their testers as SDETs. There’s a widely held view, particularly among junior testers, that “automation is the place to be”. I don’t agree.
Not all developers have the mindset and ability to become a skilled tester — if they did then there’d be no need for software testers at all. Likewise, not all testers have the mindset and ability to become developers; and an SDET is a developer. Though the two roles share a common skill set to some extent, in particular the ‘tester’s mindset’, the SDET role requires a skill set that not all testers possess. The test analyst role also requires a skill set that not all SDETs possess.
Testing is, without a doubt, a highly technical and skilled trade. The days where junior testers laboriously executed simple test cases are long gone, and even a junior tester is now expected to possess a wide range of specialist skills.
The last five years has seen great advances in the way we deliver software products. There has been a proliferation of tools to support and speed up the development process: from frameworks like React.js, Angular and Bootstrap to Deployment tools like Docker and Heroku, they allow companies to bring new products to market more rapidly than ever (see footnote 1). With more and more companies applying lean startup thinking and agile development to reduce their time to market, the tester’s job can no longer be seen simply as the detection and reporting of defects.
It has never been possible to test exhaustively, but with release timescales shifting from months to weeks (and sometimes to days), and with automated tests providing business-critical regression and integration testing, the test analyst must consider the control of risk in a wider context: that of the business.
The ISTQB defines testing as:
The process consisting of all lifecycle activities, both static and dynamic, concerned with planning, preparation and evaluation of software products and related work products to determine that they satisfy specified requirements, to demonstrate that they are fit for purpose and to detect defects.
This defines a tester’s responsibility as ensuring a product satisfies it’s requirements. Their job is to ask “Are we building this product correctly?”. This is in contrast to the Business Analyst, whose job is to ensure those requirements are correct for the business, or to ask “Are we building the right product?”.
The changing nature of software development, in particular the adoption of agile practices, lean startup thinking and cross-functional teams has led to a blurring of lines between these roles. In many companies, BAs spend part of their time testing software, while testers spent time refining acceptance criteria and critiquing the product in terms of it’s business value. Indeed, the crossover of skills between Test Analysts and Business Analysts is so great that some companies have brought them both under the same management.
This blurring of lines between test and business analysis means that todays testers not only need all the analytical and test-focussed skills that have always been required, but also the ability to understand the wider business context that our products are developed in.
Of course, the level of technical knowledge required by test analysts has also increased. Indeed, many companies (including Google) are beginning to view testing as an Engineering practice. Their testers are called Software Test Engineers, rather than QAs or testers. Test Engineers already have to deal with a multitude of devices — far too many to include them all in comprehensive test plans. With the growth of the Internet of Things, the number of interconnected systems that will need quality control can only increase. A few years ago, a release would likely include some new features in a piece of desktop software. These days, it’s also likely to include changes to various mobile apps, internal and external APIs, web platforms and possibly apps for smart watches, cars and other devices in the home. Combine this with growing public concerns around security and privacy, and the tester’s job is quite obviously a difficult one.
The more complex our systems and products become, and the more regularly we release to production, the more important a testers’ intuition, imagination and experience become.
- As well as the proliferation of open-source frameworks and libraries, there exist a whole raft of tools that reduce the pain associated with communication, deployment and monitoring. For example, a web tech company might use GitHub to collaborate on code, Docker, DigitalOcean and Commando to deploy and scale their infrastructure, Amazon S3 for static storage and Fastly as a CDN. They might handle email with SendGrid and monitor their apps with tools like New Relic, Sentry and Segment.io. Tools like Zendesk, Intercom and Slack have radically changed the way customer support and internal communication works.
- Photo by Jeremy Kieth, CC-Attribution