How do the big name companies test?

A brief insight into approaches, concepts and behaviours from the industry leading names

It’s important to have perspective. If the only reference for software testing and quality standards is your own company, you may be missing valuable insights and ideas you haven’t considered before.

I’m going to run through some of the top companies in the industry, and pick out some core values and concepts that help drive their quality processes.

For a more detailed list of companies and resources, check out this page.

Airbnb

• One approach is to rule by edict, but given our culture of engineer autonomy this would have been very poorly received. The other approach is to lead by example and build a movement.
• The bar to writing tests should be, “so low you can trip over it.”
• A way to run individual tests quickly and reliably on a dev machine, a way to run the full suite quickly and painlessly in the cloud, and a way to make sure that every PR had a passing build before it was merged.

Amazon

• Testing / quality assurance does not stop at the top of the traditional pyramid.
• Think ‘clear understanding & alignment between builders, product managers, design’, ‘continuous validation and iteration’ , ‘unit testing‘ etc. on the left and means like ‘customer feedback’, ‘monitoring’ on the right.
• There are dedicated QA people at Amazon, but they are not people you off-load testing to, but instead they act as teachers of the quality and testing skill-sets, helping the teams do what is needed, auditing and teaching.

eBay

• Quality must be a shared understanding across product, business and engineering.
• It is more important to focus on whether the customer outcome and business impact were clearly delivered upon, not just releasing bug-free code.
• Inviting engagement from all sections of the community and creating a safe environment where we acknowledge that mistakes happen.
• What started as a gating factor at the end of the line quickly evolved into transparency and more confidence with each release.

Google

• Experiment on our applications early and obtain feedback from real human beings.
• The ideal feedback loop has several properties: It’s fast, It’s reliable, It isolates failures.
• Every developer is expected to do their own testing. The job of the tester is to make sure they have the automation infrastructure and enabling processes that support this self reliance.

Microsoft

• Teams should treat test code the same way they treat product code.
• If the tests can be run in every environment from local development through production, then they will have the same reliability as the product code.
• Lower the bar for using test infrastructure to generate quality signals that can be trusted.

Slack

• Quality is a shared responsibility.
• The Quality Engineering team is focused on creating a culture of testing, increasing test coverage, and helping the company ship high-quality features faster.
• Abstract it all away — To make it easier to read, write, and maintain our end-to-end tests

Conclusion

Keep it simple. Be transparent. Move fast. — Quality is the responsibility of everyone at all levels, the more you drive these core concepts, the faster and more efficiently you will deliver.

Thanks for reading! If you enjoyed this post, follow for more content in the future.