5 Steps to Build High Quality Products at Scale

How Userlane scaled from 1 to 4 Engineering Teams while keeping Quality up

Photo by Agence Olloweb on Unsplash

Many companies aspire to “build the best product”, but that’s easier said than done, and that we at Userlane have scaled from 1 to 4 engineering teams and in the meantime maintained our high quality standards, and we’ll here share how that worked for us.

Creating high-quality software is no “Rocket Science”, all you need to do is to implement effective QA Techniques, use the right kind of tools suited to your quality requirements, hire engineers passionate about quality, train your resources, plan for it, learn from mistakes, document your learnings and manage all of these by keeping a good note of risks involved and don’t forget to have fun while doing it.

Sounds simple, isn’t it? Let’s see with a few examples how we do that at Userlane.

Step 1: Perform a Quality Gap Analysis

Gap analysis is an investigation of the space between two states. The initial one is the present system state. The latter one is the state we expect in the future. At Userlane, even before the QA Team was established, Testing was still administered at different levels, like Automated testing by Developers, Functional Testing by Product Owners, Feature Testing by Customer Support, etc (in a startup environment, it’s normal for employees to wear many hats on the job and the same was the case when we were an early-stage startup where everyone used to wear multiple hats). But then we decided to perform a Gap Analysis to learn where we were and where we wanted to be in terms of Quality and what steps could be taken to cease that gap.

Step 2: Develop a Quality Administration Plan

A quality administration plan is a document that characterizes an acceptable level of product quality and depicts how the undertaking will accomplish this level. It’s anything but a mandatory document, however, at Userlane we created it as it assisted us to schedule all the tasks needed to make sure that the quality we offered, lived up to our client’s requirements and expectations. Following are the key components of the quality administration plan:

• Quality objectives/standards
• Key test deliverables
• QA activities and processes
• Quality roles and responsibilities
• Quality tools
• Plan for test management and reporting

Step 3: Plan the Tests

The test plan document is a living and breathing artifact — it is dynamic in the sense that it should always be up to date. So, when the project execution has changed, the test plan should also be updated. At Userlane we follow 5 basic principles while developing any test plan:

1. Keep it concise: A short and concise test plan is way more effective and easier to read. There is no fixed length for a test plan — it can vary by team size and project. More complex projects may have longer test plans than simpler projects. A super long test plan may lose the essence of test planning and is more likely to be ignored.
2. Keep it organized: A test plan is often checked for some specific information, so it is very important to keep the information organized, therefore the teammates or stakeholders can easily find what they’re looking for.
3. Make it easy to understand: Similar to keeping the test plan organized, you need to make it easy to understand. Keep the language simple so that the test plan can be read and understood by anyone.
4. Make it easy to update: Plans change. A project is subjected to change at some point, and the test plan document should be updated in order to remain relevant. Maintain the versioning for test plans and write/store the test plan in a way that’s easy to update.
5. Share it with your stakeholders. It will give them information about your testing plan and processes. The quality of your test plan gives a perspective about the quality of the testing your team will perform.

After several weeks of Research & Development, testing a number of releases and collecting feedbacks from various stakeholders, we at Userlane have developed our own format for Test Plan document but Test plans need not be done a certain way, but if you’re new to writing test plans, this template can be a good place to start. It covers:

• Test Plan Identifier: Create a unique number to identify the plan for a project or a release.
• References: A list of documents that has to be referred while developing the test plan. These might include a project implementation plan, functional specs, or design prototypes.
• Introduction: A summary of the test plan, including the purpose of the testing project and scope.
• Test Items: The systems and subsystems which will be covered under testing.
• Items To Be Tested: The features that will be tested within the systems/subsystems during the testing project.
• Items Not To Be Tested: A list of features that requires no testing effort.
• Approach: The overall test approach i.e. how the tests will be performed.
• Pass/Fail Criteria: Criteria for the test (minimum number of defects, percentage of passed tests to mark a test as complete.
• Suspension Criteria: Specify when and under what conditions to pause the test, something like a certain number of defects.
• Test Deliverables: The artifacts developed and maintained by the QA team that is to be delivered upon completion of the test. These could be test cases, test data, outputs, list of bugs found during testing, test summary or test report, QA sign off etc.
• Testing Tasks: List of all testing activities like Manual, Automated or Non-Functional testing ( load, performance etc.) that must be performed during the QA cycle.
• Environmental Needs: Any specific tools or hardware that is required for the tests.
• Responsibilities: Which resource will perform what task etc. This may include testing responsibilities, leading the team of testers, managing the entire QA team etc.
• Staffing And Training Needs: What resources are needed? Does anyone need some kind of special training in order to conduct the test?
• Schedule: Details related to the testing timelines and when the testing will start/end.
• Risks And Contingencies: The overall risk of the project as it moves into the testing phase. Something like — last moment requirement changes, lack of resources or delayed delivery of the software from the development team.
• Approvals: Who provides the QA sign off on the testing project and approves it to proceed to the next step?

Step 4: Shift Left — Test Early & Test Often

A “shift-left” testing approach mirrors test, early test often thought and proposes leading testing exercises from the earliest starting point of the development process instead of making it a final step as conventional techniques regularly recommend. Initially, at Userlane, we mostly used to perform testing towards the end of the development cycle, gradually we shifted left and now we start to test early and keep testing continuously throughout the entire development cycle. This is how we perform it at Userlane:

• Planning tests early. Pushing testing techniques off until the last week can create bottlenecks and setback progress. Thus, consider arranging a test plan from the beginning phases of the software development to detect and fix the defects as soon as possible.
• Reviewing requirements with stakeholders. The shift left idea doesn’t really draw all testing exercises nearer to the start of the cycle. It can mean engaging testers in correspondence with clients and other stakeholders with the intent of reviewing and examining the requirements.
• Testing often. Doing smaller tests more oftentimes all through the development stages and making a ceaseless feedback stream allows for prompt validation and improvement of the system.
• Automating tests. Implementing automated tests sooner rather than later and increasing test coverage would also expedite and improve the testing process. Test automation can be performed at several levels, at Userlane we have test automation at Unit, Integration, System, End-to-End, Functional or Non-functional levels.
• Prevention instead of reaction. Shifting left can zero in on defect prevention rather than reacting to bug reports by the users. For example, testers can always pair with engineers and contribute to the development process or add new tests, fix the broken tests prior to hitting the build. Also, testers can provide fast inputs to influence development decisions.
• Team collaboration. Quality is a Team responsibility! Engineers know that each development decision is a quality decision too. Software quality does not depend on the software development model, instead it depends on the method how we all together as a team structure the whole software development process. Team collaboration, with testability in mind is a must for achieving highest quality standards.

Step 5: Document to Improve the QA practices

It is a very common observation that projects that have all the documents in place have a high level of maturity as compared to the un-documented project. Still, there are a few organizations that pay little attention to documentation.

Documentation can help to save time, cost, effort and make the entire QA process transparent and systematic.

At Userlane we document with a simple goal in our mind: Documentation should not become an overhead and rather be a value add. The following are a few tips to improve your documentation coverage and benefit from it:

• QA processes should be documented such that they are repeatable, and are not having any dependencies on any QA resources.
• Document the learnings so that the same mistake can be avoided in the future.
• Use standard templates for all kinds of documentation such as Test Plans, Test cases, Test data, Bug reports, Root Cause Analysis etc.
• QA should involve in the very first phase of the project so that QA and Documentation go hand in hand.
• Don’t just create and leave the document, Update as and when required.
• Use version control and it will help you to manage and track your documents easily.
• Store all project-related documents at a single location, accessible to every team member or stakeholder for reference as well to update whenever required.

While Software companies can’t foresee all potential bugs in software, a comprehensive QA approach limits the danger of fostering a defective product. The above mentioned approach worked for us, and may be it (as it is or with some modifications) can work for you too!