From Tech Lead to Architect — Part 2: a key to successful Quality Attribute Workshop
Tips & tricks on Quality Attribute Workshop: how to conduct, what to ask your clients, and how to handle unclear answers in order to get all the needed inputs.
Dmytro Ovcharenko, Senior Solutions Architect at SoftServe DevCenter in Dnipro, continues series of blog posts about basics of IT Consulting.
In my previous article I walked you through IT consulting process. Now I`ll dwell upon one specific part of it — the Quality Attribute Workshop (QAW). I`ll shed light on what pitfalls you might come across here and my experience how to handle it.
Why you need the Quality Attribute Workshop (QAW) and what stages it consists of?
What is QAW?
In plain words it`s a meeting with stakeholders on the client side, that is conducted within Discovery or Pre-Sale session to agree on expectations from a future decision.
Why is it needed?
To determine non-functional requirements for a system.
What if you skip or fail it?
Expectations will remain at the “highly-available / highly-secure / highly-performed” level, where everyone understands this “highly” in their own way. As a result, your client may still have questions for a ready-made solution, but you will have nothing to appeal to.
Why is QAW so tricky?
Things are not always so straightforward and there can be difficulties — from the clients’ incomprehension of what they want to get, to political disputes between stakeholders, when everyone takes their stand and cannot reach a common solution. You have to work through this to get what you want from them anyway.
QAW step-by-step
Part 1 — preparation within the team
We collect material for the future brainstorm:
- case studies to work out non-functional requirements
- templates for scripts and architecture (if it is an assessment)
- concerns, constraints, etc.
Now, it takes me a few hours: an hour to prepare general scenarios without measurements and about an hour to detail and refine those scenarios with the team.
Part 2 — work with clients
We usually go to face-to-face meetings. Now, under quarantine and closed borders, we work online, but the principle remains the same.
I strongly recommend gathering all stakeholders on the client side before the main sessions for a short session (not more than half an hour for them not to get too bored) to explain what we will do and why.
Then you can proceed to the main part of the QAW, which consists of two stages:
- Brainstorming to generate scenarios
- Scenario prioritization session
That was a bit of theory, now let`s get down to real life.
Hands-on advise to make the most out of QAW
Build the discussion around clients’ use cases
The more specific you are while explaining your ideas the better.
Too generic explanation — “We plan to make system A that will respond to system B in a certain way; the data loss will not exceed 0.001%”
Much more clear and closer to the client`s reality explanation — “The user needs to log in to the system to be able to generate a report. This critical feature is always available (in 99.95% of cases), while the system chat can be turned off for a while, which is not critical.”
This is how we move script by script: we discuss, listen to what stakeholders think, and let everyone speak without criticizing.
Agree on measurements
Ask about clear indicators or ranges for each scenario. Be ready, that clients may not know what they want. Yet it`s crucial to get a precise answer to secure yourself.
I suggest taking a proactive position and advise something, referring to your experience or the world’s best practices. If that doesn’t work, you can go for small provocations. For example, you discuss the latency of a SaaS healthcare solution. The client does not know what it should be. Try to offer 10 seconds — it will make them think faster and offer better options. And eventually you`ll agree on some compromise number.
Use this approach to go through all your scenarios.
Remember prioritization
If prioritization is not worked through, all previous steps will lose their meaning because architecture is built on the understanding of priorities. For example, if the most important quality of the system is security, then we are likely to lose performance. Such things need to be considered and clarified right away.
Stakeholders on the client side can start disputes, as everyone loves to fiercely defend their interests.
What helps me in such situations is giving each participant a certain number of points to distribute between the scenarios. This method clearly shows what prevails.
Track time
Determine in advance the needed time to discuss each topic, state it in the meeting agenda, and adhere to this timing to go through all the mentioned steps and find out what is truly essential.
If stakeholders have an hour for everything, we can briefly run through the main points that will help understand the complexity of future infrastructure, deployment, and the general course of the client’s thoughts. It’s important to clarify the security and performance requirements of a system. Depending on the context, there may also be other requirements, such as sustainability or usability (for web application).
Here, we clearly will have no time to go script after script, so I prepare a quality attribute script in advance, leave the question on the measurements so that we brainstorm specifically them, and not the scripts. Of course, even in this case, there may not be enough time to discuss everything. So, define the most important points for yourself, clarify them at first, and agree to finish the rest in correspondence.
As a result of all the steps described above, we form a Utility Tree
The name is not obvious, but it is a table where we describe all our scenarios and define for each of them the level of Importance and Complexity. Risks, assumptions, etc. can also be described there.
Medium and High priorities are key for us. We will build a design and achieve measurements primarily for these scenarios. The combination of L-H indicates a problem, that we always try to investigate and take into account at the design stage.
QA Scenario refinement
The last step, when we refine the requirements, their content, and bring clarity. If you have more questions, it is better to ask for another session to find out everything before the work starts.
Here is an example of a detailed scenario description. I usually do a shorter version.
In most cases, we send this document to all stakeholders and ask for confirmation. I remember only one case when we had to continue the discussion for about two more hours struggling with prioritization.
How to distribute team roles for QAW?
As you can see, QAW is a rather complex process that requires a lot of consideration: from the proper organization of work with stakeholders to finding out all necessary technical details.
To make your work more convenient, I advise to clearly outline the roles.
Facilitator moderates discussions:
- communicates with the client
- holds attention of an audience
- clearly follows an agenda
- resolves controversies during any conversation
- conducts small talks and maintain a friendly atmosphere (it`s important!)
Business Analyst is an excellent fit for this task.
Question-keeper:
- asks questions on the technical part, risks, and other relevant topics
- helps the facilitator to direct the conversation in the right direction.
It is an expert in a particular technology.
Scrubber writes down everything that is voiced during the discussions: requirements, risks, assumptions.
Based on these records, non-functional requirements will be further elaborated, so this role is very important.
Project Manager can handle this role perfectly, if a team has one. I am often a scrubber myself. In this role, I ask additional questions and watch the reaction of stakeholders, which is very useful.
What recording tools to use?
MS Teams + SharePoint is a nice option.
Files and tasks are stored in one place, and it is convenient to share materials and build team interaction.
Business Analysts often use XMind.
I also have a Word template that I can easily fill out.
But be careful with using a board: it ends quickly, you need to write legibly, take pictures of all records, and then transfer them to electronic form.
Some useful sources:
Quality Attribute Workshop Collection
How to Clarify Architectural Requirements to Ensure Successful Development
Guide on QAW by Carnegie Mellon Software Engineering Institute
Feel like this is something you`d like to try? Don`t hesitate.
