My Experience as a Quality Engineering Manager — 5 takeaways that may give you an edge in 2021.
As with anything we’ll need a bit of context: I am a graduate of what I have found to be two extremely complimentary fields of study, Business Administration and Computer Science. I have nearly 7 years experience in Software development, the most recent of which I spent as Manager, Quality Engineering. My story starts as a Jr. Software Development Engineer in Test and ends with Director level responsibility on several industry leading projects of immense scale and complexity. I have been relocated internationally to engage a flagship product for an organization with a valuation in the billions. I have built and managed internationally distributed teams and in the course of my work I have been deeply involved in up stream engineering process.
Very early on in my career I became acutely aware of a lack of expertise and leadership with regard to quality; I found a niche, an underserved need that I was able to capitalize on and in doing so was able ascend in my career quite quickly, gaining some valuable insight in the process.
With that I’d like to share what I feel are general takeaways, for anyone who finds themselves faced with a challenging new project or taking on responsibilities that, surely, include quality.
- Leverage institutional insight.
- Become data driven.
- Recognize constraints as choices.
- Increase ownership.
- Be a storyteller.
Now with regard to how you have found yourself reading this and how this may relate to you: Have you been promoted to leadership? Started with a new employer? Perhaps presented with a new and unique challenge? If so you’ve likely earned your stripes and are already uniquely aware of many quality pitfalls that paralyze organizations large and small; that being said — assumptions kill progress. It’s also true that no matter your experience you will likely lack “buy in” to what are essentially unfounded conclusions. The trust of the entire organization is incredibly important when it comes to getting things done in the name of quality, so where do we start?
Leverage Institutional Insight
My advise is to spend as much time as possible, with as many stakeholders as possible. We need to understand the challenges your team faces, this means speaking to the people doing the work.
SCRUM.org describes what I refer to as stakeholders as “… Those performing the work as well as those receiving the work”
You’d be surprised how many Managers and Directors I have seen enter the picture only to ‘move the furniture around’ without properly consulting the developers doing the work to identify what many understand to be root cause — my experience has been consistent, restaffing is more often due to a failure in leadership and only creates the perception of change.
Now keep in mind, during your ‘stakeholder interviews’ you will get a lot of opinions from assets without in-depth knowledge of quality practices or engineering process — don’t be led astray but keep in mind these “complaints” are not entirely unfounded. Each stakeholders perception of the situation is based largely on their unique challenges but remain based in reality; empathy and recognition alone will get you a long way with stakeholders, ultimately this will allow for a broad understanding of the issue at hand, a feel for team morale and the level of trust in, and across, teams.
I’ll give you a common example, Product Owners may be disappointed, frustrated and frankly confused with regard to an inability to release quality fixes and features on time and in alignment with their roadmap. Promises are made and they work extremely hard with development to express a need along with priority and to develop a realistic estimate of completion, at the end of day roadmaps are relied upon for all manners of business function.
The importance of predictability is HUGELY understated in Engineering — a tendency to over promise and under deliver remains a pervasive reality. Trust and transparency are absolute requirements in a functional work environment.
Now, being that testing is often the last step prior to release and unfortunately quality is normally understood as a separate process from development, product owners may see delayed releases or uncaught bugs as a failure of the Quality Organization, and who can blame them? I will get into why this is misdirected and how the function of the Quality Org is misunderstood later, this is simply an example, and a good one; most stakeholders don’t know the root cause or the solution, if hey did they wouldn’t need you! However, this does indicate there is a problem and, luckily, a solution!
Like anything else, orientation and hypothesis are essential. So where do we go from here now that you have a wealth of insight from all over the organization? Simply put, you need to organize and document your findings; for this you will want to focus on a short list of issues. At this stage (granted you have experience in the field) you should have a strong idea of areas on which you want to focus. I’ll give you a few examples:
- Misuse or under leveraging of tools
- Misunderstanding of the role of QE
- Poor planning practices
- A lack of, or ‘soft’, quality gates
- Unclear “Definition of Done"
Become Data Driven
Now that you have some areas of focus, onto our next task — collect data to guide our root analysis and, where we have strong qualitative direction from our interviews, prove or disprove root cause.
Contrary to what you may expect, data is easy to find, data is everywhere. Your team very likely leverages tools like Jira for issue tracking. If your team is making any effort whatsoever to leverage tooling properly you should have data points including severity of bugs, escaped defects, test coverage and so on. You should also be able to track burn-down, actual vs. estimate, completion rates and a number of other KPIs; keep in mind that Quality and Quality outcomes are a result of every input, at every stage of development — development data is Quality data, and we can use it.
Jira filters and Jira APIs (Zapi) provide an even more robust opportunity to pull data, these features have their equivalent in competitor products and of course this can be done by dynamically exporting directly into tools like Excel.
To dynamically import data from Jira filters into Excel perform the following:
- Login to Jira
- Create and save a JQL filter a filtered within the my project.
- Find the ‘export’ option, choose CSV and right-click (current fields), copy or inspect and copy csv URL.
- Open excel workbook, select get data from the Data tab, select from web optionPaste the csv URL into the web address box, click Ok
- You can refresh data dynamically by selecting Data tab, Queries & Connections group, click Refresh All
I strongly recommend that project data is organized by module or component; Complexity should always be reduced into its smallest solvable unit, doing so here provides us with ‘clean’ data insights which allow for goal setting following the SMART model.
The importance of data driven decision making cannot be overstated, for a deeper dive including dynamic data retrieval via Twitter APIs, Python driven data manipulation, visualization and NLP Sentiment Analysis look for my article here.
Recognize constraints as choices
Everyone has challenges, everyone has resource restrictions; at the end of the day we are presented with choices, and these are absolutely choices, that drive quality outcomes: Scope, Time and Resources (cost).
Unfortunately QA leads and Directors don’t have unlimited authority to incur change, with many players and moving parts in any organization we need to be extremely collaborative to make progress with regard to quality. Assessing and communicating risk must quickly become a focus and a skill — decisions must be well informed, take it upon yourself to ensure risk and reward are well understood and do your best to develop a process that keeps this ever apparent.
It’s important to understand the need for honest and accurate assessment of any development effort; those in leadership, who are tasked with cross-functional alignment require accuracy to make good decisions, period. In SCRUM we rely on the individual contributors, in collaboration with their team, for estimates and planning. In this regard many teams set themselves up for failure by overpromising and underdelivering, by avoiding short term conflict we risk long(er) term predictability. A common frustration I very much share is a lack of honest insight into scope, time and/or resource requirements. Those in leadership being told what they want to hear, or worse yet being told what stakeholders think they want to hear, can be extremely damaging.
We need to be actively working to present solutions in the form of choices. I personally do my best to avoid presenting a problem without a solution, even when I don’t have a well developed method of execution. If you are not the most technical person on the team, and you very likely are not, don’t worry it isn’t your goal to solve the problem; you are, as any good leader should, only looking to illicit novel solutions from your team. You’re smart, everyone recognizes this or you wouldn’t be here — develop confidence in this and be willing to ask the “stupid question” or make the less than optimal suggestion. After all engineering is about iterative improvement, right?
Be conscious of how you present challenges, you would be well suited to not be perceived as someone with problems, but rather, someone with solutions who is well informed of the challenges at hand.
At the end of the day the Quality Org is unable to address many quality shortcomings without collaboration, across the board; for this you will need to increase ownership.
I recommend for anyone to start managing with the assumption that most people are hard working and, having been hired as a Software Engineer, uniquely capable of getting the job . So what do Engineers need? How do you get them at their best? In my opinion this comes down to trust — trust in you and, perhaps more importantly, your trust in them.
Trust from a direct report is reliant on your ability to listen, to collaborate on a functional process that will allow them to excel. Once again the “boots on the ground” know best as to what is hindering their work, they also usually have a good idea of what works — we can help them by refining their ideas, by helping them understand the big picture; trusting them with the information they need to understand the WHAT and WHY will garner trust and allow for collaboration that respects (resource) limitations.
Goal setting and career development — this is less common than you’d think, people get busy and many (non QE specific) managers frankly do not have the expertise to aid in SDET or QA growth. However, do your best to not miss this opportunity. Take interest in your direct reports, work with them to set goals, track progress and give honest feedback. Don’t stop there, work with your reports to get them exposure on their efforts, allow them to complete POCs, to present new tools and ideas in lunch & learns — this not only creates trust but helps Engineers ‘help themselves’ with regard to their career. As an added bonus this increased activity from the quality team will keep its function and effort top of mind for stakeholders; the increased transparency will be appreciated, it will create transparency in return and aid to elevate the quality team. It’s a win-win.
As is true with all managers its our job to protect our team from unwanted outside influence, to give them the time and focus they need to finish the job; do not be easily swayed by outside influence and make clear with the org, including your manager, just how important this autonomy is. We need to prevent unplanned work to be predictable; we need to limit direction going straight to your team from anyone other than yourself — living up to this managerial function will allow for increased ownership and a path to self determine success. This again makes a case for a well defined process with hard quality gates, process requirements and documentation.
Finally, lead by example; I can say from first hand experience that the “player-coach” model is effective (for smaller teams) for many reasons, one being that allows us to lead by example. I pride myself on being a manager who has done the technical work myself and when you have, it shows. If you lack the skill to ‘be the best’ on your team then be damn sure you are the hardest working. By being willing to roll up your sleeves and do the work yourself your team will quickly recognize the expectation and, believe me, team members will jump at the chance to prevent a manager having to do what is essentially their responsibility — its another opportunity to set themselves apart.
With regard to the org outside of our direct reports, education is likely your best ally here, helping stakeholders understand the impact of upstream process, of decisions made at each stage of development with regard to quality is KEY. Providing the process, transparency, expertise and the education required to empower all stakeholders to take ownership of quality is what we are looking for, this allows for SCRUM teams to self determine their success.
Testing is simply a feedback loop into the quality and release readiness of product code; testing is a part of development, part of any reasonable “Definition of Done" and is often responsible for exposing rework, if required. As I mentioned earlier the role of the Quality Org and all manner of testing is often misunderstood in this regard — we must champion this principle and work with stakeholders to plan accordingly, again this means expanding ownership outside of the Quality Org.
The reality today is organizations are moving towards a flat(ter) organizational structure that puts the responsibility on the entire team for all things, including quality; new tools and technology are working hard to make this a functional reality and I only expect this trend to continue — this means the importance of quality education, of facilitating the ability for teams to self manage quality is only growing.
As an aside, don’t be afraid to work yourself out of a job, I have done this several times over. Whether by developing good resources into great resources, automation or by creating an embedded quality structure without the need for a “Manager”. This should be the goal, and believe me there are no shortage of new challenges out there.
Be a storyteller
We need trust and buy in, we need to educate and we need to make things stick → we need to become good story tellers. Whether it’s communicating a resource need, suggesting changes to upstream workflows — you will need to tell a story. With regard to our data this means it has to be consumable, nothing we will be doing exists in a vacuum and many of the individuals who control our fate are outside of Engineering all together.
Visualize data, make clear how you are sure of your findings — make sure this can be consumed by stakeholders without a technical background, if you are trying to impress technical colleagues with complexity — DON’T, as with code, simpler is better.
With that, we have a great opportunity to work on our story telling. It’s time to organize our findings and recommendations in a report — this assessment will likely be your first task when joining a team and it will pay dividends; Here, work very hard to call out each issue at its root, be accurate and unrestricted in your assessment — sugar coating or swaying your findings or recommendations to please executives or in consideration of known limitations will only delay your inevitable failure. I expect many will find this refreshing and, if you do in-fact have challenges — as you surely do — then your report should have blunt and honest findings and recommendations; keep in mind recommendations are just that, they provide information to make good choices, nothing more.
This report should be shared widely and leveraged often, use this to bring the conversation back to root cause, the reality of the situation (data) and the recommendations within. Now you can (and probably should) create a separate “All Up” plan that distills your findings into action items — HOW will each point of failure be addressed, by WHO, specifically and — this is very important — how will you measure success going forward.
Perhaps this is a good time to drive home the importance of documentation; if I can give one piece of advise, specifically for Quality Managers, it would be to document everything. Treat your task management system (Jira) as your source of truth, Stories as contracts and encourage daily updates and comments on in-progress items; start documenting “sign off” on all manner of negotiations with development leadership — this fosters clarity and creates a ledger of sorts that will record each decision made and who was involved; this is absolutely not a means for blame and it’s very important its never used in this manner. This approach should empower teams to leverage historical data, provide actionable insights and facilitate iterative improvement that prevents repeat failure — if repeat failure does occur we have missed a valuable and expensive opportunity.
As a final takeaway worth mention: keep it simple. Not every complex project requires complex process or solutions to achieve quality.