Data Science Competency Matrix(1): A Career Development Compass
Have you ever found it challenging to discuss with your manager what you need to do to progress in your career? Or perhaps, understanding the criteria behind your performance evaluations has been elusive? Similarly, as a manager or coach, have you faced challenges in assessing your team members’ performance and providing guidance for their career development? This post explores the concept of the Competency Matrix as a compass for career development and a foundation for fair, consistent, and equitable evaluations. This is part of a series of posts about Product Data Science (check out other posts here). In this post, I’ll focus on the product data science role, although the concept of a competency matrix is applicable to all roles in product and technology¹.
Competency Matrix²
The Competency Matrix delineates the expectations and responsibilities of employees, such as Data Scientists, at each stage of their careers. It outlines what individuals need to demonstrate to meet the expectations of their respective levels. Consequently, it serves as the cornerstone for discussions between employees and their managers regarding feedback, performance assessments, and career advancement. Moreover, it provides clarity to cross-functional partners (XFN) by establishing expectations for team members.
A Competency Matrix is characterized by two dimensions: Competency and Scope. Competencies represent the behaviors or capabilities expected from an individual, while Scope defines the area to which those behaviors or capabilities apply and indicates progression through levels as it expands.
Scope is determined by two factors:
- The area of influence which grows along with the level, progressing from task → project → team → across teams → across the organization.
- The frequency with which the individual demonstrates a competency or behavior within a level, ranging from never → sometimes → most of the time → always.
Most competencies are role-agnostic and remain consistent across the industry. However, each role typically includes a role-dependent competency that reflects the individual’s mastery of that domain.
The following are the most commonly used competencies across the industry (this is my summarization of what I have observed. I do not claim it is based on an exhaustive review of all companies):
- Business impact and alignment
- Execution and Delivery
- Ownership
- Communication and Collaboration
- Domain Knowledge and Technical Skills
While different tech companies may categorize these competencies under different names (e.g., execution + domain knowledge = craftsmanship or Ownership = Direction/Leadership), they primarily utilize these five categories as the foundation for their competency frameworks.
Core Competencies and Their Dimensions
In this section, I’ll explain each competency and its dimensions, focusing on the product data science role. It’s important to note that the following discussion pertains to individual contributor (IC) roles. Management roles, while sharing the same competencies, entail additional dimensions and expectations related to leadership, coaching, and team management.
Business Impact and Alignment
This competency is outcome-oriented, focusing on what ultimately matters: Impact. Employees are expected to deliver results that propel the business forward, showcasing behaviors conducive to achieving impactful outcomes. This competency is evaluated through two dimensions corresponding to outcome and behavior.
Business Impact (Outcome): This dimension focuses on making a tangible impact on the business’s bottom line or other critical metrics. Examples include increasing user sign-ups, acquiring paying customers, generating revenue, or changing vital business metrics.
Business Acumen (Behavior): This dimension evaluates an individual’s capacity to comprehend and influence business issues and situations to achieve desired outcomes. To demonstrate strong business acumen, individuals should:
- Understand the business operations within their sphere of influence and identify opportunities for improvement.
- Identify and comprehend key metrics relevant to their area of responsibility.
- Have a solid grasp of the scale of key metrics and activities (e.g., daily user sign-ups on the platform).
- Understand how various factors affect metrics (e.g., the impact of the day of the week or seasonality).
- Develop well-founded hypotheses about the factors influencing business outcomes (e.g., factors impacting user churn).
Indications of good business acumen for product data scientists include instances where they guide product managers to decisions or discoveries during discussions or impart knowledge about the business domain. When their opinions carry weight in product-related matters, it signals a strong understanding of business dynamics.
Execution and Delivery
This competency centers on how an individual operates and delivers their work, comprising three dimensions: quality, quantity, and complexity.
Quality: This dimension emphasizes the caliber of execution by the individual, with factors including:
- Successfully executing projects through to business delivery.
- Clearly defining their approach before initiating tasks.
- Comfortably launching imperfect solutions and iterating upon them to improve them.
- Documentation: Defining requirements, updating progress, and capturing all assumptions and outcomes.
- Adherence to coding best practices: ensuring reproducibility, maintainability, and version control.
Quantity: This dimension primarily focuses on the volume of work an individual can complete within a specified timeframe.
Complexity: This dimension concerns the individual’s ability to effectively tackle complex problems, including:
- Ability to execute complex projects, addressing unknowns through appropriate breakdowns (methods or approaches).
- Starting with simple solutions and gradually increasing complexity based on the organization’s appetite and the scale of opportunity.
Ownership
This competency evaluates how an employee can take ownership of a task or project from inception to completion and lead cross-functional (XFN) partners to achieve desired results. It consists of three dimensions:
Independence: This dimension focuses on an individual’s level of autonomy, including:
- Embracing ambiguity and striving to provide solutions, especially when the scope is initially unclear.
- Leading initiatives by gathering requirements, defining deliverables, aligning with XFN partners, and understanding priorities.
- Facilitating delivery by executing tasks, resolving blockers, escalating issues when necessary, delegating work as needed, delivering results, and regularly updating stakeholders.
- Demonstrating enthusiasm for solutions and ensuring follow-up even after transitioning to new projects.
Prioritization: This dimension concerns how well an individual recognizes and acts based on what matters most for the business, including:
- Identifying key priorities for the business.
- Balancing the trade-offs between value and effort.
Initiative: This dimension assesses an individual’s ability to lead or take charge of tasks or projects to drive business impact, including:
- Taking the initiative to identify the next problem to solve.
- Proactively seeking feedback and ensuring that the impact of solutions is realized.
Communication and Collaboration
This competency evaluates an individual’s ability to convey critical information to team members and cross-functional (XFN) partners, as well as their capacity to form alliances to achieve desired outcomes. It comprises three dimensions:
Effective Communication: This dimension focuses on an individual’s ability to convey critical information orally and in writing. Key factors include:
- Clarity³:
— Ensuring communication is concise and clear.
— Avoiding unnecessary jargon.
— Emphasizing the most crucial points. - Brevity:
— Starting with succinct explanations and elaborating when necessary.
— Providing sufficient detail to convey meaning without unnecessary complexity.
— Avoiding lengthy and convoluted explanations. - Quality:
— Proficiency in explaining complex concepts in easily understandable terms.
— Adapting language to suit both technical and non-technical audiences.
— Practicing closed-loop communication to ensure understanding.
Interactional Capabilities: This dimension focuses on the ability to interact with others effectively to build productive work relationships. Key aspects include:
- Accepting feedback gracefully and addressing challenges constructively.
- Negotiating effectively and guiding others toward shared objectives.
- Advocating for and persuading others to utilize proposed solutions responsibly.
Cross-functional Capabilities: This dimension assesses the ability to form partnerships and alliances with XFN partners to foster a productive work environment, including:
- Collaborating efficiently across teams to achieve shared goals.
- Recognizing and appreciating the skills and contributions of others.
- Sharing knowledge with teammates and actively seeking opportunities to learn from others.
Domain Knowledge and Technical Skills
This competency encompasses qualities and behaviors indicative of mastery within a domain, problem space, tool, or methodology, facilitating innovation and the delivery of novel solutions. Below is an example within the field of product data science:
Data Analytics:
- Proficiency in pulling data, identifying trends, and effectively communicating findings to stakeholders (e.g., via dashboards), utilizing descriptive statistics and SQL.
Programming:
- Competence in Python/R for script-based data science.
- Adherence to coding best practices encompassing documentation, performance, and maintainability.
- Commitment to reproducibility and version control.
- Engagement in peer review processes, such as Code Review.
Storytelling with Data:
- Ability to provide clear insights tailored to both cross-functional (XFN) and non-technical audiences, influencing decision-making.
- Skill in crafting effective visual representations of data.
- Capability to persuade and guide stakeholders through data-driven narratives.
Measurement (Metrics):
- Aptitude in defining metrics for products and initiatives.
- Capacity to establish frameworks to measure success.
Research Design:
- Proficiency in defining the research process for identifying opportunities or validating hypotheses.
- Expertise in designing analyses end-to-end, spanning from product ideation to delivery, to address research questions.
Statistics and Modeling:
- Knowledge of inferential statistics.
- Proficiency in predictive modeling and forecasting.
- Familiarity with advanced statistical methods, including hierarchical modeling, survival analysis, Bayesian statistics, and others.
Conclusion
In summary, I highly recommend developing a competency matrix for your organization as a cornerstone for career discussions and providing actionable feedback to your data scientists. This matrix will establish clear expectations for each level and offer guidance on behaviors conducive to positive outcomes. It serves as a roadmap for career development and performance reviews, particularly effective when accompanied by specific examples for each competency and its dimensions. In the next post, I will explore how to leverage a competency matrix for career coaching and performance evaluation.
[1] I used an LLM for copyediting to improve the grammar and readability of the post.
[2] My perspective on competency matrices and this approach was initially shaped by my experience working at Indeed, where mentorship and structured evaluations were highly valued. Later, I had the opportunity to review competency matrices and career ladders from several companies, including CircleCI, Dropbox, Meta, and LinkedIn. Drawing inspiration from these sources, I adopted various elements to form my own view.
[3] On the topic of clear communication, I found Jon Paris’s post on communicating with extreme clarity to be particularly helpful.