The goal of a development and release management process is to provide guidelines to help improve the flow of information, accountability, quality, and velocity of software releases. The goal is not to impose executive oversight or control. On the contrary, tight product planning and release management enables more empowerment at the lead level and helps prevent unplanned projects from derailing the roadmap.
These guidelines borrow practices from a variety of sources but can best be described as a variant of the agile scrum methodology.
Adopting a quasi agile scrum methodology does not mean that you should favor speed over quality. If a feature isn’t ready you shouldn’t release it. Ideally, changes/fixes that address the feature’s “readiness” will be made and the feature will be released in the next sprint.
Squads are an autonomous unit of PMs, engineers, and designers primarily responsible for a portion of the user experience or business function evident in their squad name. A squad should have at least 1 PM lead, 1 Engineering lead, and 1 Product design lead. Squads use OKRs and sprints to represent the impact of the work they do to continuously improve a user’s experience.
Planning Milestones & Development Sprints
Weekly Product Review Meeting
Product review meetings are a weekly meeting to review plans and works in progress. The purpose is to inform, discuss and decide on a range of initiatives in different stages, with the goal towards keeping each other informed, gathering feedback and building consensus on next steps. This is the meeting where Feature Briefs, critical updates that cause roadmap changes, designs, and demos are reviewed (along with a variety of other points of interest).
Stakeholders should consider product review meetings their primary source of product information and their primary opportunity to provide feedback on product direction and functional design. Product should make this opportunity a priority. It is vital that relevant stakeholders outside of the tech org are involved as early as possible. With that said, if a stakeholder chooses not to attend a meeting or engage in the process, they forfeit their input.
Major Initiative Kickoff Meeting
The major initiative kickoff meeting happens once as a squad begins to design and scope a Major Initiative. It is where a Feature Brief, that has been given the go-ahead to continue exploration after a product review meeting, is broken down into design, analysis, and engineering tasks resulting in artifacts such as wireframes, EPQs, and/or user research studies. Tasks identified in designing and scoping the initiative are assigned during this meeting.
A squad’s lead PM, lead engineer, lead analyst, and lead designer are required to attend this meeting while optionally inviting stakeholders and other key designers/engineers/analysts who are likely to be significantly impacted by the Major Initiative or take on tasks coming from the Feature Brief. If a stakeholder chooses not to attend a meeting, they forfeit their input.
Design Review Meeting
Design review meetings have 3 possible subjects: visual/UX design review, user research review or technical design review.
Invitees should include all squad leads (engineering, product & design) or their chosen proxies, and all stakeholders likely to be impacted or interested. Lead Product Managers, Engineers, & Designers have the responsibility of encouraging stakeholders who are likely to be interested to attend.
Regardless of the format, the purpose of the review should be made clear by the Lead Product Designer. The designer should also make it clear what kind of feedback they need for from the group.
The feedback gathered may lead to additional design and/or research tasks to be created. The Lead Product Manager should work with the Designer and/or Researcher to estimate the level of effort (LOE), and plan the next review accordingly.
The design review meeting can happen any number of times between the major initiative kickoff and feature planning meetings. The number of times this meeting happens is at the discretion of the lead PM based on his or her judgment of when the squad is prepared to begin work on any part of a major initiative.
The lead PM is responsible for clearly communicating, to all interested parties, when a scheduled design review meeting will involve making a decision on design finalization.
Feature Planning Meeting
The major initiative planning meeting is where Stories in a technical design and/or UX specs are translated into Tasks and estimated using Story Points. This meeting is scheduled on an ad hoc basis, and typically should happen once for a Major Initiative, and usually covers several weeks of work for an entire team.
Feature Planning is led by a PM and involves an engineering lead and the engineer(s) who will own at least one task from the technical and/or UX designs. An effective practice is to have a PM walk through UX Designs and prompt engineers to break down each part of the design that requires a Task, or the lead engineer talks through the technical design to prompt the PM to create tasks for each component of architecture, logic and/or implementation.
Sprints are 2 weeks periods in which all tactical work for major initiatives and features is completed.
Example Sprint Calendar
No Meeting Day
In the Sprint calendar above you’ll notice that I’ve designated Weds as a “no meeting” day. While managers often have to break this rule, the goal is to dedicate at least one day a week of uninterrupted work. I have found that this practice contributes greatly to productivity.
Sprint Kickoff Meeting
Sprint kickoff is comprised of two primary activities:
- Preparing for the Sprint kickoff meeting
- Moderating the Sprint kickoff meeting
Preparing for the Sprint Kickoff meeting
In the days and weeks before a specific sprint kickoff meeting, the following activities should be completed:
Weeks before Sprint
- Finalize technical designs
- Finalize UX specs (when applicable)
- Features (or “Epics”), Stories, and Tasks are drawn from the product feature brief, technical designs, and UX specs, prioritized, and added to a Backlog (Feature Planning meeting).
Days before Sprint Kickoff Meeting
- PMs meet with their squad to prioritize and estimate all backlog stories, tasks, and bugs in a Backlog Grooming meeting,
- Forecast squad velocity for the forthcoming sprint.
Sprint Kickoff Structure and Agenda
The Sprint Kickoff Meeting happens on the first day of the Sprint, which is also the last day of the preceding Sprint. It includes all members of a squad (PM, Engineering, Design, Data Science).
The goal should be to spend minimal time in the actual Sprint Kickoff meeting. The PM should come to the meeting with a prioritized backlog of dev ready stories and tasks along with a calculated squad velocity. Sprint kickoffs generally take about an hour for each week of work, but one should endeavor to reduce that time to 1 hour. Preparation is key. The more prepared, the shorter the meeting.
Using story points, you can get an approximation of how much work a squad can realistically complete in a single sprint. In order to normalize for the ebb and flow of squad output, use an average of story points completed in the past 4 sprints to calculate a squad’s velocity. That velocity is then discounted based on team members actual availability, taking into account things like vacation time, to forecast the number of story points a squad can complete in an upcoming sprint. The calculation looks like this:
Velocity = (Sum of completed points in the last 4 sprints) / 4 (1 - (Count of squad member days / total squad member days))
The PM should strive to continually optimize the meeting for efficiency. This is a tactical meeting, strategic discussions should be avoided. If strategic questions come up, acknowledge and make note of them for later discussion.
- Previous sprint retrospective
- Did we achieve the objective/goals we set for the sprint
- What went well with the sprint
- What could we have done better going into the sprint and/or during the sprint?
- Convert learnings into action items to be applied to the forthcoming sprint (and beyond)
- Reframe the problems evident in the squad OKR and how the feature being worked on should address the problem for the user and impact KR(s)
- Outline any new information gained that may alter the team’s perspective on addressing the problem and/or the OKR
- Discuss any new information that may impact the sprint
- Confirm team capacity
- Pull stories and tasks from the Backlog into the sprint that approximately reflect team capacity
- Review acceptance criteria for each story and make any appropriate updates based on technology, skill, or team member changes since the last sprint
- Determine the needs, sign up for work, and verify estimates of the work owned
- Create overarching goal(s) for the sprint (1–4 bullets that encapsulate the goals and intended results)
- Call for a group consensus on the sprint commitments and goals considering the feasibility and extenuating circumstances.
Daily Stand-Up Meetings
On each day of a sprint, the team holds a daily meeting called the “stand-up.” Meetings are typically held in the same location and at the same time each day. Ideally, stand-ups are held in the morning, as it helps set the context for the coming day’s work, but there is no specific requirement around this in consideration of remote teams working in various timezones. These meetings are strictly time-boxed to 15 minutes. This keeps the discussion brisk but relevant.
Anyone in the org can attend any stand-up, however only those directly involved in the sprint may speak.
The daily stand-up is not used as a problem-solving or issue resolution meeting. Issues that are raised are taken offline and dealt with by the relevant subgroup immediately after the meeting.
During the daily stand-up, each team member, moving from person to person in a consistent manner (clockwise or counterclockwise) answers the following three questions in sequence:
- What did you do yesterday?
- What will you do today?
- Are there any blockers in your way?
By focusing on what each person accomplished yesterday and will accomplish today, the team gains an understanding of what work has been done and what work remains.
The daily stand-up is not a status update meeting in which information is collected about who is behind schedule. Rather, it is a meeting in which team members make commitments to each other.
If an engineer says, “Today, I will finish the new data module,” everyone knows that in tomorrow’s meeting, she will say whether or not she finished. This helps the team realize the significance of these commitments, and that their commitments are to one another, not to a company executive or business stakeholder.
Any blockers that are raised in the stand-up become the PM’s responsibility to resolve as quickly as possible. Typical blockers include:
- I need help debugging a problem with X.
- I’m struggling to learn X and would like some help.
- I can’t get the vendor to call me back.
- X is on vacation, and no one knows how to do Y
- I can’t get the X team to give me the information I need to compete Y
- X has asked me to work on an off roadmap project “for a day or two”
In cases where the PM cannot remove these impediments directly, she still takes responsibility for resolution, and if necessary, escalation.
Backlog grooming meetings occur opposite sprint kickoff meetings and aim to be just as efficient with squad time, if not more-so. These meetings aim to ensure all squad members:
- have the opportunity to thoroughly understand stories and tasks that will be added to a sprint at some point in time;
- suggest additional stories and tasks;
- suggest removal of stories and tasks if they are no longer relevant or the squad believes they won’t be addressed in the next 4 months;
- estimate the work involved in completing each story or task using Story Points;
- collaborate on setting the priority of addressing each story or task in preparation for the forthcoming sprint kickoff (and beyond).
The end goal of Backlog Grooming is to gain squad consensus on story point estimates and prioritization of the entire backlog. If that goal is achieved, in the interest of effective time expenditure, squads should cancel backlog grooming meetings until consensus is lost and/or new information, stories, or tasks are gained.
At least a lead PM or lead engineer plus one engineer is required for a backlog grooming meeting. Ideally, the entire squad will attend backlog grooming meetings in order to gain the most benefit from the time spent.
- Story points are abstract with no direct representation of time
- Story points are estimated based on engineering’s perspective of relative complexity (relative to all other stories/tasks that have previously been estimated) and confidence in their understanding of implementation requirements
- Story points use a fibonacci scale from 1–13
- Stories/tasks that are estimated at an 8 or 13 should be broken up into multiple stories/tasks, if possible
- Story points are assigned to a story/task by at least 2 engineers using the Poker Planning method.
- On first estimation, each team will create a straightforward and familiar task and assign it 1 point. This task will serve as the benchmark for estimating all other tickets until the engineering team has a common understanding of the relationship between complexity/confidence and points.
Sprint Updates should be sent companywide by the On Call PM, who sends a Sprint Update after each sprint kickoff. If your company is relatively small (under 200 people), the updates should be sent company-wide. Sprint updates should communicate progress on OKRs across all squads along with major initiative or feature specific status updates. Not all sprints will include user-facing features; regardless, sprint updates should be distributed for each sprint.
A Sprint Update should include the following:
- Executive summary highlighting key takeaways from the sprint
- Status, next steps, and percentage complete for each feature under development (using a color coding system such as yellow for minor risk, red for major risk and green for no risk are useful when communicating at-a-glance status information)
- Bugs reported
- Feature requests received since the last update
It is useful to maintain a historical record of each update so I would recommend using a Google Group (or similar) email list, explicitly for this purpose.
During “Internal Testing” a formal sign-off process should occur. A squad’s lead PM is responsible for ensuring that all required parties sign-off on the release prior to its public launch. If there are issues acquiring sign-off, the lead PM should escalate as appropriate. However, as the team becomes more effective as collaborators escalation should rarely be necessary.
Internal Testing Preparation
- Engineering continues to commit code to production, however, all committed code (user-facing or not) sits behind a feature flag
- Bug fixes are deployed and made live as needed
- Functional bugs require PM review to ensure the feature is functioning as
- For Internal testing, feature flags, and a domain-based whitelisting system should be used to make it easy to enable a feature for employees only
Internal Testing Execution
Once a feature is available to employees behind a feature flag in production and the lead PMs have QA’d the feature:
- The lead PM will email Release Notes to the company
Release Notes should include:
- Overview of the feature(s) being released
- Request for assistance testing the feature(s)
- Instructions for how to test the feature(s)
- Instructions for how to submit feedback and bugs (release-feedback@)
- Target date for starting feature rollout to users
- Deadline by which feedback and bugs must be submitted
Lead PMs (and QA Analysts when available) vet and respond to bugs and feedback submitted by anyone in the company
- Lead PMs prioritize vetted bugs and feedback in the backlog
- Feature launch date extensions are at the discretion of the lead PM with consideration for feature value delivered with or without addressing vetted bugs and feedback.
- Should reported bugs and/or feedback delay the target date for feature rollout, the lead PM will notify the company of the cause and revised target date for feature rollout.
The following sign-offs should be obtained before features become public:
- Lead PM
- Lead Engineer
- Business Analyst
- Product Design Lead (front end impacting only)
- QA Lead (when available)
- Primary Stakeholders (including Privacy, Security, Legal and PR)
- Sign-offs and Release status should be managed on a Release Management board
Post-Release Impact Analysis
After the launch of a new feature, PMs should:
- Finish building out dashboards to track all success metrics defined in the Feature Brief
- Perform a deep-dive analysis on product impact ~30 days after launch and distribute to relevant stakeholders
- Identify potential product improvements based on the performance data.
30-day Impact Assessment
The 30-day product impact assessment should go beyond restating or summarizing the performance dashboards. PMs should work with Product Analytics to perform a deeper analysis to better understand:
- Performance to date against success criteria outlined in the feature brief
- Impact on the squad KRs
- Potential causes or drivers of performance
- Financial or business impact of the new feature (if any)
- Meaningful correlations between feature usage and other major metrics
If possible, PMs should also incorporate qualitative research from users (for consumer-facing features) or internal stakeholders (for admin tools, etc) to support the conclusions made in the Impact Assessment. Once Analytics has signed off on the Impact Assessment, PMs should distribute it to all the stakeholders, including the engineers and designers that worked on it, and any business partners.
Using the performance results from the dashboards and/or the 30-day Impact Assessment, PMs should work with product analytics, design, and engineering to determine areas for improvement:
- Identify pain points in the feature and validate with user testing
- Develop a list of potential product improvements
- For each feature improvement idea, figure out if it would meaningfully contribute to this quarter’s OKR. If so, re-shuffle the roadmap to include it.
- If it does not contribute to this quarter’s OKR, but it could improve performance along another metric, add it to the Feature Request Inbox for prioritization next trimester.
Functional Team Workstreams
Different functional teams are working on different features/stories during the course of a sprint. Product and Design are working “ahead” of engineering in that they are preparing artifacts so that engineering can code and QA can then test. In general, the temporal cadence should look something like this:
- Product Managers — PMs should have 3–4 sprints worth of stories ready at all times. During any given sprint cycle, PMs, in addition to orchestrating the sprints and releases, are generally working on features and stories about a month out from development.
- Product Designer — Designers should be at least 1–2 sprints ahead of engineering with a goal towards having final pixel perfect specs and prototypes ready for engineering.
- Engineers (and QA) — During the current sprint engineers are of course coding. In addition, they are reviewing user stories for comprehensiveness and breaking them down into engineering tasks.
Project Management Responsibilities
Product’s goal is to provide support in orchestrating work, not to provide oversight or “manage” the engineering process. At the same time, every engineering team is different in terms of what they want or need with regard to project management.
The following is a guideline on how responsibilities could be divided, but it is really up to you and your team to determine how Project Management should be handled.
Product Lead responsibilities
- Quarterly roadmap planning
- Triaging internal feedback
- Driving backlog grooming
- Driving the major initiative and feature planning process
- Driving the sprint kickoff process
- Leading sprint kickoffs and retrospectives
- Delivering development ready UX specifications
- Working with engineering to understand the engineering tasks required to complete each story
- Helping engineering to validate tech tradeoffs and the cost/benefit associated with these tradeoffs
- Gaining a non-superficial understanding of engineering tasks and their dependencies
- Assessing release viability and making a go/no go determination
- Leading daily stand-ups and providing support to remove blockers
Engineering Lead responsibilities
- Making best efforts to provide realistic estimates
- Challenging product assumptions and be vocal about product ideas and concerns
- Defining the tech requirements and engineering buckets of work to deliver a story
- Delivering approved Engineering Project Questionnaires that describe the architecture, logic, and implementation of a Feature.
- Ensuring that individual engineering tasks created by their team are adequate and well-planned
- Communicating to product when needs aren’t being met (requirement details, design specs, etc.)
- Communicating to product when unforeseen circumstances demand consideration of scope increases
- Communicating to product as soon as sprint commitments and/or initiative/feature timelines are identified as at risk
- Owning the quality of the work and deliver against commitments
Quality Assurance Engineers
I don’t advocate for dedicated QA engineers, especially for start-ups, where resources are scarce.
Ultimately, quality control is the responsibility of the product team (PMs, engineers, product designers), with the PM having final say on whether a feature is “production ready.”
Relying on QA to test code is both inefficient and encourages poor engineering and product management practices. Instead of compiling code, passing it to QA engineer to test and report back bugs, fix the bugs and pass it back (often several times), engineers should be encouraged to write defect free code.
For every internal build, PMs should run through all of the user flows themselves. Not only to ensure it is free of obvious defects, but to make sure the feature is working as intended and is free of any “functional bugs” — meaning user experience or functional issues that are not programming errors as much as design problems. These types of issues arise frequently during the development process, as you can’t plan for every use case, or foresee every possible issue until you actually sit down and use the feature. If PMs rely solely on QA to run through test scripts, they simply won’t get to know their product as well, nor will they learn to spot potential issues with features that were coded to specification.
The following tools and practices should make dedicated QA engineers unnecessary:
- Engineers run unit tests and clearly document their code
- All code is reviewed by an engineering peer before being committed to staging or production (code review)
- PMs and product designers review every build
- Every feature is released internally, company-wide, and every employee is encouraged to use the feature and report any issues
- A good way to encourage participation is to have a “bug hunt,” offering a reward to whoever finds the most confirmed defects
Like QA Engineers, Project Managers are unnecessary for most technology companies (large and small). The project management function should be shared between product managers and engineers, with PMs doing most of the heavy lifting.
This approach eliminates the unnecessary middle-person between the engineers, product designers, data scientists, stakeholders and product managers.
Moreover, orchestration is fundamental to product management. Adding technical project managers to your organization impacts the ability for PMs to learn and grow in this dimension, which is vital to becoming an effective PM. Instead, hire more junior PMs.
Products, Major Initiatives (MIs), Features & Stories
There is a fundamental hierarchy and unit of size to the functional components that constitute a Product:
- Major Initiative
These have a 1:1 relationship with Feature and are, therefore, the most common functional components found on a Feature Roadmap. They are supersets of features and almost always require more than 1 sprint to launch. In cases where Major Initiatives are relatively small, they may have a 1:1 relationship with a Major Initiative.
Feature, Story and Task Management
Features are usually identified in Project Planning meetings and they constitute development milestones that can be reached sequentially or in parallel with each other. Stories and tasks are typically more technical in nature and typically not named in a manner conducive to at-a-glance understanding by stakeholders, which is why they are less commonly represented on Feature Roadmaps.
The organizational model for Features and stories is a digital representation of boards. Each board represents a feature and is comprised of stories represented by lists of cards. Think of the cards as index cards that you can pin and rearrange on cork boards. “Kanban boards” are often used for this purpose, but you should not feel constrained to using the Kanban method. Software tools such as Trello or Asana are easy to use, lightweight, collaborative project management tools that are particularly well suited to managing features and stories.
“Epic” is an Agile Scrum term for a big feature. I recommend avoiding the term, as it is not broadly understood, making product communications harder to understand for those in the organization not directly involved in product development. Instead, I recommend sticking with the widely understood term, “Feature.”
A Feature Brief (or product requirements doc, aka PRD) is a more comprehensive representation of an “Epic Story,” which in strict Agile Scrum is often no more than a single line of text. While breaking requirements down into simple and easily comprehensible user stories is a highly efficient practice, you need to strike a balance between meeting short term tactical goals and maintaining visibility into and communicating the solution, and most importantly, the user needs it seeks to meet.
I encourage deviating from strict Agile Scrum practices and using Feature Briefs/PRDs as a canonical solution-oriented document defining technical architecture, logic and implementation plans required to launch a Feature.
Feature Briefs evolve over the product development lifecycle. Initially, they serve as a means of defining user problems, proposing a variety of solutions to address them, and information gathering until a single solution is selected, based on research and experiment results.
A Feature is generally composed of several user stories defined in a Feature Brief. Stories are short, simple descriptions of a capability or user action, told from the perspective of the person who most benefits from the new capability, usually a user. However, on occasion, there will be a 1:1 relationship between Feature & Story.
User Stories vs. Job Stories
User stories are best used to succinctly describe what value a Feature delivers to a user. User stories focus on what motivates the user, making them intrinsically more user-centered in how we think about product features.
Job stories are best used to describe how the Feature should work. Job stories provide more context than user stories by articulating a situation in which the user experiences the Feature.
Here are some examples to emphasize the difference:
User Stories commonly have a one-to-many relationship with Job Stories. Best practice is to minimize the number of Job Stories to increase the probability of completion in a single sprint.
A User Story and its associated Job Stories are listed together as Stories in Backlogs. Stories should include:
- Definition of Done
- Acceptance Criteria
A Task is a single unit of engineering (or data science or design) work that supports a User Story. In a major initiative kickoff, design review, or feature planning meeting, a squad reviews a Story and defines the Tasks required to support it. This is done through collaboration with an engineering lead (and/or DS lead and/or Design lead) and the lead PM for a squad along with any/all squad engineers/designers/stakeholders/etc., they deem valuable to participate.
A Task should be granular enough to be completed by a single engineer, scientist or designer in less than 1 sprint of 2 weeks. A Task should include a link to the Story it supports.
Tasks are typically of a technical nature when concerning engineering or data science work. When concerning design work, tasks are typically representative of a deliverable.
Examples of Tasks include:
- Display x button
- Create endpoint for x API call
- Add column to x table for y
- Deliver design specs for x (Design)
- Define a training model for x (DS)
Artifacts & Tools Summary
Major Initiative Tracking
Major Initiatives are discovered, defined and managed on a squad by squad basis. Each squad maintains a persistent Feature Roadmap during a quarter while squad roadmaps are combined to holistically capacity plan for design and engineering resources during the planning period.
Major Initiative Status
Major Initiatives in a squad’s persistent Feature Roadmap displays a status that represents the progress of a story through the product development lifecycle.
A Feature Brief serves as the canonical artifact for a feature, defining user problems, proposing a variety of solutions to address them, and information gathering until a single solution is chosen. Feature Briefs have a 1:1 relationship with Major Initiatives. Once a solution is defined the Feature Brief expands to include more details, such as technical designs and product design specs.
Release Management Board
The Release Management board is used for gathering and archiving a checklist of sign-offs and pre-flight tasks. The Release Feedback board is used to review feedback and bug reports submitted to release-feedback@ during the internal testing period.