Planning and Managing Unplanned (“Other”) Work
Teams are called upon to continuously deliver value to customers. Some work is planned, but other work is necessary to keep the delivery wheels tuning and customer satisfaction at it’s expected exemplary level. This work is often called “unplanned” or “interrupt” work. It’s referred to in derogatory terms, the evil negative twin of planned work, but I believe it gets a bad wrap. The only reason it is unplanned is laziness, and without doing this work, delivery would grind to a halt. The only thing negative is it isn’t accounted for in planning and forecasting future delivery of work. This article aims to give you simple tools to account for it in a managed and orderly way.
I’m going to avoid the negative wording of “un-planned work.” I’m going to refer to this work as “other work.”
There are two problems managing Other work. We need to fix both -
1. Other work isn’t correctly anticipated and pushes out Planned work.
2. Other work is always assumed higher priority than the planned work.
How much Other work is normal? There is no correct amount of “Other Work,” there is only a problem if the expected amount is different to the actual amount. Without an estimated expected amount of other work, it’s simply not possible to correctly know the delivery pace of planned work. This causes many issues in planning delivery of that planned work. An important part of improving a team or organization’s ability to forecast is knowing how much other work is needed. Seems obvious in hindsight, but I see time and time again an overestimated team delivery pace based on ignorance of how much other work will be arriving and deemed too important (or more important) than planned work for the same period.
It’s important to realize different team contexts should expect different rates of other work just due to their circumstances. Lets defined how we will measure Other work, and explore the boundaries is what could be expected for some common team contexts.
Measuring the amount of Other work
The first step to managing Other work appropriately is making it visible. Visible through measurement and visible for discussion about ways to mitigate the impact. Two measures are helpful in understanding Other work -
1. Number of Other items arriving per hour/day/week/month (Arrival Rate of Other Work) — to make sure you have the right amount of team coverage in case this work needs an immediate turnaround.
2. Ratio of Other work and All work items arriving in the same period (Other Work Ratio) — to make sure the ratio is within expected ranges for determining if process or policy change is warranted.
To capture this data a way to count all items of work unit being received by the team is necessary. Simplest mechanism is to write each piece of work on a post-it note and count them each time period. Use a different color post-it note for Planned and Other work. Fancier mechanisms exist, tracking tools and spreadsheets will give the same result. Whatever mechanism you use, the needed data is a count of Planned work and Other work items arriving in the same period.
To make this simple for you, I created a simple spreadsheet that has only three required pieces of data for each item. Each row of data represents one item, its Arrival Date (or Start Date), Completion Date and Type of item. From these three fields over 18 charts are built, and these include Arrival Rate by type of work, and the Planned versus Unplanned percentage (which now reading this article should be renamed Planned and Other work). Figure 1 shows the meager input data requirements, in fact, even the Start Date is optional. Figure 2 and Figure 3 show the charts that are built using this data, Throughput and Other Percentage Rate. For future planning of Planned work, forecast using only the Planned Throughput values. This will ensure that you don’t set the team up for failure by planning using Throughput that includes Other work. The percentage Other work chart is useful during team retrospectives. If the ratio is higher than expected or the trend is increasing, some mitigating process changes can be discussed.
Get the spreadsheet here: http://bit.ly/Throughput
For process improvement, knowing a little more about each piece of other work helps, this includes -
1. The source of the request. Who is asking for it.
2. What the required work is, what specific skills are needed.
3. How long it took to do this work.
You don’t need this information all the time, a sample of a few weeks is often enough to pick the major impacting requests. Use a simple post-it note sampling and grouping mechanism that works like this -
1. Pick a specific post-it note color for Other work
2. Give each team member 10 (pick your number) of this color post-it note
3. Instruct the team members that when they get called upon to do Other work they capture it on the post-it note
a. When
b. Who
c. What
4. Instruct the team member that when they exhaust their 10 (again, pick your number) post-it notes they need to come and ask for more (you might tell them they can say No at this point)
5. Capture all of the post-it notes and cluster them on a whiteboard or wall
6. Discuss in the next planning and retrospective meeting whether this is the common amount and type of Other work and plan accordingly for the upcoming period
7. Discuss in the next retrospective how and why this Other work is arriving and if it can be managed better through new process
8. Repeat every three months or so.
This process captures everything required to understand the rate and reason for Other work.
Team Context and Other Work Expectations
Different teams will be expected to take on different amounts of Other work. It is intuitive that if a team is performing rapid response to customer issues, then they will be exposed to true high priority Other work on a continual basis. Teams developing a new product or feature isolated from any current maintenance of existing products should have almost no Other work. Most teams fall somewhere in between the extreme continuum.
If a team has been delivering for any period of time, the ratio and amount of Other work is easily computed as we have covered earlier. Start tracking all work items on post-it notes or the suggested spreadsheet. If a new team is being formed, it gets harder. You have to look for similar teams within the organization as a reference ballpark, and then you need to apply local context to formulate an estimated Other work load. Once the team begins delivering the actual rate and ratio of Other work can be computed.
Probing questions often help uncover the type of team context -
Q. What do we get called upon to support from a customer perspective?
Q. What do we get called upon as internal experts?
Q. Given what we plan to release, how much feedback do we expect that will urgently need to be done?
Q. What have we skipped doing recently that we might have to catch up on (eg. Applying service packs)?
Q. Do we maintain our own development and test environments?
As a starting point, here are the likely ratios seen in Software development operations -
● Rapid support and operations teams — 75–95%
● Teams maintaining and doing minor updates to existing products: 25–75%
● Teams developing new products and not supporting any production code: 0–25%
Remember, a higher ratio of Other work isn’t bad. It’s just the context of how much that team has to drop everything for emerging important work. The only bad thing is when this is a surprise and not properly accounted for in plans. The Other work ratio helps know how much capacity in the total team pace estimate to reserve for Other work. In a forecasting scenario, the total team historical or estimated pace is reduced by the Other work percentage, ensuring that the planned work for a period is indicative of how much planned work has historically been delivered, not inflated because of delivered Other work in prior periods.
It is also important to detect and avoid team abuse. Sometimes the Other work is carried out after hours or during overtime. In essence, the team appears to be delivering both Planned and Other work in the allotted period, but it’s artificial and unsustainable longer term. Design a way to capture the amount of work that can be delivered in a sustainable way. This means keeping your staff happy and productive and not burning them out by expecting Planned commitments to be delivered even though you used bad accounting and forecasting methods.
Types of Other Work
There isn’t one type of Other work to consider. Understanding what types of Other work a team will be exposed to, helps understand and make better predictions about what might occur in the future.
Regular Other Work
Regular other work is continuous work needed to keep the wheels turning. Some examples are the basic rate of ongoing product support and team training and development. There is no particular correlation between this work arriving and any release or delivery event (if there is a correlation, this would be in the Planned Event category of Other work).
To tease out this regular work, ask the team to look at all Other work that arrived during a period, and sort into groups. The Other work items that don’t fall anywhere else are Regular work! This bucket is the default catch-all.
If this source of Other work becomes large, it indicates that the planning process is missing some important aspect. Looks to see the cause of this work being needed and specifically mention that during planning to see if some aspect is being missed when designing the work needed. Look for opportunities to batch similar work types. For example, if Other work is coming along following up on questions from stakeholders, consider scheduling a 15 minutes block each afternoon where one development team member is available to answer questions.
Re-occurring Other Work
Re-occurring other work that is pre-scheduled to occur regularly or is batched at some particular size. Some examples are new hire equipment setup, team off-site gatherings, software upgrades and updates. It’s a good strategy to turn one-off regular Other work into batched re-occurring Other work where economies of scale make sense. For example, applying service pack to test environment servers, or refreshing test database data from a sample of production data are often more economical in larger batches. Don’t go crazy though, the correct batch size depends on balancing the holding cost (waiting) with the setup cost (per unit setup). There is no right answer to batch size, but one is often too small, so keep doubling the batch size until the holding time is deemed too impactful.
Seasonal (or cyclical) Other Work
Often some seasons have higher (or lower) arrival rates of regular, reoccurring and planned other work arrival rates. This is more of an adjustment figure to account for increases and decreases throughout the calendar year based on product or geographic reasons. For example, summer vacations cause a dramatic drop in delivered work in many parts of the world and just prior to major e-commerce periods causes a major increase in production support to make certain the most stable system is in production during that time.
Event driven Other Work
An increase or decrease in other work coincides just prior or just after a planned (or unplanned) event. For example, critical production support issues could be anticipated to increase just after a major release to all customers. Knowing when these events occur to allow us to better plan how much capacity might be lost during these periods.
The key to predicting an increase of Other work due to one of these events is to brainstorm likely upcoming events and ask the team to predict what Other work might be generated. The process I use is to brainstorm with the team a list of future events, for example, Product Y beta, Team Offsite, Company X Customer Conference of 10,000 users. Then brainstorm possible sources of Other work both prior and post that event. There is often some organizational memory of impact of these events, so prepare somebody to “tell the story” of what has happened in other years. Figure 5 is an example of an event driven Other work brainstorming session. These factors need to be accounted for in future plans.
Forecasting Other work Arrival Rates
Given historical Other work rate data, forecasting future rates can help plan future load and staff coverage. This type of forecasting is called Time Series Forecasting, because the rates of Other work might grow or shrink in trend over time, as well as have some days or weeks have their own higher or lower trends (called seasonality or cycles).
The classic multiplicative time series model says a value at time (t) consists of the multiplication of four components -
Value(t) = Trend(t) x Cycle(t) x Seasonal(t) x Irregular(t)
The trend component is how the value baseline might rise or drop over time. The cycle and seasonality component is how the values oscillate around the trend. And the irregular component is the random increase or decrease that occurs and is unpredictable. For example, the number of visitors to a website will hopefully grow over time (trend), and some days of the week will have higher visitation than others (seasonality), and there will be some variation just “because” (irregular). Other cycles in data might occur if you “zoomed out,” for example the stock market index closing price, there are boom and bust periods.
In the software development world, there is often a trend of growing Other work over time as our products get older or we support growing customer base. There are seasonal cycles either weekly or daily based on how those systems are maintained. And there is noise, where natural variation bumps up or down the load. To predict Other work we need a forecasting tool that accounts for at least trend and seasonality. We can ignore “Cycle” as long as we are forecasting near term. We can ignore the Irregular component, because its random after all and defies predictability by definition.
The easiest method to forecast future demand from a series of historical weeks or month Other work counts is to plot it on a graph and fit a linear regression line. Excel does this simply as shown in Figure 6. If the data we were plotting didn’t have seasonal (daily) trends, then this would be an accurate prediction. As Figure 6 shows though, the actual (blue circles) falls a long way from the orange linear fitted line historically. This would have been a forecast error. To forecast data with seasonality cycles, you need to do a little more work.
The simplest way to forecast data with seasonality is to adjust the predicted trend value by the average amount of error seen in the known historical samples for the SAME day of the week. Figure 7 shows how the average error seen for each day in the first three weeks is used to bump up or down the trend line prediction for the Week 4 prediction. It’s simple, but often effective and definitely better than linear trend line.
The algorithm is pretty simple -
1. Fit a linear trend line to the available historical data, and extend it out into the future (orange line)
2. For each actual value, compute the error from the trend line
3. Compute the average error for each similar day of the week (Eg. All Mondays, all Tuesdays, etc)
4. For the future trend line periods, adjust the trend line value by the average for that day of the week (black dashed line)
The benefit of this algorithm compared to others is its resilience in cases the data doesn’t have seasonality, and the fact that it doesn’t need “magic numbers” (Eg. Holt-Winters requires three parameters that need to be solved for optimal results).
To make this simpler for you I’ve created a spreadsheet which you can download here http://bit.ly/DemandForecasting
Summary
Other work is as important as planned work. In order to be successful, Other work is necessary to keep the wheels of progress turning. The error is in not expecting Other work to emerge, and not having a way to deal with it when it does. Other work competes for the same team members that planned work does, and ignoring it will only cause heartache when planning and forecasting upcoming periods.
I’ll leave you with the following takeaways -
1. Start capturing Other work data using the spreadsheet suggested or the post-it note process described
2. Talk about Other work during planning meetings to make sure plans incorporate this work as well
3. Talk about the sources and amount of Other work during team retrospectives. What can we do to make this work more orderly?
4. Forecast future Other work rates to make sure you grow capacity to deliver Other AND Planned work in the future
Spreadsheets:
Capture Other work rate and percentage: http://bit.ly/Throughput
Forecast future Other work demand: http://bit.ly/DemandForecasting
PS. I would love to hear your stories about Other work management and forecasting. Feel free to email me on troy.magennis@focusedobjective.com or follow me on Twitter: @t_magennis.
