Demos Over Deadlines

Image: Smoke Art Cubes to Smoke — MattysFlicks — (CC BY 2.0)

This post is part of the Managing Software series.
< Previous

As an engineer, I served on many teams: some very efficient and functional, some less so. One predictive factor in the health of the development teams was time pressure. Too much time pressure inevitably led to burnout, poor morale, poor employee retention, and poor team performance.

As an engineering leader, I try hard to balance two competing goals: Ship quickly, but avoid putting too much time pressure on the team. I prefer to give the team clear priorities and then get out of the way while they execute. That said, one of the goals I set for teams is to frequently demo newly built features. If there are no new features to show off, there is nothing to demo. By focusing on demos instead of deadlines, we turn what was a source of stress (time pressure) into a fun game or challenge (demo cool new stuff).

Since switching to demos over deadlines, I’ve noticed remarkable changes on the teams I’ve built. Developers take a deeper sense of pride in the features they’re building. They get credit and recognition for producing great work. They get to feel the appreciation of the rest of the company on demo day as coworkers applaud the new demos.

But to really understand the power of the demos over deadlines approach, first we need a deeper understanding of what is so wrong with deadlines, and at the heart of it is one key problem: It’s impossible to accurately estimate software completion.

Software Estimates Lie

Most engineering estimates are lies. In 2012, CNN looked at the top 50 Kickstarter projects and found that 85% of them shipped late. “Why Is Software Late? An Empirical Study of Reasons for Delay in Software Development” published in IEEE Transactions on Software Engineering found an average effort overrun of 36% in their survey of 72 software projects.

Estimation Challenge

Before we get into this, forget about search engines and virtual assistants, and try to estimate off the top of your head the average number of bees in a colony. Write down your answer now.

Why are software estimates usually wrong (commonly by orders of magnitude)?

• Scope creep — new features were added, pushing out the ship date.
• Undiscovered scope/complexity. Software development is an exploration. We’re doing something we’ve never done before and in the process we’re discovering new APIs, new protocols, engineering new components from scratch, and occasionally solving a problem that nobody has ever solved before. This is not trivial work, but when we estimate, we trivialize it. We almost always underestimate the scope and complexity of the problem.
• Developers think about how long it will take them to implement something, not how long it will take to implement, test, integrate, deliver, fix, retest, and redeploy before it’s finally done. The difference in the two values is orders of magnitude.
• Upstream/library vendor issues. Software developers often find bugs in the components they use and need to work with upstream developers to get the problems fixed — or fork the project and take on unplanned maintenance costs.
• Life happens. Key people get sick, have accidents, get married, have babies, etc. There’s no planning for these things.

Estimates are worthless lies because software almost by definition is something that has never been built before — it’s digital, so if it had been built before, you could just install it and use it.

Proper estimation requires a thorough understanding of the work involved, and a good record of the average time it takes to complete each task, but if you’ve never built a thing before, you have no idea how long it will take. Even if you have, you have no idea how long it will take. If you are personally building the next one, presumably you’ll be faster the second time around. If somebody else is building it, you may have had existing knowledge that the next developer will have to discover, so they might take longer. Different people complete the same task on different timelines. You could estimate by asking three developers to time themselves while they complete the task and then average the time, but you can probably see the problem with that approach.

Why Estimate?

The forces that drive us to estimate often feel insurmountable and unavoidable. A customer wants to know how much it will cost to build something for them. An investor wants to ensure that their money isn’t going to waste, and you need to keep investors happy so you don’t run out of money. You have to ship something before Christmas. Let’s take a look at each scenario and discover whether or not are deadlines are real or imaginary.

Customer Hired You on Contract

A customer is planning to hire you on contract, and wants to know how much it will cost to clone TikTok before starting. The only valid answer to that question is, “that depends on the full scope of the features, and we won’t know that until we’ve done a lot of work.”

Instead of giving the customer a number, show them other apps you’ve built. Provide some testimonials from other people who have confidence in your work, and then tell them you’ll dedicate a certain amount of resources to the project for a certain cost per month. Don’t charge for the product delivery. Charge for the time, and then work with the customer over time to control scope and cost and make sure it fits in their budget. Meet regularly to discuss priorities, report on progress, and show demos of the newly completed features.

One great bonus of this approach is that it reduces risk for both you and your customer. If you work for a month and discover that the project is way more complex than the customer envisioned, you can work together to adjust the scope so that it fits better with their business needs.

Investor Milestones

One of the biggest problems with software is that people are making commitments to deliver future software. In the startup world, people are trying to raise funds to build a business with software that hasn’t been created yet. In their investor pitch decks, they promise milestone deliverables. If we can hit targets a, b, and c within 18 months, we get a follow-on investment and we live to code another day.

In lieu of deadlines, we recommend undated milestone targets. Do spell out high level goals you’d like to achieve. Don’t attach them to specific dates. We can’t see the future, and opportunities and priorities change. By leaving dates off of your milestones, you’ll have the flexibility to focus on what’s most important right now, and that can provide better value for both your business and your investors. And since there are no dates, there is no point at which your business is considered a failure by investors unless you set priorities wrong and run out of money. As long as you’re alive and kicking and producing good work, your investors will be happy.

For investors, traction trumps everything. If you let them know an even better opportunity has come up and you’re delivering on that, as long as you do it, and as long as it adds substantial and obvious value to your business, they’re going to be thrilled.

If you are talking to an investor who’s insisting on delivery dates, keep looking until you find one who trusts you with the timeline. If they ask, tell them “we prefer demos over deadlines. So far we’ve delivered a, b, and last week we shipped c. Let me show you how it works.”

Dive into the demo and show off the cool new feature instead of getting bogged down in when the next one will ship. If you keep features scoped small enough to ship something cool every few weeks, you’ll have a new demo to show off next time you meet, and they’ll see you making continuous progress.

Planned Ship Dates

Companies also announce new products in advance of their ship date. Here’s the easy but not always possible solution: Don’t do that. Don’t announce products before they’re ready to ship and you’ll never be tempted to push hard and cut corners to meet an imaginary, self-imposed deadline.

Sometimes companies use early announcements to encourage customers to wait for your product rather than purchase from a competitor. For example, at the time of writing, both Playstation and Microsoft Xbox have announced that they will have new consoles in time for the holidays. If one had announced and the other hadn’t, people might have made plans to switch camps.

This is a risky move unless:

• You have a well established relationship with the vendors you depend on and you’re certain they can deliver for you.
• You have a well established process and a track record with great insights into the timing of the components that will go into the product. (This is very rare in software due to lack of repeatable production).
• Design is complete or almost complete, allowing for most of the product scope to be already discovered and accounted for.
• You have some flexibility and can cut scope to meet the deadline, even if the deadline was “next month” instead of “6 months from now”.

Ship dates frequently backfire when delivery dates are missed, leading to PR problems that can have a negative financial impact. For example, as I write this, a recent headline reads, “Shares Tank After the Firm Warns on Profits and Delays Three Big Games.”

Now, the bad news. While it is possible for well established companies with pre-existing vendor relationships to achieve those goals, almost none of that is ever true for software, for all the reasons already described.

That said, you can predict whether or not you will hit a specific delivery date as the work involved nears completion, if you’re paying special attention to the scope and moving low-priority items into follow up deliverables after the launch date. If you want to get a software product finished by a specific date, you need to be able to scale back the scope.

Brooks’s Law

“Adding manpower to a late software project makes it later.”

You may have heard of Brooks’s Law: Adding manpower to a late software project makes it later.

Some explanations of the phenomenon include:

1. New software developers require time to ramp up on any substantial application. Plan for 2–3 months before the developer can approach the average productivity of other members on the team. In the process of ramping up, new developers will have lots of questions about the project that only the early developers can answer, distracting your most productive people from their primary work on the project.
2. New developers often make mistakes that developers more familiar with the project and each component’s requirements would not have made, leading to an increase in rework.
3. Communication overhead increases exponentially with each new developer on the project, leading to a combinatorial explosion of communication complexity and reduced visibility. For example, each new developer might open several direct-messaging channels with other project developers. Developers may decide they need to break out into meetings to coordinate, opening up several new sub-group communication channels, and so on. The more developers you have on the team, the harder it is to coordinate work on that team.
4. The hardest part of software development isn’t writing the code. It’s decomposing complex problems into individually solvable, smaller problems. It’s often very difficult to break a task down into smaller pieces that individual developers can work on safely without getting in each other’s way. As Brooks points out: “nine women can’t make a baby in one month.”

Additionally, you must be aware that software development projects progress more slowly the closer you get to delivery, as evidenced by this real month over month progress graph. Notice the contrast between the big jumps in the first couple months and the much smaller jumps later on in delivery:

Real month-over-month progress graph.

“Nine women can’t make a baby in one month.”

Estimation Challenge: Solution

When I asked you to write down your estimate for the number of bees in a colony, did you write down questions? We don’t really have all the information we need to make an estimate, do we? What are we trying to count, exactly?

• Are we talking about the number of bees who participate in a colony during the course of the total life of the colony?
• The total for a full year?
• The average number actually in a beehive at a particular point in time?
• Are the numbers different for different geographies?

Did you provide a range, or a specific number? Most software developers will give you a specific number when you ask them for an estimate, trading accuracy for precision, but measured software engineering task timelines fall onto a power law curve with a very wide range. Nobody consistently delivers a feature “in one day” or “in about 2 weeks”.

Discovering Hidden Complexity

Most software engineers, when pressed, will provide an estimate, even knowing that they don’t know everything they need to know to do so accurately. What’s their average ticket velocity? Do they know? How many tickets will they need to complete to do the job? Have they sat down and figured out all the components they’ll need to build and tasks they’ll need to complete? Have they rated the complexity of each task and broken them down into all the necessary sub-tasks to achieve a reasonable estimate?

The truth is that the process of doing that breakdown is a big chunk of the process of designing a software application, and it works best if discovery and implementation progress incrementally, rather than all design up-front because design intentions are often at odds with the technical realities we discover when we begin to implement the design in code.

Everything in software development is composition: The process of breaking large, complex problems down into smaller, independently solvable problems, and then composing those solutions. The hardest part isn’t building and assembling those component solutions. It’s breaking down the problems correctly and designing the interfaces that the components will use to compose together and communicate.

In other words, you can’t accurately predict software completion dates without actually performing a substantial amount of the software engineering work.

Even if you’re measuring historical velocity and applying those velocity measurements to the number of currently open issues in the project, as you work on the project, many of the current issues will be split into many more issues as the work progresses and additional complexity is discovered. Real issue burn-down charts are not straight lines or smooth curves. They look more like stock tickers with random, unpredictable up and down movements as scope is uncovered and items are completed.

Real software burn down chart. Dots represent the number of open issues at that point in time.

It tends to grow at the beginning of a project until it hits a fairly steady range (dependent on the complexity and number of developers working on the project) and then moves mostly sideways with occasional sharp up and down spurts, until you reach 70% — 90%, and then it starts to average noticeably downwards.

Honeybee colonies are typically home to 10k to 80k bees.

• If you guessed a number less than that range, you lose. You’ve missed your deadline. Game over.
• If you guessed a number above that range, your boss figures out your game (under promise, over deliver) and starts cutting your estimates in half for public product announcements. You miss a deadline set by the marketing team in response. You lose.
• If you guessed a range and your top number is below 10k, you lose.
• If you guessed a fixed number inside the range, subtract the difference between your guess and 45,000 and multiply by -1. That’s your score. Congratulations! You didn’t lose. But you didn’t exactly win, either.
• If you guessed a range with both numbers in the range, congratulations! You get one point.

What’s the solution to the seemingly impossible challenge of software estimation? Don’t do it.

The keys to freeing yourself from estimation lies are simple:

• Demos over deadlines.
• Control scope.
• Set priorities.

If you need a deadline or you have an imposed budget, don’t try to cram every feature into the given timeline. Instead, design the system to be modular and easy to adapt to changing scope, and then adjust the scope as needed. This requires careful coordination with the product teams and stakeholders to identify what the real priorities should be. Solve the problems in order of priority. Mercilessly cut lower priority features out of the project.

Instead of trying to push more work through the clogged delivery pipeline, clear the clog by reducing the amount of work that needs to fit through the pipeline.

If you have a fixed budget, learn how to trim your big software dreams down to a lean MVP. Cut the fat. Watch your burn down charts and your month-over-month progress chart, and make sure you’re not giving your team more work than they can get done without cutting corners.

The result of applying demos over deadlines is a happier, more productive development team that will never ship late again.

Next Steps

DevAnywhere.io offers 1:1 mentorship to software leaders on a variety of topics including:

• Strategy
• Team building, hiring, and onboarding
• Cultivating healthy development culture
• Effective software development process
• Continuous discovery and continuous delivery
• Managing software quality
• Budgets and compensation
• Conflict resolution
• Technical debt

Ask us about our leadership track where we help everyone from aspiring tech leads to CEOs understand how to effectively manage software teams, and follow us on Twitter.

Eric Elliott is a tech product and platform advisor, author of “Composing Software”, cofounder of EricElliottJS.com and DevAnywhere.io, and dev team mentor. He has contributed to software experiences for Adobe Systems, Zumba Fitness, The Wall Street Journal, ESPN, BBC, and top recording artists including Usher, Frank Ocean, Metallica, and many more.

He enjoys a remote lifestyle with the most beautiful woman in the world.