Software Estimation: Why It's Hard and How to Do It Less Badly

Every developer has been asked "how long will this take?" and has, at some point, given an answer that was wrong by a factor of two or more. Software estimation is the part of the job most people are worst at and least willing to admit it. Clients experience it as broken promises. Managers experience it as roadmaps that slip a quarter at a time.

This post is about why estimates go wrong and what we do about it at Wolf-Tech. Nobody estimates software well. The realistic target is estimating it less badly, and being honest about the parts you cannot know yet.

Why software estimation fails

Developers are optimists, but the more useful observation is that estimates fail in repeatable ways, and once you can name the failure modes you can build defenses against them.

The first is unknown unknowns. When you estimate a feature, you break it into the parts you can see: a migration, an endpoint, a form, some tests. What you cannot see is that the third party API you are integrating rate limits you at 10 requests per second, or that the legacy table you need to read has no primary key, or that the client's SSO provider does not support the flow you assumed. None of these were on the list because you did not know they existed. Bottom-up estimation only counts the work you can imagine, and the work you cannot imagine is where the time goes.

The second is the planning fallacy. Kahneman and Tversky described this in the 1970s and it has held up: people estimate their own tasks by imagining the best case scenario in which nothing goes wrong, even when they have personally lived through many cases where things went wrong. You know your last five migrations each took a week longer than planned. You still estimate the sixth as if it will be the first clean one.

The third is a lack of reference data. Most teams do not record how long things actually took, so each estimate is made fresh, from intuition, with the same optimism as the last one.

The fourth failure is about vocabulary rather than psychology.

Effort is not duration

"Three days" can mean three days of focused work by one person, or that the feature will be done three days from now. These are effort and duration, and confusing them is probably the most common cause of a missed deadline that had nothing to do with the code.

Effort is working hours consumed. Duration is calendar time between start and finish. Duration is always longer, because the person doing the work also attends meetings, reviews pull requests, waits for staging, waits for the client to answer a question, and gets pulled into a production incident on Tuesday. A nominally full time developer often has 50 to 60 percent of their hours available for delivery. Three days of effort becomes a week of duration without anything going wrong.

Clients hear duration. Developers usually mean effort. Say which one you mean, every time. We quote both: effort in person days, and the earliest realistic calendar date given who is available.

Reference class forecasting

The best correction for the planning fallacy is to look at the task from the outside. Instead of asking "what are the steps and how long is each one," ask "when we did things like this before, how long did they take?"

This is reference class forecasting. It was developed for large infrastructure projects and works at the scale of a two week feature just as well. You need a reference class (past tasks similar enough to be comparable) and a record of what actually happened to them.

The reference class does not need to be precise. "Integrations with a payment provider we have not used before" is a reference class. If your last four took 6, 9, 11 and 14 person days, your estimate for the fifth starts from that distribution, and you adjust only if you can point to a concrete reason this one is different. "This one seems simpler" is the planning fallacy talking.

Most teams do not keep the data. Start now. A spreadsheet with task name, category, original estimate, and actual effort is enough. After six months it is worth more than any estimation technique.

Three-point estimates

When you do have to estimate bottom-up, because the work is novel or the reference class is empty, a single number is the wrong output. Use three.

For each task, write down an optimistic estimate (everything goes right), a realistic one (the usual amount of friction), and a pessimistic one (the thing you are worried about happens). The classic PERT weighting combines these as (optimistic + 4 × realistic + pessimistic) / 6, which gives you an expected value that leans toward the realistic case but is pulled up by the pessimistic tail.

The number matters less than the exercise. Writing a pessimistic case forces you to name the risk. "Pessimistic: 8 days if the client's API sandbox is as unreliable as their docs suggest" can be acted on; someone can check the sandbox before the project starts. A single estimate of "4 days" hides that risk.

The gap is also information. A task estimated at 2 to 3 days is well understood. A task estimated at 2 to 12 days is not, and it should not be treated as a 5 day task because that is the weighted average. It needs investigation before anyone commits to a date.

The cone of uncertainty

The cone of uncertainty, popularised by Steve McConnell, describes how estimates improve as you learn more: at the very beginning of a project, before requirements are settled, an estimate can be off by a factor of four in either direction. Once requirements are agreed, the range narrows to roughly plus or minus 50 percent; after design and some implementation, plus or minus 20 percent. Only near the end does it converge on the actual.

The exact multipliers are debated. What matters is the shape. When someone asks for a number at the first meeting, the honest answer has a wide range, and a narrow one at that stage is a guess delivered with a straight face.

We use the cone to label our estimates. A number after a one hour intro call is a "rough order of magnitude" and we say so in writing. A number after discovery is a "budgetary estimate" with a stated range. A number after the first sprint on a well understood codebase is a commitment. Clients who know which stage they are at stop treating the first number as a promise.

Spike stories for the parts you cannot estimate

Some work resists estimation because nobody on the team has done it before. A spike is a time boxed investigation whose output is an answer rather than a feature. "Spend one day finding out whether the vendor's webhook payload includes the fields we need, and whether their retry behaviour is documented." Either you get the answer or you learn the question is harder than expected, and both are useful. The spike itself is easy to estimate because it is capped.

Spikes work well for third party integrations, performance questions ("will this query hold up at 10 million rows?"), and anything in a legacy codebase where the answer to "how does this module work?" is "nobody knows." On legacy modernization projects we sometimes front load a week of spikes before producing any estimate, because the alternative is estimating a codebase we have not read.

Presenting estimates to clients

The techniques above produce ranges. Clients want numbers. How you bridge that gap decides whether the estimate helps the relationship or poisons it.

Present the range and explain what would push the outcome to either end. "This is 15 to 25 person days. The low end assumes your API documentation is accurate and the staging environment is available from day one. The high end covers the case where we need to work around the missing primary keys in the orders table." The range now reads as a specific set of risks rather than vagueness, and the client can act on it. A surprising number of estimation problems are solved by a client who now knows to chase their IT department for staging access.

Do not present a single number with an unspoken buffer. Padding 10 days to 15 works until the client finds out, and then every future estimate is discounted. The range with reasons is the buffer, made visible.

When a client insists on a fixed price for scope that is not yet clear, accepting the price and hoping is the wrong move. Change what is being fixed instead. Fix the price of a discovery phase whose output is a specification and an estimate with a narrow range. Then fix the price of the build against that specification. If a client refuses even a discovery phase and wants a fixed number for a one paragraph brief, that says a lot about how the project will go, and it is fair to walk away. We have written separately about fixed price, time and materials, and retainer trade-offs for EU clients.

Contract structures that protect both sides

A contract that pretends the estimate is certain punishes whichever party guessed wrong. A contract that acknowledges uncertainty distributes the risk in a way both sides can live with.

A short, fixed price discovery phase (one to three weeks) produces the information the cone of uncertainty says you need before committing. Its deliverables are a written scope, a prioritised backlog, identified risks, and an estimate with a range the client can take to anyone else if they choose. Neither side is locked in before they know what they are buying.

Time capped phases come next. Rather than one fixed price for the whole build, the work is split into phases, each with a budget ceiling. Work is billed on time and materials up to the cap. If the cap is reached, both parties stop and decide whether to extend, cut scope, or ship what exists. The client's downside is bounded per phase, and the developer does not eat the cost of unknown unknowns nobody could have seen.

For clients who need a truly fixed price, because of procurement rules or a funding round, the price should attach to a fixed specification with an explicit change process. Anything outside the specification is estimated and priced separately. That is the only way a fixed price stays honest.

How we estimate at Wolf-Tech

Our process for a new custom software engagement follows the shape above. After the first conversation we give a rough order of magnitude, labelled as such, usually with a factor of two or three between the ends. If the client wants to proceed, we run discovery. For an existing codebase this includes a code quality review, because the state of the code is the largest single variable in any estimate. We do not pretend to estimate a codebase we have not read.

Discovery produces a backlog with three-point estimates, a reference class comparison where we have one, and a list of open questions each with a spike attached. The estimate goes to the client as a range with the risks spelled out. Then we agree on a phase structure and start.

We also keep the spreadsheet. Every task has an original estimate and an actual, and every few months we look at where we were wrong. Categories where we are consistently optimistic (undocumented APIs, anything touching email deliverability) get a standing correction factor. Unglamorous, but it is why our ranges have narrowed over the years.

If you are budgeting a project and want an estimate you can defend, or another vendor's estimate seems too confident for the information they had, write to hello@wolf-tech.io or visit wolf-tech.io. We are happy to talk through the scope before anyone commits to a number.