How to read a delivery forecast: a guide for leaders
A good delivery forecast gives a date with odds, worked out from what the team has finished so far. This guide shows how to read one, why to distrust a single date and what to ask.
A probabilistic forecast gives you a date with odds, such as “an 85% chance by 6th October”. Read it this way: if the team carries on finishing work at the rate they have recorded, they will be done on or before that date about 85 times in 100. Plan on the 85% date, treat a 50% date as a coin toss and ask for the forecast again every fortnight.
Where the numbers come from
The team builds the forecast by counting, with no estimates of effort. Each week they record how many pieces of work they finished, and they count how many pieces remain. A simple program then plays out the remaining work thousands of times, each time drawing weeks at random from the team’s own record. Some runs draw good weeks and finish early. Others draw slow weeks and finish late. The share of runs that finish by a given date is the chance the team quotes for that date.
Statisticians call this a Monte Carlo simulation, and Daniel Vacanti describes how teams use it in his books on flow metrics, the measures of how work moves through a team. Assurance reviewers look at the team’s record of finished work too. In its guide to assuring agile delivery, the National Infrastructure and Service Transformation Authority (NISTA) tells reviewers to assess a team’s speed of delivery as a way of checking its future plans.
A worked example
Take a made-up team. Over the last twelve weeks they finished 10, 15, 7, 12, 10, 9, 16, 10, 11, 9, 15 and 10 pieces of work, 134 in all. Fifty pieces remain before the next milestone, and today is 1st September. We ran the simulation on those twelve weeks and it gave these results, rounded.
| Finished by | Chance |
|---|---|
| 29th September | 20% |
| 6th October | 85% |
| 13th October | more than 99% |
So the team tells you there is an 85% chance by 6th October. They can also tell you that 29th September is a one-in-five shot and that 13th October is close to certain.
Why you should distrust a single date
Ask the same team for one date and somebody will do the sum in their head. They round the average up to twelve a week, get about four weeks and say “the end of September”. The simulation gives 29th September a one-in-five chance. To hit it, the team would have to average more than twelve pieces a week for a month, and on their record they usually do not.
There is a second problem. When you are given one date, you cannot tell whether it is the 20% date or the 85% date. Once that date is written into a board paper, people treat it as a promise, and a team that is running late is tempted to cut testing or drop scope quietly to keep it.
What a forecast cannot tell you
The team has to assume that the coming weeks will look like the recent ones and that their count of remaining work is right. Both can turn out wrong. New work appears as the team learns, and people get moved to other priorities. That is why you ask for the forecast again every two weeks. If the 85% date moves later twice running, something has changed and you should ask what.
A forecast is also only good for the near future. We give dates for roughly the next three months. Beyond that we give a direction and tell you that we cannot see further yet.
If you take decisions on dates like these, our half-day Managing Complexity programme covers deciding when you cannot know the outcome in advance.
What to ask when a team shows you one
- Which weeks of your record did you use, and was anything unusual about them?
- How many pieces of work remain, and has that number grown since last time?
- What are the 50% and 85% dates? If the gap between them is wide, the team’s weeks vary a lot.
- What have you left out of the count?
- What would move the date, and can I help with it?
- When will you run it again?
When a team offers a single date, ask how sure they are and what the date is based on. If the answer is an estimate of effort, ask them to start counting finished work each week alongside it. Question 5 in our ten questions to ask your delivery team asks for those counts.
What to do next
Ask your delivery manager how many pieces of work the team finished in each of the last eight weeks. If they can tell you, ask for the 85% date for the next milestone. If they cannot, ask them to start counting this week.
Questions people ask
How do I read an 85% forecast?
The team is telling you that if they keep finishing work at the rate they have recorded, they will be done on or before that date about 85 times in 100. There is still roughly a one-in-seven chance of being later.
Which date should I give to ministers or a board?
We use the 85% date for anything you will repeat to someone else, and we give the odds with it. A 50% date is as likely to be missed as met.
How is a probabilistic forecast different from an estimate?
An estimate is someone's judgement of how long work will take before they have done it. A probabilistic forecast is worked out from counts of work the team has already finished, so you can check where it came from.
How far ahead can a team forecast?
We give dates for roughly the next three months. Beyond that the team does not yet know enough about the work to count it, so we give a direction and no date.
What is a Monte Carlo simulation in delivery forecasting?
It is a calculation that plays out the remaining work thousands of times, drawing at random from the team's past weeks. The share of runs that finish by a date becomes the odds for that date.