When forecasting with Kanban metrics, the 85th percentile line is usually the best fit to be our Service Level Expectation (SLE). While it gives a good tradeoff between outliers and the overall system, a frequently asked question is how much data do I need to start trusting my 85th percentile?
This page explains how much data is required to trust your 85th percentile line.
Let's us use an example to help us understand the amount of data required. In the image below, a ball is picked from a bag (it's the blue circle with a number one in it). We put it on a line as follow.
We then pick a second ball from the bag. We have a 50% probability that ball #2 will be above and 50% probability it will be below ball #1. By above, I mean it could be bigger or heavier. In other words, it's an attribute of comparision with ball #1.
We draw a third ball from the bag. This ball has a 33% probability of being between balls #1 and #2.
We then draw a fourth ball. It has a 50% probability of being between #1 and #3.
We draw a fifth ball from the bag. Ball #5 has a 60% probability to be between #1 and #4.
We draw a sixth ball from the bag. Ball #6 has a 66.67% probability to be between #1 and #5.
We can summarize the previous images in the following table:
| Number of balls on the line | The probability of landing between the edge cases |
|---|---|
| 2 | 33% |
| 3 | 50% |
| 4 | 60% |
| 5 | 66.7% |
We can extract the following mathematical formula from this example:
Finally, we can run this formula until we get a probability of hitting our 85% probability. Once 12 work items are completed, the next work item has a 85% probability of landing between our edge cases, which is the equivalent of our 85th percentile line.
| Number of balls on the line | The probability of landing between the edge cases |
|---|---|
| 2 | 33% |
| 3 | 50% |
| 4 | 60% |
| 5 | 66.7% |
| 6 | 71% |
| 7 | 75% |
| 8 | 78% |
| 9 | 80.7% |
| 10 | 82% |
| 11 | 83% |
| 12 | 85% |
Louis-Philippe Carignan