The loop of uncertainty
Why knowing what you’ve shipped doesn’t tell you anything valuable
All I need
How do you know what’s valuable?
It might seem like a surprising question, but it goes to the heart of something that’s missing in agile software development practices. It’s even missing from the Agile Manifesto itself, which states that the “highest priority is to satisfy the customer through early and continuous delivery of valuable software”, but says nothing about how to measure that value, or indeed that satisfaction. “Working software is the primary measure of progress.” Not valuable, working.
This might appear pedantic, or seem like a small omission. Isn’t it obvious what value is?
I’m not sure it is.
Climbing up the walls
Everything that frustrates people about agile flows from this gap. The Manifesto treats value as a known input, something you arrive at through customer collaboration and good judgment, then deliver in small increments. The question of whether the software was actually valuable after it shipped; whether it changed behaviour, solved the problem, or justified the investment has no home in the text. It has no ceremony in the frameworks that followed.
Scrum, co-created by two of the Manifesto’s seventeen signatories, structures work into sprints that end with a potentially shippable increment of software.
Scrum has reviews and retrospectives, which focus on the increment being shipped or on the team practices. There is no structural mechanism for asking “did the last thing we shipped matter?”
Kanban, imported from Toyota’s production system, is a flow system designed to optimise throughput and reduce frictions in the way teams work. It, too, is silent on the idea of how to understand if the software being built is valuable.
We can argue over the merits of either framework, but what they have in common is that they both stop at the point where the most important question begins.
A common picture underpins many conversations about the importance of agile practices: the cone of uncertainty.
No surprises
The cone of uncertainty predates agile software development. It originated from Barry Boehm’s 1981 study of software projects, and the name was coined later by Steve McConnell. Boehm’s data showed that an estimate made at the earliest point of a project could be off by a factor of 4 in either direction.
Part of its appeal is that it makes sense. As teams work on a product, they address risks and deliver interim steps. The curve narrows, ultimately to a point where the product is delivered or abandoned. Whether they use Scrum, Kanban, or some other framework is immaterial. This is about reducing uncertainty.
The issue is that it stops at the point of delivery, as if all uncertainty has been removed. But all that has been proved is that you can build and ship. It doesn’t demonstrate that what you built was worthwhile. Product has been shipped. The team has done its job. It has deployed software built on a hypothesis, one drawn, with luck, from the same customer conversations the Manifesto prizes. What the cone lacks is the space to measure the truth of those conversations against reality once the software is live.
Ship, and the model is spent; it has carried you as far as it goes.
So, for all the talk of outcomes, the models return you to output again and again. Even the cone of uncertainty resolves at the point of shipping, so shipping becomes the measure of progress.
The bends
This means that there’s a bunch of people out there following agile practices, and wondering why the promised benefits don’t appear. We hear phrases like “agile doesn’t work” as opposed to “the frameworks are incomplete.” It’s harder to see the implication of the framework where you’re focused on implementing it.
People are executing the frameworks faithfully, and they’re adjusting some practices to suit their team dynamics. But because the frameworks don’t go far enough, they are faithfully delivered to the same place every time: delivery without learning.
Without a way to measure whether the work resulted in the desired change in customer behaviour, the model is incomplete.
The cone of uncertainty is only the first half of a loop. Rather than stopping at the point of delivery, the team needs to extend it out to measure what happens after shipping, when they can actually get customer feedback and data on what is happening in the wild.
Feedback loops are old and well-understood, yet oddly absent from the models that underpin agile.
Reimagining the cone as a loop allows us to create the visualisation that supports hypothesis-driven development. A good structure for a hypothesis is “We believe that doing X will result in change Y. We will know we have succeeded when we observe a measurable signal Z.” This ensures that the question of customer value is set from the beginning, and the team has a measurable criterion for success.
This transforms the cone into a loop that begins and ends with customer value. The loop starts by asking how you might create customer value, and ends with you measuring whether that value was created. This in turn helps the team decide what to do next.
The first half of the loop is the cone you already know. You imagine what might create value, test the cheapest version of it, narrow toward a bet, and ship. Most models stop here. The whole point of the loop is the half that follows: once the software is live (X), you can finally measure whether the change you predicted (Y) and the signal you set (Z) actually arrived. That measurement is what carries the team back to the start, asking again what might be valuable. Keeping a team moving through that second half, rather than stopping at delivery as it always has, is a discipline of its own.
Visualising a loop rather than a cone closes the gap that’s always existed in agile software development. You’ve shipped. That’s great. What happens next? How do you know what’s valuable? You follow the loop.