Becoming a real boy
Part 3 of 3: Keeping the demo honest
This is the third part of a mini-series.
Read Part 1 here
Read Part 2 here
The test
Several years ago, I went to a test screening of a documentary. I was one of about twenty people, all loaded up with nibbles and clipboards. The clipboard held a questionnaire on pacing, understandability, what resonated, and what I believed in the interviews with the talking heads on screen. The version released to theatres a few months later was far superior to the one I watched.
Screenings of movies that are almost ready are standard practice. The producers put a version of the film in front of people who have no stake in it being good, then ask their opinions while it’s still cheap to do something about it. Test screenings can help the filmmakers understand how the film feels; it’s the latest version of a project that’s grown from idea to script through production, and is still to be released.
The incentive to hear bad news
“It Looks Done” argued that the modern prototype is fluent enough to be mistaken for the product. “The World Says Otherwise” argued that the distance between the two hasn’t closed; it has only become easier to underestimate. The questions remain. What closes it? What helps us to cover that ground more quickly?
The road from demo to product isn’t straightforward. It’s not as simple as accelerating through a known route. The organisations that pull ahead over the next decade won’t only get from demo to production fastest; they’ll find out how far the demo actually was from the product, and pivot as they learn it.
This is the part that gets missed when organisations exhort engineers to use AI to go faster. An engineer knows what a demo running locally doesn’t include. A PM knows which paths were rehearsed and which weren’t. They may even present this when presenting the demo. The trouble strikes when those gaps aren’t acknowledged.
Picture the version where it goes wrong. An exec celebrates how close to solving a problem an organisation is on the strength of a demo. Someone in the room knows it’s nowhere near finished. Saying so, in that moment, in front of everyone applauding, is not a neutral act. It risks looking unhelpful, disloyal, or unwilling to work at the pace everyone else has agreed to celebrate. Staying quiet is career-preserving. The bills will fall due regardless.
This isn’t a story about one bad leader. It’s a story about misaligned incentives. Good news is rewarded more reliably than bad news in most organisations. If the cost of delivering bad news is always high, there’s no incentive to help the organisation to learn cheaply by raising it early. It can be seen as safer to let someone else deliver it later.
We’re missing conventions that enable everyone in the room to know which version of the product they’re looking at.
No one mistakes a storyboard
In film, every stage of a production has a name, and the name sets the expectation before anyone watches a frame. A script isn’t a promise of how a scene will feel. A storyboard is a guide to a scene, a shot list a description of how the camera will move through the scene, not the shot itself. Dailies are raw material for the filmmakers to track progress. A rough cut gives the first sequence of the film, enabling the story to be set in place. Various fine cuts can be tested. A trailer sells an idea, not the whole experience. Only the final cut is the film that gets released to an audience.
Software has similar stages but lacks the consistent vocabulary of film. A design document is a script. A wireframe or click-through demo is a storyboard. Raw model output that nobody outside the team accesses is like dailies. A beta version tested on early users is a rough cut or early fine cut. What gets shown to the board is a trailer. What ships is the final cut. The names describe the making, not the thing made. Say which one is in the room, and “nowhere near done” stops being a confession. It becomes a fact, stated in the ordinary vocabulary of the trade, the way a director says it about her own material.
It’s true that software has language for alpha, beta, release candidates, and GA, but these tend to be toward the end of the process, similar to the journey from rough cut to fine cut to final cut. What’s missing is the language for the earlier stages.
But a name only holds when a room can see what it stands for. In film, a storyboard could never be mistaken for a final cut. In software, the demo and the finished product look similar, and the maturity that divides them, the harness that scores the model, the guardrails that fence it, the monitoring that catches it drifting, sits behind the screen where the room cannot see it.
Write down the distance
So you write it down. Not a name spoken into the room, which the applause can wave away, but the distance itself, set down on the deck the demo is already using: the stage you are claiming, and the work still standing between it and done. For an AI product that list is the invisible engineering made visible. The evaluation you have not run. The failure modes you have not investigated. The explanation you could not yet give a regulator. It needs no new ceremony and no new authority; it is the honest version of the slide you were going to present anyway.
A person who interrupts the applause to say the work is not finished takes a risk. A slide calling out next steps or remaining work is part of the movie. It is not aimed at anyone, it is to be expected in any honest presentation, and it creates a record of where you are that can be referred to later. Even if it’s not acknowledged in the room, the distance is made visible. The record matters.
Becoming a real boy
From here, a shared language can emerge. Not a convention handed down from the top and adopted by every team at once; that arrives about as often as the reorganisation that fixes everything. Instead, the language accretes, one honest deck at a time, because the people who wrote the distance down kept being right about it, and being right is how a word earns its way into the language everyone uses.
Reality settles every one of these arguments in the end. The only choice is whether you let it do so on a slide today, or later on the path to production.
Pinocchio didn’t need courage to become real. He needed to learn. Software earns its way to reality the same way: by naming honestly where it stands and writing down what it still owes, until the list is empty and the boy is real.