This is the second part of a three-part mini-series
Read Part 1 here
In film, the shoot becomes the easy part when everything else is done. It’s an absolute breeze compared to post-production. The edit suite is where the film moves from concept to product: the cut, the grade, the sound, the thousand invisible decisions that turn the footage into a compelling narrative. The hardest parts of software development happen in the equivalent of the edit suite. The earlier parts, getting to the prototype, are like creating a storyboard.
A demo answers one question. Does this idea have potential? If we take SVPG’s four big risks of feasibility, viability, user value and usability, the demo helps to address usability, and potentially feasibility. The biggest risks of viability and user value are still out there. Can you turn this into a live, scalable product? Will users get sufficient value out of it for you to make returns on your investment?
The chemistry read
The ability to generate a high-quality, high-impact prototype accelerates the feeling that you’re on the right track. People leave the demo believing they are most of the way to delivering a product that survives contact with the world. The distance, though, hasn’t narrowed.
Scalability, security, reliability are all bills that are yet to fall due. The customer who does something you never anticipated with the product hasn’t appeared on the horizon. What has changed is that the demo used to be more obviously unfinished, so you could feel the weight of what was still coming. Now the work feels done, or at least closer to the end than the beginning.
But we’re not there yet.
A demo is a chemistry read. Two actors read a single scene, the room feels a spark, and the spark has substance. But that spark won’t get the the film financed, or telll you whether the performances hold across two hours and a difficult shoot, let alone whether anyone will sit through the whole film. Everything that makes it a film is still ahead.
When the product is the model
And here the hard work may change, because we’re now building different types of products, and only one of them maps onto the way we have done things before.
If you are developing a traditional software product, you’re building something deterministic. You can build the way you always did, or use AI to accelerate. You test it the way you always did whether using AI or not. Your tests are there to ensure it does what you told it to, or it does not, and you can prove which, and the tests will return the same results until something in the code changes.
In traditional software, a green test stays green. Unless something changes, the deterministic nature of the software means it will continue to pass. With AI, you have no such assurance. Its output is a sample from a distribution. It’s non-deterministic, so the same test can provide different results. A demo that works is evidence the way a single coin flip proves heads is possible. There’s every chance that next time you flip the coin, it will show tails. With a model, there’s a range of outputs possible, and it’s often opaque how these relate to the inputs given. The cost of many failures aren’t yet visible: the inputs you would never have thought to try, and the cases where the model is fluent and wrong at once.
In a hybrid, where AI is being embedded in a traditional app, the old rules still apply everywhere the code is deterministic; the new ones take over the moment the path runs through the model. Inserting a model into something that’s run on deterministic logic is a new kind of problem. The tests that run after the model may not be able to tell a right answer from a plausible one, and acts on both the same way.
So there are new demands to deal with. Deterministic testing asks whether the code does what you intended. Evaluation asks whether the model is good enough, and how badly it fails when it is not. It’s a different kind of question, and the answers lie in the the harness that scores it, the guardrails that fence it, the monitoring that catches it drifting. Most of the engineering is now around the model, not in it. An AI product is not a reel you can print and screen unchanged. It is a live show that varies with each performance.
And the same system has to be able to say why it did what it did. A model returns an answer, not a reason, and “the model decided” is not a legitimate explanation. A lender in the United States has not been permitted to hide behind an algorithm too complex to interpret since 2022; from August 2026 the EU gives anyone subject to a high-risk AI decision a right to a clear and meaningful explanation of it. So, not only do you have to ensure that the product behaves well enough, often enough. You have to be able to ensure you understand why it reached the conclusion it did under a range of conditions that may not have been fully tested.
That work is nowhere in the demo. It needs to be in the product.
From chemistry read to premiere
The demo makes people believe, and that belief is valuable. It can help guide the team through the path to ten thousand concurrent users. It becomes dangerous when it allows people to imagine that they’re at a premiere rather than a chemistry read.
Pinocchio didn’t become a real boy overnight. He had to put in the work to learn how to exist in the world.