How to write a PRD for a feature that can be wrong

Every PRD I have written assumes something nobody bothers to say out loud: the same input gives you the same output. You write down what has to happen, someone builds it, someone else checks it, and the thing behaves on Tuesday the way it behaved on Monday. Put a model in the middle and that stops being true. The document has to change shape, and skipping that is how AI features get shipped without anyone being able to say whether they work.

The criteria that stop working

“The summary is accurate.” “The assistant answers the question.” “Suggestions are relevant.” None of those is testable. Two reasonable people will look at the same output and disagree about whether it passed.

On the fleet dashboards I could write “within 400ms the marker turns amber” and hand it to a tester who never had to ask me a follow-up question. That is the bar. A criterion you have to interpret is not a criterion, it is a preference you wrote down.

What goes in instead

You stop writing statements and start collecting cases. Thirty real inputs: the boring ones, the ones people actually send, and a handful you are nervous about. For each, what a good answer looks like and what an unacceptable one looks like.

Then the criterion becomes a proportion plus a hard floor. Eighty per cent of the thirty have to be acceptable, and there is a smaller set that must never fail at all. “Nothing in the safety set produces an instruction that could damage a vehicle” is something a person can check on a Thursday. “The assistant is helpful” is not.

Choosing those thirty inputs is the product decision. It is the same act as deciding what the feature is for, and I would not hand it to anyone else.

Wrong is a spectrum, so rank it

Ordinary software fails loudly. The request errors, the screen goes blank, someone raises a ticket within the hour. Model-powered features fail softly. The answer looks fine, it is off by a bit, and nobody catches it for a month.

So the document has to say which kinds of wrong you can live with. In fleet software that ordering is easy to write. An alert that fires when nothing happened costs an operator half a minute of annoyance. An alert that stays quiet while a vehicle sits in the wrong place for four hours costs a delivery, and sometimes a customer. Rolling both into one accuracy percentage throws away the only part of the analysis that mattered.

Edge cases are where knowing the system pays off

This is the part that is easier when you have built things. On deterministic work it meant knowing that route playback breaks when the GPS data has a hole in it, that an alarm can fire twice, that a dashboard which is fine at a hundred vehicles falls over at ten thousand.

The model version of that list: input arrives in a language nobody planned for. The retrieved context is three weeks stale. The model is confidently wrong in a way that reads well. The provider is slow at 4pm. Someone asks the same question twice, gets two different answers, and screenshots it.

Write those down before the build, not after the demo.

Cost and latency belong in the PRD

People file these under engineering. They are not. Eight seconds and eight hundred milliseconds are two different products; one of them can sit inside the workflow and the other has to sit beside it. And a feature that costs more per call than the work it replaces will get cancelled eventually, just after you have built the whole thing.

I learned the general version of this at scale, where how fast the map drew decided what the operator could actually see. The budget shapes what the feature can be. That makes it a requirement.

What I would measure

Not model accuracy. How often someone accepts the output without editing it. How often they act on what it suggested. How often they ask the same thing twice because the first answer was no use.

And one I have not seen anyone track: how long it takes the team to notice when quality slips. It will slip, and it will not announce itself.