August 21, 2026
AI didn’t break software engineering principles. It exposed how incomplete our application of them had become.
The discipline was built on an assumption so ordinary that most teams never wrote it down: same input, same output. Everything downstream of that assumption inherited it. A requirement could name an exact result because an exact result was available to name. A test could pass or fail because those were the only two things it could do. A bug report could say “reproduce it” and mean it.
Most AI systems do not honor that assumption. The principles built on top of it are not thereby wrong. They are underspecified — written for a class of system that has stopped being the only class of system in production.
Three departures, not one
It is worth separating three things that get filed under the same complaint.
The first is run-to-run variance. The same prompt, submitted twice, can return different text. The usual explanation is floating-point arithmetic; a 2025 analysis from Thinking Machines Lab found that is not the part that accounts for the variance most people actually see. Their account turns on batch invariance, or its absence: a request is processed alongside however many other requests happen to be in flight, and the result depends on the size of the batch it landed in. Batch size varies with load. The second run differed because more people were using the service.
The second is drift. Models are updated, retrieval corpora change, prompts get edited, a vendor deprecates a version. Behavior that was characterized in March is not the same behavior in September, and nothing in the interface announces the change.
The third is that the output is probabilistic by construction. What the model produces is a distribution over continuations; what you receive is a selection policy applied to that distribution. A policy that always takes the most likely token is a choice someone made, not a property of the system. That is not a defect introduced along the way. It is what the architecture is.
These have different causes and different remedies, and collapsing them into “the AI is unreliable” is how teams end up solving the wrong one.
What the principles look like when the output is a distribution
Requirements. A requirement that names an exact output is not satisfiable by a system that produces a distribution. The adaptation is to specify acceptable ranges of behavior — what the system may do, how often, and what falls outside. This is harder than it sounds, because it forces a conversation most teams defer: what would count as the system working.
Process discipline. Reproducing a result now requires more than the code. It requires the model version, the prompt, the retrieval index, the data, the configuration, and the evaluation criteria in force at the time. Any of those can move independently. Versioning all of them is not bureaucracy; it is the minimum needed to answer the question “what was it doing in March,” which is the question that eventually gets asked.
Measurement. Binary pass/fail describes a deterministic system well and a probabilistic one badly. The measurement that carries information is distributional — how the output varies, how it moves over time, how confident the system is and whether that confidence is calibrated. A single green test run on a probabilistic system is a sample of one, reported as a fact.
Risk management. The failure mode that matters most here is the quiet one. Variance is invisible if you only look once; nothing on the face of an output tells you the second run would have said something different. A system can degrade for months in a way no alarm is watching for, because the alarms were designed to fire on errors and this is not an error.
What none of this fixes
It would be easy to read the four adaptations above as a program for making these systems behave predictably. They are not that, and claiming otherwise would be a small lie with a long tail.
Nondeterminism is a property of how inference is made efficient at scale. Drift is a property of systems that get updated. Probabilistic output is what the construction produces. None of these is a malfunction, and no amount of process maturity converts them into determinism.
What the adaptations do is different and more modest. They make the behavior accountable rather than repeatable. They produce a record of what was expected, what was measured, what changed and when — so that when the behavior is questioned, there is something to examine other than recollection.
Where this tends to surface
The engineering version of this argument is usually made early, in the abstract, to people who are not yet in trouble. The version I find more interesting arrives late.
By the time a disagreement about an AI system becomes a real dispute, it is rarely a story about bad intentions. It is a story about a probabilistic system that shipped against expectations written for a deterministic one. Somebody promised, or was heard to promise, an exact result. Somebody delivered a distribution. Both parties were describing the same product and neither was lying.
At that point the useful question is not whether the behavior was a defect. It is what the parties understood they were agreeing to, and whether it was considered, analyzed and documented. A specification that named acceptable ranges is a record. A versioned evaluation history is a record. A shared understanding that nobody wrote down is not — and that is the gap these disputes live in.
Source
Thinking Machines Lab, “Defeating Nondeterminism in LLM Inference,” September 10, 2025. thinkingmachines.ai
Ariadne Latorre is a Florida Supreme Court certified mediator and a software engineer. She mediates business and commercial, insurance and risk allocation, LegalTech and information governance, HOA and community, and court-ordered matters, in English and Spanish, in Miami and virtually worldwide.