Microsoft Conductor and Determinism in Agentic Workflows (Part I)

At temperature 0, the model chooses the most likely token at each step. In theory, that makes generation deterministic. In practice, it does not.

Horace He and Thinking Machines Lab measured this in Defeating Nondeterminism in LLM Inference (10 September 2025): 1,000 responses to the same prompt at temperature 0 with Qwen3-235B produced 80 different responses. The most frequent appeared 78 times, and the responses began to diverge at token 103. The cause is arithmetic: floating-point addition is not associative, and inference kernels change the order of operations depending on the batch size, which depends on server load. The caller gets a result that depends on something outside their control. With batch-invariant kernels, the authors obtained 1,000 identical responses, at the cost of slower inference.

The providers acknowledge this. OpenAI offers seed and system_fingerprint to improve reproducibility, but does not guarantee it (OpenAI Cookbook). Azure OpenAI says the same and adds that variability increases with longer responses (Microsoft Learn). And in a managed service, we control neither the kernels nor the batch.

• Sensitivity to input. A tiny change in the context—one more file, a different order, a log line—can change the response disproportionately, and in a real system, the context almost never repeats.
• The model changes. New versions, aliases that point to another model, infrastructure changes. A prompt that works today may fail in three months without us changing anything.
• Determinism is not correctness. A deterministic model that is wrong is wrong in the same way every time. Reproducibility helps us debug, not get the right answer.

An agent chains together many calls, each of which depends on the previous ones. If it reads a different file in the first step, everything else starts from a different context, and a small divergence ends up producing a different trajectory. In a chain of agents, the effect multiplies at every boundary.

It gets much worse if the model itself decides which step comes next. If an LLM acts as the orchestrator (“now call the reviewer”, “this is done”), variability moves from the content to the structure: what runs, how many times, and when it stops. We no longer have a process with variable parts. We have a variable process.

The conclusion is not that LLMs are useless. It is that their variability cannot be eliminated; it has to be bounded, and that requires something predictable around them.

Some parts should never depend on a model:

• The graph: which steps exist and in what order.
• Transitions: determined by conditions on data, not by something the agent says.
• Contracts: what goes into each step and what shape its output takes. Anything that fails the contract does not move forward.
• Objective verification: compiling, running tests, running the linter. We do not ask an agent whether the tests pass: we run them.
• Stopping conditions: iterations, time and cost. An unbounded loop is a failure waiting to happen.
• Final outcomes: how the process ended in business terms, not in engine terms.

A graph reaching its end without exceptions tells us nothing about whether the artefact exists, whether it is valid, or whether anyone accepted it. Each path should end in an explicit outcome: accepted, rejected, blocked, or not deliverable. The engine’s generic end state is an implementation detail. The business needs to know how things ended.

In deterministic components, we expect the same result from the same input, and we check that with conventional tests. In probabilistic components, we verify the operating envelope (that the output respects the contract, that the workflow stays within the graph, and that the limits hold) and measure properties with evals against a threshold declared in advance. We are not looking for a perfect system, but one that is observable, falsifiable and testable against explicit contracts.

A deterministic workflow does not mean the orchestrator decides what “ready for development” means or whether a state transition is valid in your process. Those rules belong to the domain. The division of responsibilities that works for me:

• The domain defines meaning and validity.
• The orchestrator executes the sequence and applies the limits.
• Scripts perform deterministic mutations.
• Agents handle bounded semantic decisions.
• People resolve what cannot be automated.

Disclaimer

The details about Conductor and the example refer to version 0.1.42 (October 2026). The project evolves quickly, so it is worth checking them against the current documentation.

Note on the use of AI: written with the help of artificial intelligence tools, drawing on the author’s experience, ideas and judgement. The author directed, reviewed and validated all the content. The opinions, analysis and reflections are the author’s own.

1. Microsoft Conductor: github.com/microsoft/conductor (README, docs/workflow-syntax.md, src/conductor/engine/router.py, src/conductor/engine/limits.py, src/conductor/engine/workflow.py, src/conductor/gates/human.py, src/conductor/config/schema.py, src/conductor/engine/context.py, release v0.1.42, 6 October 2026).
2. Horace He and Thinking Machines Lab, Defeating Nondeterminism in LLM Inference, 10 September 2025.
3. OpenAI Cookbook, How to make your completions outputs reproducible with the new seed parameter.
4. Microsoft Learn, Reproducible output (Azure OpenAI).