R3: Deterministic AI

The core challenge: you want the same input to produce the same output, but LLMs are inherently probabilistic. True bit-for-bit determinism is impossible — but functional determinism (consistent, accurate, verifiable outputs) is achievable. This module shows you how. See R1: Prompt Engineering for the prompting techniques that underpin these strategies.

The Determinism Spectrum

Not all tasks need the same level of consistency. Choose the right target:

Why AI Hallucinates

Hallucination is a feature of autoregressive generation, not a bug. Understanding the mechanics helps you defend against it:

1. Next-token prediction — the model picks the most probable continuation, not the most truthful
2. No knowledge boundary — the model doesn’t know what it doesn’t know
3. Training data noise — conflicting or outdated information in training data
4. Context window pressure — as conversations grow, earlier context gets compressed
5. Sycophancy bias — models tend to agree with the user’s stated position

Key insight: You can’t eliminate hallucination — you can only reduce its frequency and catch it before it reaches users.

The Determinism Toolkit

Control Lever 1: Temperature & Sampling

# Maximum determinism settings
response = client.chat.completions.create(
    model="gpt-4o",
    temperature=0,
    seed=42,
    max_tokens=500,
    messages=[...],
)
# Check: response.system_fingerprint — if it changes, outputs may differ

Control Lever 2: Structured Output

Force the model into a constrained output space where hallucination is structurally impossible:

from pydantic import BaseModel
 
class ClassificationResult(BaseModel):
    category: str  # constrained to enum in JSON Schema
    confidence: float
    reasoning: str
 
response = client.beta.chat.completions.parse(
    model="gpt-4o",
    temperature=0,
    response_format=ClassificationResult,
    messages=[
        {"role": "system", "content": "Classify the support ticket."},
        {"role": "user", "content": "My VM won't start after resizing."},
    ],
)
result = response.choices[0].message.parsed

Control Lever 3: Grounding

Connect the model to real data so it generates from evidence, not memory:

See R2: RAG Architecture for full implementation details.

Multi-Layer Defense Architecture

No single technique is sufficient. Production systems stack multiple layers:

Layer 1: INPUT VALIDATION
  → Sanitize prompts, detect injection attempts, enforce length limits

Layer 2: PROMPT DESIGN
  → System message constraints, role prompting, few-shot examples

Layer 3: GENERATION CONTROLS
  → temperature=0, seed, max_tokens, structured output / JSON mode

Layer 4: GROUNDING
  → RAG retrieval, tool calling, citation requirements

Layer 5: OUTPUT VALIDATION
  → Groundedness check (≥4.0/5.0), content safety filter, schema validation

Layer 6: HUMAN OVERSIGHT
  → Confidence thresholds for escalation, feedback loops, audit logging

Evaluation: Measuring Determinism

FrootAI Play 03 (Deterministic Agent) implements all six defense layers with automated evaluation. Play 01 (Enterprise RAG) applies grounding-focused determinism with groundedness scoring.

Key Takeaways

1. True determinism is impossible — target functional determinism (semantic consistency)
2. Hallucination is inherent to autoregressive generation — defend in depth
3. Stack all six layers — no single technique is sufficient alone
4. Structured output is the strongest single lever — constrain at the decoding level
5. Measure with groundedness scores and consistency checks, not vibes

For orchestration patterns that manage these controls at scale, see O1: Semantic Kernel.