Model Drift

Model drift is the change in an AI system's output behavior over time: the same prompt, the same workflow, quietly producing different — often worse — results than it did at launch. The causes range from provider model updates and deprecations to shifts in the data flowing through the system.

Why it matters

Marketing teams build workflows against a model's behavior today, then assume permanence. Models don't offer it. Providers update versions, adjust defaults, and retire endpoints; a prompt tuned precisely to one model's quirks can degrade meaningfully on its successor. The insidious part is silence: nothing errors, nothing alerts — the tone slides, the format loosens, the scoring shifts — and it's discovered weeks later when a human finally reads the output stream closely. For teams running AI at volume, drift is an operational certainty to be managed, not an anomaly.

How it's used

Drift management is a pillar of prompt ops: keep a small evaluation set — a dozen representative inputs with known-good outputs — and re-run it whenever the provider announces a model change, and on a monthly schedule regardless. Pin model versions where your platform allows it, so upgrades happen on your schedule with testing, not silently. And keep humans sampling production output even from "solved" workflows — the sampling tier of human-in-the-loop design exists substantially because of drift.

Related terms

Prompt ops · Prompt library — a versioned prompt library with test dates is the basic drift-detection instrument.