Model monitoring
Model drift: what it is and how to detect it
A plain explainer on why a model can get worse after launch, how drift detection works, and what to decide before the first alert fires.
Model drift is when a model gets worse after launch because live data no longer matches the data it was trained on. Drift detection compares live inputs and outputs against a baseline and alerts you when the gap passes a threshold. An alert means check the model; real accuracy needs outcomes to compare against.
plain answer
What is model drift?
Model drift is the gradual loss of a model's usefulness after it goes live, because the data it sees in production no longer looks like the data it learned from. Google Cloud's model monitoring documentation gives an example: a model that predicts customer lifetime value is trained on spending patterns, but as customer habits change, the factors that predict spending change too. The features and values used in training may then no longer be relevant for predictions today. Google calls that deviation in the data drift. The model itself has not been edited. The world around it has moved, which is why a model that tested well at launch can still need re-evaluation or retraining later.
plain answer
Model drift detection
Model drift detection compares what the model sees now with a baseline, and raises an alert when the gap passes a threshold you set. The baseline is either the training data or recent production data. Azure Machine Learning defaults to recent production data and lets you point to training data instead, and Databricks suggests a baseline that reflects what the model was trained and validated on, ideally its validation set. The comparison uses a distance measure on each input column: Google Cloud uses Jensen Shannon divergence for numerical features, and Azure Machine Learning uses a Jensen Shannon distance for numerical features and a Pearson's chi squared test for categorical ones. Run the check on a schedule, and decide before it fires who receives the alert and what happens next.
plain answer
What is the difference between data drift, prediction drift and accuracy?
Input drift means the data coming into the model has changed. Output drift means the model's answers have changed. Azure Machine Learning tracks data drift, prediction drift and data quality as built in signals, and Google Cloud measures input feature drift and output inference drift. A drift alert is a prompt to check, not proof that the model is wrong. Accuracy needs real outcomes: Azure Machine Learning states that without ground truth data you cannot perform model performance monitoring, and that your application has to collect that data as it becomes available. So use drift signals as the early warning, and outcome comparison as the confirmation.
kriv role
What does Kriv AI do about model drift?
Kriv AI helps teams decide what to monitor, who owns each alert, and what triggers a review, a retrain or a rollback, as part of AI readiness and governance work. This page is an explainer. It recommends no vendor product, gives no legal advice, and does not claim that any regulation or standard is met by drift monitoring. If you want a second opinion on a model already in production, a short discovery call is where we scope it, and pricing is set after that call.
Sources
Cited sources
- Google Cloud documentation, Introduction to Model Monitoring (page updated 2026-11-07), read 2026-11-08
- Microsoft Learn, Monitor model performance in production, Azure Machine Learning (page updated 2026-02-11), read 2026-11-08
- Databricks documentation, Data profiling (page updated 2026-09-11), read 2026-11-08
Straight answers
Frequently asked questions about Model drift: what it is and how to detect it
What is model drift in machine learning?
Model drift is a change between the data a model was trained on and the data it sees in production, which can make its predictions less useful over time. Google Cloud's documentation describes it as a deviation in the data, for example when customer habits change and the features used in training stop being relevant. Drift is a prompt to re-evaluate or retrain the model.
How do you detect model drift?
Pick a baseline, usually training data or recent production data, then compare each input column and the model's outputs against it on a schedule, using a distance measure and a threshold. When the gap passes the threshold, send an alert to a named owner. The three vendor guides listed under Sources each describe this baseline and threshold pattern. These vendor guides cover tabular and classic machine learning models; checking the output of a language model needs different tests.
Can you detect model drift without ground truth labels?
You can detect drift in inputs and outputs without labels, because that compares distributions. You cannot measure accuracy without them. Azure Machine Learning states that without ground truth data you cannot perform model performance monitoring, and that your application must collect that data as it becomes available.
How often should drift monitoring run?
On a regular schedule or as needed. Google Cloud's documentation says monitoring jobs can be run as needed or on a regular schedule, and Azure Machine Learning sets up a monitoring job that runs on a regular schedule. The right interval depends on how quickly your data changes and how costly a wrong answer is, which is a decision for your own risk owners.
Talk to the team that would do the work
Bring your requirements to a working session with the person who'll actually deliver.
Book a Discovery Call