Skip to content

GRA · AI and Financial Model Risk

Spatial financeWhere capital meets hazard, exposure and transition

Model monitoring and drift

A model validated at launch can fail later as customers, products, markets and data pipelines change. Monitoring tracks performance, input distributions and outcomes to detect drift and trigger review. Machine learning models that retrain frequently add their own risks. This page sets out how drift arises, how monitoring detects it, the measures used, and who decides when a model must change or stop.

01تۆمارکردن

Model monitoring and drift on the record.

02پرسیارەکە

Model monitoring and drift: how the risk develops.

Changes over time and review thresholds. The dossier below sets out how the risk develops, what is measured, where the data come from, how it is modelled and who decides.

میکانیزم

Data drift

The distribution of inputs shifts as new customer segments arrive, products change or upstream systems alter fields. The model meets data it was not trained on.

میکانیزم

Concept drift

The relationship between inputs and outcomes changes, for example when an economic shock alters who defaults. Accuracy falls even if inputs look stable.

میکانیزم

Outcomes arrive late

Defaults and claims emerge months or years after decisions, so true performance is known late. Proxies and early indicators fill the gap with uncertainty.

میکانیزم

Retraining adds risk

Frequent retraining can absorb drift, and it can also learn from feedback loops or corrupted data. Each retrained version needs controls proportionate to its use.

03Exposure, loss and capital

What model monitoring and drift means for exposure, loss and capital.

The exposure, loss, scenario and capital views a risk officer, supervisor or ministry of finance would ask for, with the frameworks they report against. Structures follow supervisory and actuarial practice. Values are illustrative: never prices, ratings or advice.

Transmission channels

How a shock in model monitoring and drift reaches credit, market, underwriting, liquidity and operational risk, and feeds back.

Channel structure after the Basel Committee’s work on climate-related risk drivers and transmission channels; links shown are typical paths, not estimated exposures.
NGFS climate scenarios · Network for Greening the Financial SystemIFRS S2 Climate-related Disclosures · ISSBTCFD recommendations · Financial Stability Board TCFDPrinciples for the effective management of climate-related financial risks · Basel Committee on Banking Supervision
04نیشاندەرەکان

What is measured, and how well.

Each indicator carries its unit, its cadence, the kind of source behind it and the evidence label a value would carry. No values are shown: a programme supplies them under its own authority.

نیشاندەریەکەڕیتمSource typeناونیشان
Population stability index by inputdimensionlessمانگانەچاودێریمەزەندەکراو
Performance against validation baselinechange in discrimination or calibrationmonthly or quarterlyچاودێریمەزەندەکراو
Monitoring alerts breaching thresholdscountمانگانەMonitoring recordsچاودێریکراو
Models retrained in periodcountوەرزانەModel recordsDECLARED
Data pipeline incidents affecting modelscountوەرزانەIncident recordsڕاپۆرتکراو
Time from alert to remediationڕۆژper alertGovernance recordsچاودێریکراو
05سەرچاوەکان

Where the evidence comes from.

Named public datasets and institutional records, each with its licence class. Sources by agreement are used only under their terms and never republished.

سەرچاوەWhat it givesLicence class
PRA SS1/23 principles on model monitoringExpectations for ongoing performance monitoringFree, Bank of England terms
SR 11-7 on ongoing monitoring and outcomes analysisUnited States expectationsدۆمەینی گشتی
EBA guidelines on loan origination and monitoringMonitoring of automated credit modelsFree, EU reuse terms
Academic research on dataset shiftMethods for detecting and adapting to driftFree to read, terms per publisher
Bank Pillar 3 model back-testing disclosuresBack-testing results for capital modelsPublic records, terms per bank
06بڕیارەکان

Who decides, and what the evidence serves.

AI and Financial Model Risk prepares evidence. The decisions below are taken by the authorities named, under their own rules, and never by the platform.

کێ بڕیار دەداتپرسیارەکەEvidence this page serves
Model ownersWhen to recalibrate, redevelop or retire a modelMonitoring results against thresholds
Model risk functionsWhat monitoring standards apply by tierPolicy and monitoring evidence
چاودێرانWhether monitoring is adequate for capital and conduct modelsBack-testing and review findings
Data ownersHow upstream changes are controlledPipeline incident records

Outputs this topic produces

  • Guide to data drift and concept drift
  • Method note on monitoring thresholds and alerts
  • Brief on controls for frequently retrained models
  • Questions for model owners on late-arriving outcomes
07Core questions

What experts ask.

Monitoring questions ask how soon a problem will be seen.

  • Which performance measures are monitored, and how often?
  • How quickly are outcomes, such as defaults or claims, observed, and what fills the gap meanwhile?
  • Has the input data distribution shifted, and does it matter?
  • What thresholds trigger review, recalibration or withdrawal, and who set them?
  • How are continuously retrained models governed?
08Mechanisms

Mechanisms and drivers.

Data drift occurs when the distribution of inputs changes, for example a new customer segment or a change in a data source. Concept drift occurs when the relationship between inputs and outcomes changes, for example when an economic shock alters how income predicts repayment. Concept drift is more dangerous and harder to detect, because outcomes may take months or years to observe.

Feedback loops add complexity: a model’s decisions change who becomes a customer, which changes future data. Retraining can restore performance but can also absorb errors or bias, so retrained models need governance of their own.

09Methods and data

How it is studied, and where the methods stop.

Method or dataWhat it showsسنوورەکان
Performance tracking against outcomesAccuracy and calibration over timeOutcomes arrive with delay
Distribution shift statisticsChanges in input dataShift does not always mean performance loss
Benchmark and challenger modelsRelative performanceChallengers share some data limitations
Threshold and alerting frameworksWhen to escalateThresholds involve judgement
11پرسیارەکان

پرسیار و وەڵام.

What is the difference between data drift and concept drift?
Data drift is a change in the distribution of inputs; concept drift is a change in how inputs relate to outcomes. Either can reduce model accuracy.
What is a population stability index?
A measure comparing the distribution of a variable or score between two periods, used to detect shifts in the population a model scores.
Should models retrain automatically?
Automatic retraining can keep models current but needs controls on data quality, performance checks and approval proportionate to the model’s use.
Why is concept drift hard to detect?
Because it involves changes in the relationship between inputs and outcomes, and outcomes such as defaults may only be known long after decisions are made.
Is automatic retraining a solution?
It can help but needs governance: retrained models can absorb errors, bias or temporary conditions, and must still be tested and approved.
Does the platform monitor firms’ models?
No. GRA does not monitor, validate or approve any model.
ڕاپۆرتەکانبەشداربە

لەم لاپەڕەیەوە بەڵگە پێشکەش بکە.

Reports بڵاوکراوەی وەرزیی کۆنسۆرتیۆم ـە. ئەوەی تۆ لەسەر ئەم بابەتە دەیزانیت دەتوانێت لە وەشانی داهاتوودا ببێتە بەڵگە، کەیس یان ڕاستکردنەوە، بە ناوی خۆتەوە و لەژێر CC BY 4.0 بڵاو دەکرێتەوە کاتێک کاری خۆتە.

Global Risks AllianceAI and Financial Model RiskMethods and TechnicalExponential Technology

هەرگیز داتای کەسی، وردەکاریی هەستیاری ژێرخان یان هیچ شتێک مەنێرە کە مافی بڵاوکردنەوەت نییە. بەشدارییەک پێش بەکارهێنان پێداچوونەوەی بۆ دەکرێت؛ ناردنی ماف نابەخشێت بە نووسەرایەتی یان بەڕێوەبردنی ڕاگەیاندن، و پاڵپشتی دارایی هەرگیز دۆزینەوەیەک ناکڕێت.

زیرەکی دەستکرد و مەترسی مۆدێلی دارایی

Bring an exposure or capital question to the Alliance.

زانیاری و پرسیار ئەمڕۆ کراوەن. پرسیارێک ژمارەی ئاماژەی پێدەدرێت و هەنگاوی دواتر دیاری دەکات؛ داواکاری، پارەدان، تەرخانکردنی کۆمپیوتەر و تۆمارکردن تەنیا کاتێک دەکرێنەوە کە تیم، مەرج و توانای پێداچوونەوەیان هەبێت.