Credit and fraud model evaluation
Models are tested on out-of-time samples that respect how data arrive in practice, with discrimination, calibration, stability and group disparities reported together rather than as a single accuracy figure.
GRA · Fintech Nexus
Spatial financeWhere capital meets hazard, exposure and transition
Machine learning in fintech
Artificial intelligence and machine learning are both tools and objects of study in this field. Fintech Nexus examines how models used in credit, fraud, trading, compliance and customer service are evaluated and governed, and uses machine learning with care where it helps find patterns in incident, fraud and transaction data.
01Register
Machine learning in fintech on the record.
Method recordSTEM method
- MethodArtificial intelligence and machine learning
- Applied inFintech Nexus
- Data4 inputs
- Checks4 validation steps
- Canonical pageSTEM Commons
02Application
What artificial intelligence and machine learning does here.
Artificial intelligence and machine learning develop algorithms that learn patterns from data to classify, predict or generate outputs. They are increasingly used for hazard detection, forecasting and decision support, and are themselves a source of risk. In capital markets and investment, cyber and digital resilience and digital public infrastructure, machine learning can scale detection and forecasting across interconnected systems. Its own failure modes must be weighed when automated outputs inform consequential decisions. The canonical description is on the STEM Commons.
Drift and challenger monitoring
Live model outputs are compared with validation baselines and with a simpler challenger model, so degradation is detected and investigated before it changes outcomes for customers.
Generative AI failure testing
Language models used in customer service, research or code are tested for incorrect answers, prompt injection, data leakage and unauthorised actions, with results logged against the workflow in which they run.
03Monitor
Model training.
The screen a team would use to follow this method: its measures, states and decisions. Names are examples and values are illustrative.
Model training
Fintech · Machine learning in fintech · Run R-7209
Drift on one sliceGRA · Fintech Nexus
Evaluation by slice
| All | 0.88 | 0.03 | 5810 | Pass |
| Region A | 0.86 | 0.08 | 847 | Pass |
| Region B | 0.71 | 0.06 | 728 | Review |
| Rare class | 0.56 | 0.09 | 110 | Below bar |
Method card
- Method
- Artificial intelligence and machine learning
- Split
- time-based, no leakage
- Features
- 27
- Label
- MODELLED
Population stability
04Data
What the method consumes.
Datasets and records used by artificial intelligence and machine learning on this platform, with the role each plays and the label its values carry. Sources are listed in the source register.
| Data | Role | Label |
|---|---|---|
| Labelled fraud and default outcomes | Training and out-of-time evaluation | OBSERVED |
| Firm model inventories | Scope and risk tiering | DECLARED |
| Supervisory AI surveys | Adoption and governance patterns | REPORTED |
| AI incident databases | Failure typologies | REPORTED |
05Stress test
How a shock to assets becomes a system loss.
The equity the system loses after interbank contagion, against the initial hit, as leverage, interconnection and recovery rates change, the core of a macroprudential stress test. Teaching model with stated illustrative values, not a prediction.
Controls
- Fall in external assets
- 2.0%
- Leverage (assets over equity)
- 10 times
- Interbank share of assets
- 10%
- Recovery on interbank claims
- 40%
- Banks in the system
- 20
- Core banks
- 20% of banks
Readouts
- Equity lost to the initial shock
- 18.0%
- Equity lost after contagion
- 45.0%
- Added by contagion (DebtRank)
- 27.0 points
- Banks in default
- 0
Numbers, methods and sources
| Round | System equity lost (%) | Banks in default |
|---|---|---|
| 0 | 18.00 | 0 |
| 1 | 28.80 | 0 |
| 2 | 35.28 | 0 |
| 3 | 39.17 | 0 |
| 4 | 41.50 | 0 |
| 5 | 42.90 | 0 |
| 6 | 43.74 | 0 |
| 7 | 44.24 | 0 |
| 8 | 44.55 | 0 |
| 9 | 44.73 | 0 |
| 10 | 44.84 | 0 |
| 11 | 44.90 | 0 |
| 12 | 44.94 | 0 |
| 13 | 44.96 | 0 |
| 14 | 44.98 | 0 |
| 15 | 44.99 | 0 |
| 16 | 44.99 | 0 |
| 17 | 45.00 | 0 |
| 18 | 45.00 | 0 |
| 19 | 45.00 | 0 |
| 20 | 45.00 | 0 |
| 21 | 45.00 | 0 |
| 22 | 45.00 | 0 |
| 23 | 45.00 | 0 |
| 24 | 45.00 | 0 |
| 25 | 45.00 | 0 |
| 26 | 45.00 | 0 |
| 27 | 45.00 | 0 |
| 28 | 45.00 | 0 |
| 29 | 45.00 | 0 |
| 30 | 45.00 | 0 |
Methods
- DebtRank Battiston and others (2012), Scientific Reports 2:541
- Differential DebtRank Bardoscia and others (2015), PLOS ONE 10
- Contagion in financial networks Gai and Kapadia (2010), Proc. R. Soc. A 466
- Clearing in financial networks Eisenberg and Noe (2001), Management Science 47
- Uncertainty fan (parameter Monte Carlo) Spiegelhalter, Pearson and Short (2011), Visualising uncertainty about the future, Science 333
- One-at-a-time sensitivity (tornado) Saltelli and others (2008), Global Sensitivity Analysis: The Primer, Wiley
Data
No observed data enters this model: it runs on the teaching values set by the controls, to be replaced with a programme's own evidence.
Across the six pillars, led by Nexus Consortium
- Experiments Labs
- Built in Fintech Nexus in Foundry
- Tested at Fintech Nexus in Universe
- Codified in Finance, Insurance and the Economy in Standards
- Published in Fintech Nexus in Reports
- Taught in Faculty of Finance and Economics in Academy
06Validation
How results are checked.
Models are evaluated on held-out data that respects spatial, temporal and group structure. Robustness, calibration and fairness are tested alongside accuracy.
| Check | How it is done |
|---|---|
| Out-of-time validation | Performance is reported only on periods after the training window |
| Fairness and stability tests | Disparity measures and population stability are reported beside accuracy, with the chosen metrics justified |
| Independent validation | A team not involved in development reviews the data, assumptions and results |
| Adversarial testing | Models exposed to adaptive adversaries are tested against evasion and manipulation |
Performance degrades when deployment conditions differ from training data. Models can encode biases and fail without clear warning on rare events.
07Exposure, loss and capital
What machine learning in fintech means for exposure, loss and capital.
The exposure, loss, scenario and capital views a risk officer, supervisor or ministry of finance would ask for, with the frameworks they report against. Structures follow supervisory and actuarial practice. Values are illustrative: never prices, ratings or advice.
Transmission channels
How a shock in machine learning in fintech reaches credit, market, underwriting, liquidity and operational risk, and feeds back.
NGFS climate scenarios · Network for Greening the Financial SystemIFRS S2 Climate-related Disclosures · ISSBTCFD recommendations · Financial Stability Board TCFDPrinciples for the effective management of climate-related financial risks · Basel Committee on Banking Supervision
08Connections
Topics that rely on this method.
09Stewardship
Who maintains the method.
Artificial Intelligence and Autonomous Systems stewards artificial intelligence and machine learning for the Consortium and reviews how it is applied across platforms.
10Connected
Where this page leads across the Consortium.
11Questions
Questions and answers.
What are the limits of artificial intelligence and machine learning in this field?
Performance degrades when deployment conditions differ from training data. Models can encode biases and fail without clear warning on rare events.
How are results checked?
Models are evaluated on held-out data that respects spatial, temporal and group structure. Robustness, calibration and fairness are tested alongside accuracy.
Are the values on this page real?
The data names, roles and checks are real descriptions of practice. The canvas shows a design of the method's use, not a run; no result is reported here.
Where else is this method applied?
The canonical method page lists every platform that applies it.
ReportsContribute
Contribute evidence from this page.
Reports is the Consortium's quarterly publication. What you know about this subject can become evidence, a case or a correction in the next edition, credited to you and published under CC BY 4.0 where it is your own work.
Global Risks AllianceFintech NexusExponential TechnologyFinance and Insurance Readiness
- EmailGateway GW-GRASend it from this pageEvidence, a correction, a case, a method, a review, a translation or profile input. It arrives tagged with this page, so the editor who owns it receives it.
- ZenodoOpen depositDeposit it and submit to the communityPublish your own record on Zenodo with your ORCID iD, then submit it to the Reports community there. Curators review it; the DOI stays yours.
- ReportsGlobal Risks AllianceSee every way to contributeThe contribution types, how each is reviewed, what credit you receive, the licence and what is never accepted.
Never send personal data, sensitive infrastructure detail or anything you have no right to share. A contribution is reviewed before use; sending it confers no authorship or editorial rights, and sponsorship never buys a finding.
Fintech Nexus
Bring an exposure or capital question to the Alliance.
Information and enquiries are open today. An enquiry receives a reference and names the next step; applications, payments, compute allocations and registrations open only when their team, terms and review capacity exist.
Legal
Legal documents.
Maps, data and figures on this site are shown for context and general information. They are not advice, forecasts or warnings; the limits of the information are set out in the Disclaimers and Data Notices.