GRA · Actuarial and Quantitative Risk
Spatial financeWhere capital meets hazard, exposure and transition
Quantitative model validation
Quantitative models for pricing, reserving, capital and risk must be shown fit for purpose before they are relied on. Validation tests data, methods, assumptions, implementation and outcomes, and is carried out by people independent of development under professional and supervisory standards. This page sets out how quantitative model validation works, where it falls short, the evidence it uses and who decides.
Quantitative model validation on the record.
Quantitative model validation: how the risk develops.
Back-testing, benchmarking and challenger models. The dossier below sets out how the risk develops, what is measured, where the data come from, how it is modelled and who decides.
Validation covers the whole model
Data quality, conceptual soundness, implementation, assumptions and outcomes all need testing. Checking only outputs misses errors that cancel or hide.
Back-testing needs time
Comparing predictions with outcomes takes years for long-tail and rare risks, so validation relies on other tests in the meantime.
Independence and skill
Validators must understand the methods and have authority to require changes. Weak independence turns validation into confirmation.
Code is part of the model
Errors in spreadsheets and code have caused large losses. Implementation testing, version control and reproducibility are part of validation.
Exposure, hazard and the exceedance curve.
Which places drive loss in a severe year, and how far it reaches into reinsurance. Illustrative design values.
What is measured, and how well.
Each indicator carries its unit, its cadence, the kind of source behind it and the evidence label a value would carry. No values are shown: a programme supplies them under its own authority.
| Indicator | Unit | Cadence | Source type | Label |
|---|---|---|---|---|
| Models validated in cycle | share by tier | annual | Validation records | DECLARED |
| Validation findings by severity | count | annual | Validation records | DECLARED |
| Actual-to-expected results | ratio | annual | Experience monitoring | IMEKADIRIWA |
| Reserve run-off deviation | percent | annual | Statutory reserve development | Published statistics |
| Implementation errors found | count | annual | Validation and audit records | DECLARED |
| Models with reproducible code and data | share | periodic | Model documentation | DECLARED |
Trace a shock in quantitative model validation to the balance sheet.
Assembled from the Nexus Standards area profile Insurance and reinsurance: the shocks it names, the systems and sectors that carry the loss, the controls that dampen it and the data that observe it. Pick an initiating shock and step through the cascade to see which exposures it reaches and where losses could concentrate. The Map view places the run on an example geography with open data. Every weight is a stated assumption. An exercise, not a forecast, a valuation, a rating or investment advice. To model a place of your own choosing, open the full System Builder.
Majanga
- Tropical cyclone
- Earthquake
- Ice-jam flood
- Moto wa nyika
- Storm surge (set off downstream)
- Intense rainfall (set off downstream)
- River flood (set off downstream)
- Flash flood (set off downstream)
- Dangerous surf (set off downstream)
- Widespread power outage (set off downstream)
- Mass displacement (set off downstream)
- Coastal flood (set off downstream)
- Aftershock sequence (set off downstream)
- Fault surface rupture (set off downstream)
- Soil liquefaction (set off downstream)
- Slope slide (set off downstream)
- Rockfall (set off downstream)
- Tsunami (set off downstream)
Systems and couplings
- Bima na bima mtawanyo
- Benki Muunganiko
- Taasisi za fedha za maendeleo Muunganiko
- Wadau wa kibinadamu na asasi za kiraia Muunganiko
- Kilimo na matumizi ya ardhi Muunganiko
Controls
NS-0Katiba na kanuniNS-3UNI za rekodi za hatari (sarufi, ulinganishi wa urithi, mamlaka ya ugawaji)NS-9UNI za kimuktadha (utekelezaji, matukio, matukio ya programu)NS-10Sera na vifungu (wajibu unaosomeka na mashine: ufafanuzi, ufungamanishaji, tathmini na utekelezaji)NS-11Uthibitishaji na ufuatiliaji (ushahidi, uthibitisho, uchunguzi wa ulinganifu usio na imani)NS-14Ulinganifu na upatanisho na viwango vya njeNRS-F3Bima, hatari za maafa makubwa na mapengo ya ulinziNRS-H113Familia ya majanga: Udhaifu wa mfumo wa fedha (NHS-1 soc.financial-system)NRS-U17Wasifu wa jukwaa la Universe: Insurance NexusNRS-H71Familia ya majanga: Maafa ya muda mrefu wa kijiolojia na ya kianga (NHS-1 planet.deep-time)NRS-H65Familia ya majanga: Kuvukwa kwa mipaka ya sayari (NHS-1 planet.boundary)NRS-F8Uhamishaji wa hatari kwa vigezo na kwa faharasa
Observation
- OECD Data Explorer (incl. Creditor Reporting System, insurance and pension statistics) (listed for reference)
- Insurance statistics (listed for reference)
- USGS earthquake GeoJSON summary feeds (listed for reference)
- Direct economic loss attributed to disasters in relation to global gross domestic product (GDP)
- Direct economic loss attributed to disasters in relation to global gross domestic product
- (a) Number of commercial bank branches per 100,000 adults and (b) number of automated teller machines (ATMs) per 100,000 adults
Example geography
- Western Europe: 152 hexagonal cells
- Rain rate, last half hour (IMERG), NASA GIBS
- Night lights tonight, outages (VIIRS Day/Night Band), NASA GIBS
- Surface water through cloud, 30 m (OPERA DSWx-S1 radar), NASA GIBS
- Carbon monoxide, smoke transport (AIRS), NASA GIBS
Where the evidence comes from.
Named public datasets and institutional records, each with its licence class. Sources by agreement are used only under their terms and never republished.
| Chanzo | What it gives | Licence class |
|---|---|---|
| IAA ISAP on models (ISAP 1A) | Model governance standard for actuaries | Free, IAA terms |
| Actuarial Standards Board ASOP on modelling | United States standard for actuarial models | Free, ASB terms |
| Financial Reporting Council Technical Actuarial Standards | UK standards including models | Free, FRC terms |
| EIOPA guidelines on internal model validation | Supervisory expectations | Free, EU reuse terms |
| Statutory loss development data | Reserve run-off for back-testing | Public records, terms per supervisor |
What quantitative model validation means for exposure, loss and capital.
The exposure, loss, scenario and capital views a risk officer, supervisor or ministry of finance would ask for, with the frameworks they report against. Structures follow supervisory and actuarial practice. Values are illustrative: never prices, ratings or advice.
Exposure and loss at return periods
How losses from quantitative model validation grow with severity, gross and net of reinsurance, against capital.
| Return period | Annual probability | Gross loss | Net loss | Net, capital = 100 |
|---|---|---|---|---|
| 1 in 10 | 0.10 | 14 | 10 | 10 |
| 1 in 50 | 0.02 | 33 | 21 | 21 |
| 1 in 100 | 0.01 | 46 | 27 | 27 |
| 1 in 250 | 0.004 | 68 | 39 | 39 |
| Average annual loss | 3.1 | 2.2 | 2.2 |
Who decides, and what the evidence serves.
Actuarial and Quantitative Risk prepares evidence. The decisions below are taken by the authorities named, under their own rules, and never by the platform.
| Nani anaamua | The question | Evidence this page serves |
|---|---|---|
| Validation functions | Whether models are fit for purpose | Test results and findings |
| Chief actuaries and model owners | Which findings to remediate and how | Findings and impact analysis |
| Supervisors | Whether validation meets standards | Validation reports |
| Audit committees | Whether validation is independent and effective | Governance records |
Outputs this topic produces
- Guide to validating actuarial and quantitative models
- Method note on reserve back-testing
- Brief on implementation risk and reproducibility
- Questions for boards on validation independence
The questions experts ask.
- Is the model’s theory and design appropriate for its purpose?
- Is it implemented correctly, and do results reproduce?
- How do its predictions compare with outcomes that were not used to build it?
- How do results compare with alternative models and external benchmarks?
- Under which conditions would the model break down?
What validation examines.
Validation frameworks typically cover conceptual soundness, data quality, implementation, outcomes analysis and ongoing monitoring. Independence from model development is essential, because developers tend to share the model’s assumptions. Validation is proportionate to the materiality and complexity of the model.
Back-testing compares predictions with realised outcomes, for example the number of times losses exceeded a value-at-risk estimate. For rare-event models, back-testing has little power, because the events of interest seldom occur. Benchmarking compares results with alternative models, industry studies or external data. Challenger models, built with different assumptions or methods, reveal how much results depend on modelling choices.
Sensitivity analysis and stress testing explore how outputs change with inputs and assumptions, identifying the drivers that matter most. Reproducibility, the ability to re-run a model and obtain the same result from the same inputs, is a basic requirement that versioned code and data make possible.
Governance gives validation its force. Findings must reach people with authority to restrict or stop a model’s use, and model inventories, tiering by materiality and tracking of open findings help ensure that validation results lead to action rather than sitting in reports.
Methods and their limits.
Outcomes analysis
Comparing predictions with realised outcomes; weak for rare events.
Challenger models
Comparing with alternatives to reveal model dependence.
Sensitivity analysis
Identifying drivers and failure conditions.
Limits: validation can show that a model is not obviously wrong, but it cannot prove that a model is right, especially for rare events and changing conditions.
How it is studied, and what the methods cannot say.
- 01Actuarial mathematicsReserving with development triangles · Premium build-up · Annuity valuationApplied here
- 02Uncertainty quantification and model validationReserve ranges · Parameter and model uncertainty · Sensitivity rankingApplied here
- 03Probability and statisticsGeneralised linear models in pricing · Bayesian credibility · Goodness-of-fit testingApplied here
- 04High-performance, cloud and edge computingMonte Carlo simulation · Version control and reproducibility · End-user computing controlsApplied here
Questions and answers.
What does validation check?
What is reserve back-testing?
Why does code matter for validation?
Does GRA validate models?
Can a validated model still fail?
Why are challenger models useful?
Changia ushahidi kutoka ukurasa huu.
Reports ni chapisho la kila robo mwaka la Muungano. Unachojua kuhusu somo hili kinaweza kuwa ushahidi, kisa au sahihisho katika toleo lijalo, kikitajwa kwa jina lako na kuchapishwa chini ya CC BY 4.0 pale ambapo ni kazi yako mwenyewe.
- EmailGateway GW-GRASend it from this pageEvidence, a correction, a case, a method, a review, a translation or profile input. It arrives tagged with this page, so the editor who owns it receives it.
- ZenodoOpen depositDeposit it and submit to the communityPublish your own record on Zenodo with your ORCID iD, then submit it to the Reports community there. Curators review it; the DOI stays yours.
- ReportsGlobal Risks AllianceSee every way to contributeThe contribution types, how each is reviewed, what credit you receive, the licence and what is never accepted.
Usitume kamwe data binafsi, maelezo nyeti ya miundombinu wala chochote ambacho huna haki ya kukishiriki. Mchango hupitiwa kabla ya kutumika; kuutuma hakutoi haki za uandishi wala za uhariri, na udhamini haununui matokeo.
Bring an exposure or capital question to the Alliance.
Information and enquiries are open today. An enquiry receives a reference and names the next step; applications, payments, compute allocations and registrations open only when their team, terms and review capacity exist.
Legal documents.
Maps, data and figures on this site are shown for context and general information. They are not advice, forecasts or warnings; the limits of the information are set out in the Disclaimers and Data Notices.