Skip to content

GRA · Actuarial and Quantitative Risk

Spatial financeWhere capital meets hazard, exposure and transition

Quantitative model validation

Quantitative models for pricing, reserving, capital and risk must be shown fit for purpose before they are relied on. Validation tests data, methods, assumptions, implementation and outcomes, and is carried out by people independent of development under professional and supervisory standards. This page sets out how quantitative model validation works, where it falls short, the evidence it uses and who decides.

01تۆمارکردن

Quantitative model validation on the record.

02پرسیارەکە

Quantitative model validation: how the risk develops.

Back-testing, benchmarking and challenger models. The dossier below sets out how the risk develops, what is measured, where the data come from, how it is modelled and who decides.

میکانیزم

Validation covers the whole model

Data quality, conceptual soundness, implementation, assumptions and outcomes all need testing. Checking only outputs misses errors that cancel or hide.

میکانیزم

Back-testing needs time

Comparing predictions with outcomes takes years for long-tail and rare risks, so validation relies on other tests in the meantime.

میکانیزم

Independence and skill

Validators must understand the methods and have authority to require changes. Weak independence turns validation into confirmation.

میکانیزم

Code is part of the model

Errors in spreadsheets and code have caused large losses. Implementation testing, version control and reproducibility are part of validation.

03Spatial finance

Exposure, hazard and the exceedance curve.

Which places drive loss in a severe year, and how far it reaches into reinsurance. Illustrative design values.

Coastal exposure under cyclone and earthquake zones, beside the loss curve it produces.Illustrative design values, not a model output or a price.
04نیشاندەرەکان

What is measured, and how well.

Each indicator carries its unit, its cadence, the kind of source behind it and the evidence label a value would carry. No values are shown: a programme supplies them under its own authority.

نیشاندەریەکەڕیتمSource typeناونیشان
Models validated in cycleshare by tierساڵانەValidation recordsDECLARED
Validation findings by severitycountساڵانەValidation recordsDECLARED
Actual-to-expected resultsratioساڵانەExperience monitoringمەزەندەکراو
Reserve run-off deviationpercentساڵانەStatutory reserve developmentئامارە بڵاوکراوەکان
Implementation errors foundcountساڵانەValidation and audit recordsDECLARED
Models with reproducible code and datashareperiodicModel documentationDECLARED
05شبیهسازی دووانە و زنجیرەیی

Trace a shock in quantitative model validation to the balance sheet.

Assembled from the Nexus Standards area profile Insurance and reinsurance: the shocks it names, the systems and sectors that carry the loss, the controls that dampen it and the data that observe it. Pick an initiating shock and step through the cascade to see which exposures it reaches and where losses could concentrate. The Map view places the run on an example geography with open data. Every weight is a stated assumption. An exercise, not a forecast, a valuation, a rating or investment advice. To model a place of your own choosing, open the full System Builder.

مەترسیدەرەکان

  • Tropical cyclone
  • بوومەلەرزە
  • Ice-jam flood
  • ئاگری دارستان
  • Storm surge (set off downstream)
  • Intense rainfall (set off downstream)
  • River flood (set off downstream)
  • Flash flood (set off downstream)
  • Dangerous surf (set off downstream)
  • پچڕانی کارەبای فراوان (لە خوارەوە دەستپێدەکات)
  • ئاوارەبوونی بەکۆمەڵ (لە خوارەوە دەستپێدەکات)
  • Coastal flood (set off downstream)
  • Aftershock sequence (set off downstream)
  • Fault surface rupture (set off downstream)
  • Soil liquefaction (set off downstream)
  • Slope slide (set off downstream)
  • Rockfall (set off downstream)
  • Tsunami (set off downstream)

کۆنترۆڵەکان

  • NS-0 دەستوور و بنەماکان
  • NS-3 UNIەکانی تۆماری مەترسی (ڕێزمان، خشتەی گواستنەوەی کۆن، دەسەڵاتی تەرخانکردن)
  • NS-9 UNIی ناوەڕۆکی (دامەزراندنەکان، نموونەکان، نموونەکانی پڕۆگرام)
  • NS-10 سیاسەت و بڕگەکان (پابەندبوونە ئامێر-خوێندراوەکان: پێناسە، پابەندکردن، هەڵسەنگاندن و جێبەجێکردن)
  • NS-11 سەلماندن و چاودێری (بەڵگە، گواهییەکان، چاودێریی گونجان لەگەڵ zero-trust)
  • NS-14 پابەندبوون و هاوتاکردن لەگەڵ ستانداردی دەرەکی
  • NRS-F3 بیمە، مەترسیی کارەسات و کەلێنەکانی پاراستن
  • NRS-H113 خێزانی مەترسیدەر: ناسکیی سیستەمی دارایی (NHS-1 soc.financial-system)
  • NRS-U17 Universe platform profile: Insurance Nexus
  • NRS-H71 خێزانی مەترسیدەر: کارەساتە کەونی و کاتی قووڵەکان (NHS-1 planet.deep-time)
  • NRS-H65 خێزانی مەترسیدەر: پێشێلکردنی سنوورەکانی گۆی زەوی (NHS-1 planet.boundary)
  • NRS-F8 گواستنەوەی مەترسیی پارامێتری و لەسەر بنەمای نیشاندەر

چاودێری

  • OECD Data Explorer (incl. Creditor Reporting System, insurance and pension statistics) (listed for reference)
  • Insurance statistics (listed for reference)
  • USGS earthquake GeoJSON summary feeds (listed for reference)
  • Direct economic loss attributed to disasters in relation to global gross domestic product (GDP)
  • Direct economic loss attributed to disasters in relation to global gross domestic product
  • (a) Number of commercial bank branches per 100,000 adults and (b) number of automated teller machines (ATMs) per 100,000 adults

جوگرافیای نموونە

  • ئەوروپای ڕۆژئاوا: 152 خانەی شەشپاڵوو
  • ڕێژەی بارین، نیو کاتژمێری ڕابردوو (IMERG)، NASA GIBS
  • ڕووناکییەکانی شەو ئەمشەو، پچڕانی کارەبا (VIIRS Day/Night Band)، NASA GIBS
  • ئاوی ڕووی زەوی لە ڕێگەی هەورەوە، 30 م (ڕادارەی OPERA DSWx-S1)، NASA GIBS
  • Carbon monoxide, smoke transport (AIRS), NASA GIBS
06سەرچاوەکان

Where the evidence comes from.

Named public datasets and institutional records, each with its licence class. Sources by agreement are used only under their terms and never republished.

سەرچاوەWhat it givesLicence class
IAA ISAP on models (ISAP 1A)Model governance standard for actuariesFree, IAA terms
Actuarial Standards Board ASOP on modellingUnited States standard for actuarial modelsFree, ASB terms
Financial Reporting Council Technical Actuarial StandardsUK standards including modelsFree, FRC terms
EIOPA guidelines on internal model validationSupervisory expectationsFree, EU reuse terms
Statutory loss development dataReserve run-off for back-testingPublic records, terms per supervisor
07Exposure, loss and capital

What quantitative model validation means for exposure, loss and capital.

The exposure, loss, scenario and capital views a risk officer, supervisor or ministry of finance would ask for, with the frameworks they report against. Structures follow supervisory and actuarial practice. Values are illustrative: never prices, ratings or advice.

Exposure and loss at return periods

How losses from quantitative model validation grow with severity, gross and net of reinsurance, against capital.

1 لە 101 لە 501 لە 1001 لە 250ناپوختnet of reinsurance800
ماوەی گەڕانەوەAnnual probabilityزیانی ناپوختزیانی پوختNet, capital = 100
1 لە 100.10141010
1 لە 500.02332121
1 لە 1000.01462727
1 لە 2500.004683939
تێکڕای زیانی ساڵانە3.12.22.2
Illustrative loss index for a example portfolio (capital = 100): the exceedance-probability form insurers and supervisors read. Not data on any institution, not a price or a rating.
NGFS climate scenarios · Network for Greening the Financial SystemIFRS S2 Climate-related Disclosures · ISSBTCFD recommendations · Financial Stability Board TCFDPrinciples for the effective management of climate-related financial risks · Basel Committee on Banking Supervision
08بڕیارەکان

Who decides, and what the evidence serves.

Actuarial and Quantitative Risk prepares evidence. The decisions below are taken by the authorities named, under their own rules, and never by the platform.

کێ بڕیار دەداتپرسیارەکەEvidence this page serves
Validation functionsWhether models are fit for purposeTest results and findings
Chief actuaries and model ownersWhich findings to remediate and howFindings and impact analysis
چاودێرانWhether validation meets standardsValidation reports
Audit committeesWhether validation is independent and effectiveGovernance records

Outputs this topic produces

  • Guide to validating actuarial and quantitative models
  • Method note on reserve back-testing
  • Brief on implementation risk and reproducibility
  • Questions for boards on validation independence
09Core questions

The questions experts ask.

  • Is the model’s theory and design appropriate for its purpose?
  • Is it implemented correctly, and do results reproduce?
  • How do its predictions compare with outcomes that were not used to build it?
  • How do results compare with alternative models and external benchmarks?
  • Under which conditions would the model break down?
10Mechanisms

What validation examines.

Validation frameworks typically cover conceptual soundness, data quality, implementation, outcomes analysis and ongoing monitoring. Independence from model development is essential, because developers tend to share the model’s assumptions. Validation is proportionate to the materiality and complexity of the model.

Back-testing compares predictions with realised outcomes, for example the number of times losses exceeded a value-at-risk estimate. For rare-event models, back-testing has little power, because the events of interest seldom occur. Benchmarking compares results with alternative models, industry studies or external data. Challenger models, built with different assumptions or methods, reveal how much results depend on modelling choices.

Sensitivity analysis and stress testing explore how outputs change with inputs and assumptions, identifying the drivers that matter most. Reproducibility, the ability to re-run a model and obtain the same result from the same inputs, is a basic requirement that versioned code and data make possible.

Governance gives validation its force. Findings must reach people with authority to restrict or stop a model’s use, and model inventories, tiering by materiality and tracking of open findings help ensure that validation results lead to action rather than sitting in reports.

11Methods and data

Methods and their limits.

Back-testing

شیکاریی ئەنجامەکان

Comparing predictions with realised outcomes; weak for rare events.

Benchmarking

Challenger models

Comparing with alternatives to reveal model dependence.

هەستیاری

شیکاری هەستیاری

Identifying drivers and failure conditions.

Limits: validation can show that a model is not obviously wrong, but it cannot prove that a model is right, especially for rare events and changing conditions.

13پرسیارەکان

پرسیار و وەڵام.

What does validation check?
Data, conceptual soundness, assumptions, implementation and outcomes, to judge whether a model is fit for its intended use.
What is reserve back-testing?
Comparing past reserve estimates with how claims actually developed, to measure reserving accuracy and bias.
Why does code matter for validation?
Because implementation errors in spreadsheets and code can give wrong results even when the method is sound.
Does GRA validate models?
No. GRA does not validate, approve or sign off any model. It publishes research on validation methods.
Can a validated model still fail?
Yes. Validation reduces the chance of undetected errors but cannot rule out failure under conditions not seen before.
Why are challenger models useful?
Because they show how much results depend on modelling choices, which a single model cannot reveal.
ڕاپۆرتەکانبەشداربە

لەم لاپەڕەیەوە بەڵگە پێشکەش بکە.

Reports بڵاوکراوەی وەرزیی کۆنسۆرتیۆم ـە. ئەوەی تۆ لەسەر ئەم بابەتە دەیزانیت دەتوانێت لە وەشانی داهاتوودا ببێتە بەڵگە، کەیس یان ڕاستکردنەوە، بە ناوی خۆتەوە و لەژێر CC BY 4.0 بڵاو دەکرێتەوە کاتێک کاری خۆتە.

Global Risks AllianceActuarial and Quantitative RiskMethods and TechnicalFinance and Insurance Readiness

هەرگیز داتای کەسی، وردەکاریی هەستیاری ژێرخان یان هیچ شتێک مەنێرە کە مافی بڵاوکردنەوەت نییە. بەشدارییەک پێش بەکارهێنان پێداچوونەوەی بۆ دەکرێت؛ ناردنی ماف نابەخشێت بە نووسەرایەتی یان بەڕێوەبردنی ڕاگەیاندن، و پاڵپشتی دارایی هەرگیز دۆزینەوەیەک ناکڕێت.

مەترسی ئەکچوەری و بڕی

Bring an exposure or capital question to the Alliance.

زانیاری و پرسیار ئەمڕۆ کراوەن. پرسیارێک ژمارەی ئاماژەی پێدەدرێت و هەنگاوی دواتر دیاری دەکات؛ داواکاری، پارەدان، تەرخانکردنی کۆمپیوتەر و تۆمارکردن تەنیا کاتێک دەکرێنەوە کە تیم، مەرج و توانای پێداچوونەوەیان هەبێت.