Actuarial and Quantitative Risk
The Global Risks Alliance (GRA)
Quantitative models for pricing, reserving, capital and risk must be shown fit for purpose before they are relied on. Validation tests data, methods, assumptions, implementation and outcomes, and is carried out by people independent of development under professional and supervisory standards. This page sets out how quantitative model validation works, where it falls short, the evidence it uses and who decides.
GRA · In development · service level R0
Quantitative model validation is the independent testing of pricing, reserving, capital and risk models to judge whether they are fit for purpose, covering data, methods, assumptions, code and outcomes. Validation functions, model owners, supervisors and audit committees hold the decisions.
Which places drive loss in a severe year, and how far it reaches into reinsurance. Illustrative design values.
Back-testing, benchmarking and challenger models. The dossier below sets out how the risk develops, what is measured, where the data come from, how it is modelled and who decides.
Data quality, conceptual soundness, implementation, assumptions and outcomes all need testing. Checking only outputs misses errors that cancel or hide.
Comparing predictions with outcomes takes years for long-tail and rare risks, so validation relies on other tests in the meantime.
Validators must understand the methods and have authority to require changes. Weak independence turns validation into confirmation.
Errors in spreadsheets and code have caused large losses. Implementation testing, version control and reproducibility are part of validation.
Each term as this page uses it. Where a standard-setting body defines the term, the wording follows that body.
| Term | Definition |
|---|---|
| Conceptual soundness | The quality of a model's design and assumptions for its intended use. |
| Reserve run-off | The development of claims on past reserves, used to test reserving accuracy. |
| Implementation testing | Checking that a model's code or spreadsheet correctly carries out its method. |
Each indicator carries its unit, its cadence, the kind of source behind it and the evidence label a value would carry. No values are shown: a programme supplies them under its own authority.
| Indicator | Unit | Cadence | Source type | Label |
|---|---|---|---|---|
| Models validated in cycle | share by tier | annual | Validation records | DECLARED |
| Validation findings by severity | count | annual | Validation records | DECLARED |
| Actual-to-expected results | ratio | annual | Experience monitoring | ESTIMATED |
| Reserve run-off deviation | percent | annual | Statutory reserve development | OFFICIAL |
| Implementation errors found | count | annual | Validation and audit records | DECLARED |
| Models with reproducible code and data | share | periodic | Model documentation | DECLARED |
The topic as a computational graph: named sources feed the methods, an evidence review gates the outputs, and the decisions stay with the authorities that hold them.
Fig. 02Nodes carry their type, two metric rows and a state; connectors show what feeds what; dashed frames name the owners; the timeline is the cadence. Design of the pipeline, not a record of a run.
Named public datasets and institutional records, each with its licence class. Sources by agreement are used only under their terms and never republished.
| Source | What it gives | Licence class |
|---|---|---|
| IAA ISAP on models (ISAP 1A) | Model governance standard for actuaries | Free, IAA terms |
| Actuarial Standards Board ASOP on modeling | United States standard for actuarial models | Free, ASB terms |
| Financial Reporting Council Technical Actuarial Standards | UK standards including models | Free, FRC terms |
| EIOPA guidelines on internal model validation | Supervisory expectations | Free, EU reuse terms |
| Statutory loss development data | Reserve run-off for back-testing | Public records, terms per supervisor |
Actuarial and Quantitative Risk prepares evidence. The decisions below are taken by the authorities named, under their own rules, and never by the platform.
| Who decides | The question | Evidence this page serves |
|---|---|---|
| Validation functions | Whether models are fit for purpose | Test results and findings |
| Chief actuaries and model owners | Which findings to remediate and how | Findings and impact analysis |
| Supervisors | Whether validation meets standards | Validation reports |
| Audit committees | Whether validation is independent and effective | Governance records |
Validation frameworks typically cover conceptual soundness, data quality, implementation, outcomes analysis and ongoing monitoring. Independence from model development is essential, because developers tend to share the model's assumptions. Validation is proportionate to the materiality and complexity of the model.
Back-testing compares predictions with realized outcomes, for example the number of times losses exceeded a value-at-risk estimate. For rare-event models, back-testing has little power, because the events of interest seldom occur. Benchmarking compares results with alternative models, industry studies or external data. Challenger models, built with different assumptions or methods, reveal how much results depend on modelling choices.
Sensitivity analysis and stress testing explore how outputs change with inputs and assumptions, identifying the drivers that matter most. Reproducibility, the ability to re-run a model and obtain the same result from the same inputs, is a basic requirement that versioned code and data make possible.
Governance gives validation its force. Findings must reach people with authority to restrict or stop a model's use, and model inventories, tiering by materiality and tracking of open findings help ensure that validation results lead to action rather than sitting in reports.
Comparing predictions with realized outcomes; weak for rare events.
Comparing with alternatives to reveal model dependence.
Identifying drivers and failure conditions.
Limits: validation can show that a model is not obviously wrong, but it cannot prove that a model is right, especially for rare events and changing conditions.
The current service level is R0: information and enquiries. An enquiry receives a reference and names the next step; applications, payments, compute allocations and registrations open only when their team, terms and review capacity exist.