Skip to content

GRA · Payments and Market Infrastructure

Spatial financeWhere capital meets hazard, exposure and transition

Operational test methods

Infrastructures and participants test their resilience through scenario tests, failover and recovery tests, threat-led penetration tests and market-wide exercises. Testing shows whether plans work under severe but plausible disruption. This page sets out the main test methods, what each can and cannot show, the records they produce, and who decides what is tested.

01Bhalisa

Operational test methods on the record.

02The question

Operational test methods: how the risk develops.

Failure exercises and recovery assumptions. The dossier below sets out how the risk develops, what is measured, where the data come from, how it is modelled and who decides.

Indlela yokusebenza

Scenario tests check tolerances

Severe but plausible scenarios test whether important services stay within impact tolerances. Scenarios that are too mild give false comfort.

Indlela yokusebenza

Red teams test defences

Threat-led penetration tests use realistic attacker techniques against live systems, under controlled conditions, to find weaknesses.

Indlela yokusebenza

Market-wide exercises test coordination

Exercises across many institutions and authorities test communication, decisions and shared dependencies that single-firm tests miss.

Indlela yokusebenza

Findings must close

Testing produces findings. Without remediation tracked to closure, the same weaknesses appear in the next test.

03Beka iliso

RTGS operations console.

The screen a team would use to follow this subject: its measures, states and decisions. Names are examples and values are illustrative.

RTGS operations console

Operational test methods · RTGS system · Business day 193

  • Value settled today
    107local bnActive
  • Payments queued
    11624Beka iliso
  • Availability, % this month
    99.9Okuqhelekileyo
  • Participants below liquidity floor
    2Alert
Queue building at peakGRA · Payments and Market Infrastructure

Intraday liquidity, system

Incident and failover log

  1. 09:26INCParticipant P-12 message delay
  2. 10:43LSMLiquidity saving cycle released 64 payments
  3. 14:00CUTCustomer payment cut-off
  4. Q1TESTContingency site failover in 93 min
RTGS operations console: the screen’s structure, terms and states for Operational test methods.Example names and illustrative values; a design, not a record or a live system.
04Izalathisi

What is measured, and how well.

Each indicator carries its unit, its cadence, the kind of source behind it and the evidence label a value would carry. No values are shown: a programme supplies them under its own authority.

IndicatorUnitCadenceSource typeIleyibhile
Threat-led penetration tests completedcountannualPublic recordsPublished statistics
Failover tests to secondary sitescount and success rateannualOperator recordsDECLARED
Market-wide exercises heldcountannualAuthority publicationsPublished statistics
Findings remediated within deadlineshare of findingsperiodicRemediation trackingDECLARED
Services tested against impact toleranceshare of important servicesannualSelf-assessmentsDECLARED
Recovery time achieved in testshoursper testTest reportsKUQWALASELWE
05Exposure, loss and capital

What operational test methods means for exposure, loss and capital.

The exposure, loss, scenario and capital views a risk officer, supervisor or ministry of finance would ask for, with the frameworks they report against. Structures follow supervisory and actuarial practice. Values are illustrative: never prices, ratings or advice.

Transmission channels

How a shock in operational test methods reaches credit, market, underwriting, liquidity and operational risk, and feeds back.

Channel structure after the Basel Committee’s work on climate-related risk drivers and transmission channels; links shown are typical paths, not estimated exposures.
NGFS climate scenarios · Network for Greening the Financial SystemIFRS S2 Climate-related Disclosures · ISSBTCFD recommendations · Financial Stability Board TCFDPrinciples for the effective management of climate-related financial risks · Basel Committee on Banking Supervision
06Imithombo

Where the evidence comes from.

Named public datasets and institutional records, each with its licence class. Sources by agreement are used only under their terms and never republished.

UmthomboWhat it givesLicence class
ECB TIBER-EU framework and annual reportsRed-team testing framework and summary findingsFree, ECB terms
Bank of England CBEST and SIMEX reportsThreat-led testing and market-wide exercisesFree, Bank of England terms
DORA regulatory technical standards on threat-led penetration testingEU testing requirementsFree, EU reuse terms
G7 Cyber Expert Group fundamental elementsPrinciples for threat-led testing and third-party riskFree to read, terms to check
Hamilton series exercise reports and similar public summariesCross-sector exercise findingsFree to read, terms to check
07Decisions

Who decides, and what the evidence serves.

Payments and Market Infrastructure prepares evidence. The decisions below are taken by the authorities named, under their own rules, and never by the platform.

Who decidesThe questionEvidence this page serves
AuthoritiesWhich institutions must run threat-led tests and exercisesTest frameworks and sector risk evidence
BoardsWhich services and scenarios to test and how to fix findingsTest results and remediation status
Testing teams and providersHow to design and conduct tests safelyFramework requirements and methods
Exercise coordinatorsWhich scenarios to run across the marketThreat evidence and previous findings

Outputs this topic produces

  • Guide to operational resilience test types
  • Method note on severe but plausible scenario design
  • Brief on market-wide exercises and their lessons
  • Questions for boards on remediation tracking
08Core questions

What the industry asks.

Plans and recovery targets are only as reliable as the tests that verify them. Many infrastructures meet their recovery objectives in planned tests but have never faced a realistic, destructive scenario involving several institutions at once. Credible testing exposes assumptions, such as that backups are clean, that providers will be available or that staff can work from alternate sites, before a real crisis tests them.

  • Do tests use severe but plausible scenarios, including destructive cyber attacks and provider failures?
  • Do tests cover the whole chain, including participants, providers and linked infrastructures, rather than one system in isolation?
  • Which recovery assumptions have been tested, and which remain untested?
  • How are lessons from tests turned into changes, and how is progress tracked?
09Mechanisms

How the infrastructure works and fails.

Testing ranges from single-system checks to sector-wide simulations.

Continuity tests

Failover and recovery

Switching to backup sites and systems to check recovery times. Planned tests may not reflect real conditions.

Threat-led testing

Simulated attacks

Testing by teams that simulate real attackers’ techniques, under frameworks established by authorities in several jurisdictions.

Sector exercises

Many institutions together

Exercises that simulate disruption across infrastructures, participants and authorities, revealing interdependencies.

A common weakness revealed by exercises is the gap between technical recovery and business recovery. Systems may be restored within target, but reconciling positions, communicating with participants and resuming normal operations can take much longer. Tests that measure the full return to service, including data reconciliation across institutions, give a more honest picture of resilience.

10Methods and evidence

How the evidence is built, and its limits.

  1. 01

    Test design

    Scenario selection, scope and success criteria. Test results depend heavily on design.

  2. 02

    Recovery metrics

    Measurement of recovery time and data loss against objectives, applying reliability and resilience engineering.

  3. 03

    Uphononongo emva kwesenzo

    Structured review of what worked and what failed, with tracked follow-up.

  4. 04

    Simulation models

    Models of payment flows under disruption, applying network science and contagion, to complement live tests.

12Imibuzo

Imibuzo neempendulo.

What is threat-led penetration testing?
Testing that simulates the tactics of real threat actors against an institution’s live production systems, based on threat intelligence, under a controlled framework such as TIBER-EU.
What is a market-wide exercise?
A simulation involving many institutions and authorities to test sector-level response, communication and recovery from a shared disruption.
What is an impact tolerance?
The maximum tolerable level of disruption to an important business service, used as a benchmark in scenario testing.
Does GRA run tests on payment systems?
No. GRA does not operate, connect to or test any live payment system, and it assesses no individual institution.
Why are sector-wide exercises important?
Because many failure modes only appear when several institutions and providers are affected at once, which single-institution tests cannot reveal.
What is the Universe payment outage exercise?
A simulated learning exercise within the annual Universe programme, governed by GRF. It touches no live payment system.
IingxeloNikela

Faka ubungqina obuvela kweli phepha.

Reports is the Consortium’s quarterly publication. What you know about this subject can become evidence, a case or a correction in the next edition, credited to you and published under CC BY 4.0 where it is your own work.

Global Risks AlliancePayments and Market InfrastructureStrategic SystemsScenario and Stress-Test
Iintlawulo Neziseko Zophuhliso Zemarike

Bring an exposure or capital question to the Alliance.

Information and enquiries are open today. An enquiry receives a reference and names the next step; applications, payments, compute allocations and registrations open only when their team, terms and review capacity exist.