A-P05 · TOPIC · operational-test-methodsGRA dossier · service level R0
GRAPayments and Market InfrastructureIn development
Topic dossier · Payments and Market Infrastructure

Operational test methods

Infrastructures and participants test their resilience through scenario tests, failover and recovery tests, threat-led penetration tests and market-wide exercises. Testing shows whether plans work under severe but plausible disruption. This page sets out the main test methods, what each can and cannot show, the records they produce, and who decides what is tested.

The interactive map needs JavaScript; the layer list and sources are below.

Layers 2

Activity and exposure, for this page

Latest available

NASA GIBS / EOSDIS · ASTER GDEM (METI, NASA) · Blue Marble (NASA) · © OpenStreetMap contributors (ODbL) · Natural Earth

Loading the latest NASA imagery.

Drag to move. Ctrl or ⌘ and scroll, pinch or the buttons to zoom; hover for the values under the pointer; reset returns to the region.

For operational test methods, in GRA's spatial-finance reading, the map opens on night lights and population density with built-up and settlement extent from NASA GIBS, framed on the United Kingdom and Ireland; every other NASA layer, live feed and forecast waits in the layer list.Satellite and model products as NASA publishes them, each with its own date; indicators, not assessments of any place; claim ceiling INTEGRATED.

Data: NASA GIBS · ASTER GDEM · Blue Marble · © OpenStreetMap contributors · Natural Earth · GDACS · USGS · CIESIN/Columbia, NASA SEDAC HBASESources and licences

  • We acknowledge the use of imagery provided by services from NASA's Global Imagery Browse Services (GIBS), part of NASA's Earth Science Data and Information System (ESDIS). Colours redrawn from the published colour map; values averaged into H3 cells where shown (modified). NASA open data
  • We acknowledge the use of imagery provided by services from NASA's Global Imagery Browse Services (GIBS), part of NASA's Earth Science Data and Information System (ESDIS). Colours redrawn from the published colour map; values averaged into H3 cells where shown (modified). NASA open data
  • VIIRS Black Marble night-time lights, NASA Goddard Space Flight Center; distributed by the NASA Level-1 and Atmosphere Archive and Distribution System (LAADS DAAC); imagery via NASA GIBS. Colours redrawn from the published colour map; values averaged into H3 cells where shown (modified). NASA open data
  • Center for International Earth Science Information Network (CIESIN), Columbia University. 2018. Gridded Population of the World, Version 4 (GPWv4): Population Density, Revision 11. NASA Socioeconomic Data and Applications Center (SEDAC). Licensed under CC BY 4.0. Colours redrawn from the published colour map; values averaged into H3 cells where shown (modified). CC BY 4.0
  • VIIRS near-real-time products (flood, land surface temperature, snow cover, corrected reflectance), NASA LANCE; imagery via NASA GIBS. Colours redrawn from the published colour map; values averaged into H3 cells where shown (modified). NASA open data
  • MODIS land products (vegetation indices, gross primary production, land cover), NASA LP DAAC and LANCE; imagery via NASA GIBS. Colours redrawn from the published colour map; values averaged into H3 cells where shown (modified). NASA open data
  • ASTER GDEM is a product of METI and NASA (ASTER Global Digital Elevation Model V003, NASA LP DAAC); imagery via NASA GIBS. Colours redrawn from the published colour map; values averaged into H3 cells where shown (modified). NASA open data
  • Made with Natural Earth. Free vector and raster map data @ naturalearthdata.com. Public domain
  • © OpenStreetMap contributors (ODbL), via NASA GIBS (coastlines, boundaries, roads and place names). Colours redrawn from the published colour map; values averaged into H3 cells where shown (modified). ODbL
  • Blue Marble, NASA Earth Observatory; imagery via NASA GIBS. Colours redrawn from the published colour map; values averaged into H3 cells where shown (modified). NASA open data
  • Context on the globe and flat map: coastlines, boundaries, roads and place names from NASA GIBS reference layers (OpenStreetMap data, ODbL, © OpenStreetMap contributors); Blue Marble imagery as an optional base; 3D terrain and the elevation read-out from the ASTER GDEM elevation colour index (5 m to 200 m classes, exaggerated for legibility). Boundaries are drawn as published and imply no position on any territory.
  • Globe: MapLibre GL JS 6.9.1, BSD-3-Clause, © MapLibre contributors.
  • Live: GDACS (EC JRC / UN OCHA) Events are indicative, not official warnings.
  • Live: USGS Events are indicative, not official warnings.
  • Context: CIESIN/Columbia, NASA SEDAC HBASE. Center for International Earth Science Information Network (CIESIN), Columbia University, and NASA Socioeconomic Data and Applications Center (SEDAC): Global Human Built-up And Settlement Extent (HBASE) 2010; Seto, Guneralp and Hutyra 2012, Global Grid of Probabilities of Urban Expansion to 2030; Global Urban Heat Island (UHI) Data Set, 2013.
  • Optional layers (35 sources in the layer list: live, forecast, observed and context) print their provider credit on the map and here when switched on; forecasts are model output and derived indices are estimates.

Layers and dates

  • Night-time lights, daily: VIIRS Black Marble (VNP46A2), NASA LAADS DAAC · 12 Oct 2026
  • Population density, 2020: Population: CIESIN, Columbia University, Gridded Population of the World v4.11 (2018), NASA SEDAC, CC BY 4.0 · static
  • Flood water, 3-day composite: VIIRS Global Flood Product, NASA LANCE · 14 Oct 2026
  • True colour: VIIRS NOAA-20 corrected reflectance, NASA LANCE · 13 Oct 2026
  • Land surface temperature, day, 8-day composite: VIIRS NOAA-20 Level 3, 8-day, NASA LANCE · 14 Sep 2026
  • Land cover type (IGBP), yearly: MODIS Terra and Aqua land cover (MCD12Q1), NASA LP DAAC · 2024

01In brief

The short answer.

Operational resilience test methods are the ways infrastructures and participants check that critical services survive disruption: scenario tests against impact tolerances, failover and recovery tests, threat-led penetration tests and market-wide exercises. Authorities, boards, testing teams and exercise coordinators decide what is tested.

02Console

RTGS operations console, in outline.

The working screen behind this subject: its structure, terms and states. Names are placeholders and values illustrative.

RTGS operations console

Operational test methods · RTGS system · Business day 193

  • Value settled today
    107local bnActive
  • Payments queued
    11624Watch
  • Availability, % this month
    99.9Normal
  • Participants below liquidity floor
    2Alert
Queue building at peakGRA · Payments and Market Infrastructure

Intraday liquidity, system

Incident and failover log

  1. 09:26INCParticipant P-12 message delay
  2. 10:43LSMLiquidity saving cycle released 64 payments
  3. 14:00CUTCustomer payment cut-off
  4. Q1TESTContingency site failover in 93 min
RTGS operations console: the screen's structure, terms and states for Operational test methods.Placeholder names and illustrative values; a design, not a record or a live system.
03The question

Operational test methods: how the risk develops.

Failure exercises and recovery assumptions. The dossier below sets out how the risk develops, what is measured, where the data come from, how it is modelled and who decides.

Mechanism

Scenario tests check tolerances

Severe but plausible scenarios test whether important services stay within impact tolerances. Scenarios that are too mild give false comfort.

Mechanism

Red teams test defences

Threat-led penetration tests use realistic attacker techniques against live systems, under controlled conditions, to find weaknesses.

Mechanism

Market-wide exercises test coordination

Exercises across many institutions and authorities test communication, decisions and shared dependencies that single-firm tests miss.

Mechanism

Findings must close

Testing produces findings. Without remediation tracked to closure, the same weaknesses appear in the next test.

04Key terms

Terms used on this page.

Each term as this page uses it. Where a standard-setting body defines the term, the wording follows that body.

TermDefinition
Severe but plausible scenarioA disruption scenario extreme enough to test resilience yet realistic enough to occur.
Failover testA test that switches operations to a backup system or site to confirm it works.
Red teamA group that simulates attackers to test an organization's defences.
05Indicators

What is measured, and how well.

Each indicator carries its unit, its cadence, the kind of source behind it and the evidence label a value would carry. No values are shown: a programme supplies them under its own authority.

IndicatorUnitCadenceSource typeLabel
Threat-led penetration tests completedcountannualAuthority recordsOFFICIAL
Failover tests to secondary sitescount and success rateannualOperator recordsDECLARED
Market-wide exercises heldcountannualAuthority publicationsOFFICIAL
Findings remediated within deadlineshare of findingsperiodicRemediation trackingDECLARED
Services tested against impact toleranceshare of important servicesannualSelf-assessmentsDECLARED
Recovery time achieved in testshoursper testTest reportsOBSERVED
06Canvas

From sources to decisions.

The topic as a computational graph: named sources feed the methods, an evidence review gates the outputs, and the decisions stay with the authorities that hold them.

Pipeline canvas · Operational test methods as a run graphDesign, not a record
Operational test methods: payment from instruction to finalityA payment instruction is validated and screened, queued and settled in central bank money when liquidity allows; settlement is final and reconciled; a contingency route takes over if the primary system fails. Design of the process, not a record of a run.

Fig. 02Nodes carry their type, two metric rows and a state; connectors show what feeds what; dashed frames name the owners; the timeline is the cadence. Design of the pipeline, not a record of a run.

07Sources

Where the evidence comes from.

Named public datasets and institutional records, each with its licence class. Sources by agreement are used only under their terms and never republished.

SourceWhat it givesLicence class
ECB TIBER-EU framework and annual reportsRed-team testing framework and summary findingsFree, ECB terms
Bank of England CBEST and SIMEX reportsThreat-led testing and market-wide exercisesFree, Bank of England terms
DORA regulatory technical standards on threat-led penetration testingEU testing requirementsFree, EU reuse terms
G7 Cyber Expert Group fundamental elementsPrinciples for threat-led testing and third-party riskFree to read, terms to check
Hamilton series exercise reports and similar public summariesCross-sector exercise findingsFree to read, terms to check
08Decisions

Who decides, and what the evidence serves.

Payments and Market Infrastructure prepares evidence. The decisions below are taken by the authorities named, under their own rules, and never by the platform.

Who decidesThe questionEvidence this page serves
AuthoritiesWhich institutions must run threat-led tests and exercisesTest frameworks and sector risk evidence
BoardsWhich services and scenarios to test and how to fix findingsTest results and remediation status
Testing teams and providersHow to design and conduct tests safelyFramework requirements and methods
Exercise coordinatorsWhich scenarios to run across the marketThreat evidence and previous findings

Outputs this topic produces

  • Guide to operational resilience test types
  • Method note on severe but plausible scenario design
  • Brief on market-wide exercises and their lessons
  • Questions for boards on remediation tracking
09Core questions

What the industry asks.

Plans and recovery targets are only as reliable as the tests that verify them. Many infrastructures meet their recovery objectives in planned tests but have never faced a realistic, destructive scenario involving several institutions at once. Credible testing exposes assumptions, such as that backups are clean, that providers will be available or that staff can work from alternate sites, before a real crisis tests them.

  • Do tests use severe but plausible scenarios, including destructive cyber attacks and provider failures?
  • Do tests cover the whole chain, including participants, providers and linked infrastructures, rather than one system in isolation?
  • Which recovery assumptions have been tested, and which remain untested?
  • How are lessons from tests turned into changes, and how is progress tracked?
10Mechanisms

How the infrastructure works and fails.

Testing ranges from single-system checks to sector-wide simulations.

Continuity tests

Failover and recovery

Switching to backup sites and systems to check recovery times. Planned tests may not reflect real conditions.

Threat-led testing

Simulated attacks

Testing by teams that simulate real attackers' techniques, under frameworks established by authorities in several jurisdictions.

Sector exercises

Many institutions together

Exercises that simulate disruption across infrastructures, participants and authorities, revealing interdependencies.

A common weakness revealed by exercises is the gap between technical recovery and business recovery. Systems may be restored within target, but reconciling positions, communicating with participants and resuming normal operations can take much longer. Tests that measure the full return to service, including data reconciliation across institutions, give a more honest picture of resilience.

11Methods and evidence

How the evidence is built, and its limits.

  1. 01

    Test design

    Scenario selection, scope and success criteria. Test results depend heavily on design.

  2. 02

    Recovery metrics

    Measurement of recovery time and data loss against objectives, applying reliability and resilience engineering.

  3. 03

    After-action review

    Structured review of what worked and what failed, with tracked follow-up.

  4. 04

    Simulation models

    Models of payment flows under disruption, applying network science and contagion, to complement live tests.

13Questions

Questions and answers.

What is threat-led penetration testing?
Testing that simulates the tactics of real threat actors against an institution's live production systems, based on threat intelligence, under a controlled framework such as TIBER-EU.
What is a market-wide exercise?
A simulation involving many institutions and authorities to test sector-level response, communication and recovery from a shared disruption.
What is an impact tolerance?
The maximum tolerable level of disruption to an important business service, used as a benchmark in scenario testing.
Does GRA run tests on payment systems?
No. GRA does not operate, connect to or test any live payment system, and it assesses no individual institution.
Why are sector-wide exercises important?
Because many failure modes only appear when several institutions and providers are affected at once, which single-institution tests cannot reveal.
What is the Nexus Universe payment outage exercise?
A simulated learning exercise within the annual Nexus Universe programme, governed by GRF. It touches no live payment system.
Payments and Market Infrastructure

Bring your expertise to this field.

The current service level is R0: information and enquiries. An enquiry receives a reference and names the next step; applications, payments, compute allocations and registrations open only when their team, terms and review capacity exist.

Skip to toolbar