With the introduction of APRA’s CPS 230 standard, resilience has shifted from paperwork to performance. Scenario testing is now essential: not just a best practice, but a regulatory expectation.
How do you run a scenario that does more than tick the box? How can you turn stress testing into stakeholder confidence? And what role do your service providers play in the bigger picture of resilience?
David Tattam and I unpacked these questions and more in our latest thought leadership webinar: The road to CPS 230: Bringing resilience to life through scenarios.
There were so many great questions during the session that we couldn’t get to them all, so we’ve answered them here.
Watch the webinar on demand and learn how to build a structured, repeatable scenario program that meets CPS 230 and strengthens your operational resilience:
Questions
Q1: Can you tell us more about involving Line 1 in the scenario planning?
Q2: How did you derive the resilience rating in Protecht ERM?
Q3: Can you talk about the relationship between testing for critical operations and broader BCM, testing etc?
Q4: Are the SLAs from third parties part of the scenario?
Q5: Should scenario testing test the critical operations at its most vulnerable time?
Q6: In a scenario that's familiar, do participants rely too heavily on their lived experience?
Q7: Has the concept of ‘severe but plausible’ ever been tested legally?
Q8: Is the 'plausible' test making an assumption that the event will happen, rather than adjusting for probability?
Q9: What about VaR (Value at Risk) analysis?
Q1: You said it was better to involve Line 1 in the scenario planning. We have quite often developed them to really test their knowledge from a crisis perspective but CPS230 brings a slightly different focus.
While it will depend on the culture and maturity in each entity, the focus on critical operations (especially understanding how they are delivered and can be disrupted) pushes this more into the realm of Line 1. They own those operations, and should be responsible for ensuring they can be maintained within tolerance levels during disruption. If they push back and say that continuity is handled by someone else, there may be a culture gap you need to bridge.
Q2: How did you derive the resilience rating in Protecht ERM?
By default, the resilience rating for each critical operation is a qualitative assessment made by the owner of the critical operation, based on the related data they have available, such as the number of vulnerabilities tracked in process maps, results of recent scenario exercises, and assessments over related controls.
We also work with customers to adapt their own methodology to determining the resilience rating.
Q3: Can you talk about the relationship between testing for critical operations and broader BCM, testing etc? Where do non-critical operations fit in: do we need to cover them at all now?
From a CPS 230 perspective, scenarios are to assess ability to remain within tolerance levels for your critical operations.
Nothing prevents you from including activities that are not critical operations in your testing program. Some disruptions might have limited impact on external stakeholders, but have a high financial impact on the entity. You should probably keep testing them because it is good risk management, rather than from a compliance perspective.
It's possible that some will keep BCM and operational resilience exercising under CPS 230 separate, but we recommend bringing them together under one umbrella.
Q4: Are the SLAs from third parties part of the scenario?
They can be, depending on the scenario. I have a couple of perspectives here.
Firstly, service providers could support your critical operations through providing a service or assets, and the SLA’s related to those could be pushed out. These can form a good basis for sensitivity analysis – how far would that SLA need to be pushed before tolerance levels are breached? Is that plausible? I’d assume for most, the SLA would need to be significantly breached before they stress tolerance levels.
Secondly, there are SLAs of service providers who support your business continuity response. These should also be tested. What happens if they don’t respond? Given they are part of response, there is probably less latitude with variation.
You can consider the above SLAs in isolation for a single service provider, but when developing your scenarios you might consider how SLAs of multiple service providers might be stretched due to external factors. Consider single points of failure or concentration risks among providers.
Q5: Should scenario testing test the critical operations at its most vulnerable time, for example immediately before a monthly payment run versus the other days of the month?
This is a common approach. APRA expect entities to remain within tolerance levels, regardless of when or how disruption might occur. We suggest capturing assumptions when developing scenarios, and this is where you would capture the ‘worst time this could occur’ narratives or details.
Q6: I love the reference to the 'giggle test' and have been guilty of a Zombie Apocalypse scenario. I understood that there were challenges in a scenario that was familiar - as participants rely too heavily on their lived experience which can be limiting? Any ideas?
This is a great callout regarding bias: I’ve heard “that’s never happened here before” and its partner in crime “that could never happen here”. If you are in a facilitator role, this is where you are ideally challenging participants and stretching their thinking about what is possible.
During the webinar, we suggested capturing why a scenario is plausible when it is developed. Even if the exact scenario hasn’t occurred, you can often refer to similar incidents, or where components of the scenario have occurred and it is plausible that they could occur at the same time. Referencing real examples can help bring it to life for the audience.
Q7: Has the concept of ‘severe but plausible’ ever been tested legally, i.e. has a court ruled on what it means?
Not to our knowledge, though of course its interpretation for CPS 230 and operational risk may differ from previous statements and standards which have been typically aligned with financial risks.
Q8: I assume we test for plausible, and we are making an assumption that the event will happen versus adjusting risk rating for probability?
The wording is plausible, not probable. Likelihood takes a backseat. This is the same as setting tolerance levels: they need to be set based on the material adverse impact to stakeholders, not how likely the tolerance levels are to be breached. The more severe a scenario becomes, the more unlikely it is almost by definition.
When developing your scenarios, or exercising them, you may want to keep pushing the envelope until it becomes implausible, perhaps following the ‘giggle test’ that David mentioned in the webinar.
Q9: What about VaR (Value at Risk) analysis?
While we touched on some quantitative measures, in hindsight we focused more on material adverse impact to stakeholders. However the definitions when setting tolerance levels also includes an entity's role in the financial system.
While it would typically be used for capital requirements and financial risk, you might consider adapting a Value at Risk measure in two ways:
- Where an operational disruption would influence the value of those capital holdings separately or combined with financial risks. That might be a stretch too far – but it brings to mind Citibank’s fat finger failure I wrote about last year
- Using VaR applied to aggregated operational risk directly (otherwise known as a loss exceedance curve). Determine what level of direct financial losses or accrual (as opposed to change in capital valuation) would affect your role in the financial system or ability to remain a going concern, assess whether that level is plausible, and then what scenarios (collection of variables) in the model lead to that outcome.
I expect most to start with traditional narrative-based business continuity exercises, but they can certainly be informed by more quantitative models, and processes in place for exit and recovery planning.
Conclusions and next steps for your organisation
Resilience isn’t just about recovery: it’s about readiness.
CPS 230 raises the bar for operational resilience, but with the right tools and strategies, it’s an opportunity to drive meaningful improvement. Whether you're refining your scenario testing framework or just beginning your CPS 230 journey, this webinar offers the insight to move forward with confidence.
Want to see how Protecht can help you manage scenario testing, business continuity, and third-party risk in a single platform? Request a demo of Protecht’s CPS 230 software solution and see how we turn compliance into capability:
