Oversight of the internal control environment is a critical component for compliance professionals. To achieve a level of comfort, many turn to control frameworks which outline common practices to implement those controls. In some cases, alignment with those frameworks can become a burden and 'something we have to do', rather than linking to business objectives.
In Protecht’s Control frameworks and compliance: Bridging the gap to risk management webinar, we explored how you can build an effective controls management process that aligns with control frameworks and provides assurance to compliance and risk professionals alike.
We had great feedback from our attendees, including the questions answered below. If you missed the webinar live, then you can view it on demand here:
Questions
1. Controls as policies
2. Operating effectiveness
3. Operational versus strategic controls
4. Quantitative testing
5. Key controls
6. Control design frameworks
7. Inherent and residual risk
8. Keeping stakeholders on the same path
9. Risk in Motion and tolerances
10. Obligations versus risk management
11. Linking processes with controls
12. Details of evidence
12. Reasonable versus limited assurance
13. Controls and frameworks example
14. Too much self
15. Evidence as a test-type
16. Risk versus compliance revisited
17. Mapping to multiple frameworks
18. Controls and quality assurance
19. Smaller organizations and the 3 Lines model
1. Controls as policies
When the business is first identifying controls, a common one selected is Policy XYZ, or Procedure XYZ. Sometimes these are referred to as directive controls, but other frameworks such as COSO seem to reject them outright as controls. I think Line 1 may find it confusing, so how do you recommend Line 2 practitioners guide the business around whether policies and procedures are controls or not?
I accept that internal policies & procedures are compliance requirements, but they are also principally controls. Would you report testing outcomes as control failures or non-compliance?
At Protecht, we typically recommend limiting the use of policies and procedures as controls. Usually what is documented within the policy or procedure are one or more controls. We recommend identifying the specific control (and identifying whether it is preventive, detective or reactive) or controls. If the control is simply listed as XYS Policy, this may be sufficient if you have further documentation about its objective and how its design and operation can be tested. If the test is limited to ‘yes we have one, and it was reviewed in the last 12 months’ that doesn’t provide assurance that the policy says the right things, or that it is being applied.
2. Operating effectiveness
Do you test a control’s effectiveness in case the control design is ineffective/inadequate?
I assume this means testing the operating effectiveness. Usually these are not tested if the design is ineffective, as that needs to be addressed first. However, if the design is partially effective, it may still provide some assurance that those parts are operating, particularly if the design problem cannot be solved quickly.
3. Operational versus strategic controls
Do you see a difference in how we manage controls and test operational risk versus strategic risk?
The key concepts are the same. For example, you might have a detective control that is assessing for major competitive shifts in your market which might force a change in strategy. Sometimes controls over strategic risk may be more abstract in some cases, or considered ‘soft’ controls such as culture. As much as practical, if you are implementing controls over strategic risk, consider how you would assess their design and operation, as this will help influence their design so they can be assessed more objectively.
4. Quantitative testing
How do you go about putting quantitative measures to the ability of the control to reduce likelihood or reduce impact? Do you test before you put in design effectiveness, and how much do you test to know if the measures are reasonably accurate?
It may depend on the type of control and the rigor you want to apply. Implementation of a control may include investment which requires a business case, where assumptions are made about the reduction in risk, translated into dollar terms. Sometimes these assumptions are validated with data, perhaps based on industry data, or reviewing the effect on similar controls being applied elsewhere in the organization. Assessing the value of controls is much bigger than we can get into in this response, but once implemented you may be able to track the effect of controls using data and statistical methods, though this is more relevant for high frequency events.
5. Key controls
How does Protect define or determine what is a "key" control?
What is the decision making process you use for defining key controls?
Key controls are those that you wouldn’t (or shouldn’t) perform the activity without, or are non-negotiable. Non-key/medium controls are those that are important, but negotiable.
6. Control design frameworks
Do you know of a definition or framework an organization can use to define what a good control looks like (i.e. a framework that assists with rating the effectiveness of a control)?
While the assessment of design and operating effectiveness is commonly accepted practice (or required by regulators), in our experience we haven’t found a single standard or expectation for how those assessments are completed. We include further details on assessing design and operation in our Protecht Academy Controls Design & Assurance course.
7. Inherent and residual risk
Is there a definition for inherent and residual risk?
Inherent risk is commonly defined as the level of risk before controls are applied/considered, and residual risk is the level of risk after controls are applied. Some organizations only consider residual (sometimes called current) risk. If you don’t capture both, you will need to consider an alternate method of how you will assess which of your controls are key controls. I.e. How would this risk assessment change if this control were to fail?
8. Keeping stakeholders on the same path
Do you have recommendations for facilitating the objectives approach to ensure boards, management and staff are on the same path?
This may be referring to ensuring the linkage between organizational objectives and risk and compliance activities, or perhaps ensuring control objectives are well articulated. The high-level answer is the same to both; education. Depending on your organization, you may need to develop a business case for that education (tailored to each audience) and a change management process to support any associated process changes.
9. Risk in Motion and tolerances
Does the Risk in Motion integration align to tolerance statements?
Our Risk In Motion dashboard links to KRIs (Key Risk Indicators) or tolerances. The Risk in Motion summarizes the number of KRI’s outside of tolerance, and is a single click away from a more comprehensive KRI dashboard that allows review of the specific tolerances.
If legislations are all required to be met (i.e., you can’t prioritize or risk rank any as they are all equal), should a controls assurance program cover all in a cycle? In your experience, does this represent high resource requirements?
You can, and I would argue should, risk rate your obligations. ISO 37301 on compliance management systems covers exactly this under a section titled ‘Compliance risk assessment’. It includes the following:
The risk-based approach to compliance management does not mean that for low compliance situations, noncompliance is accepted by the organization’.
You should be meeting all of your obligations, but low risk obligations may be subject to less frequent or rigorous controls assurance. Focus on the areas that will have the most impact on your organization.
10. Obligations versus risk management
Why should you not have a control with the only objective to comply obligation?
As a control is to maintain or modify risk, do you think it is important that the risk identified is correct in order for the design to works?
There may be occasions when you have a control only to meet an obligation, that you would never implement otherwise. However, to improve engagement and for your people to see it as less of a burden, try and identify why the obligation exists. It can also help with assurance – assessing whether the design meets the objective (to comply) might be a low bar compared to assessing whether it actually reduces risk in your organization. To do that, you need to understand the risks that it is modifying.
11. Linking processes with controls
Should obligations be part of your business processes (I.e. business process to be designed to be compliant to meet obligations)?
What is the linkage between controls and obligations? Would it be obligations - business processes - risks - controls?
You can look at these questions conceptually, or in the context of an ERM system with specific data linkages. There is no single correct answer, and may depend on the specific insights and reporting you want to achieve. In terms of a practical process for conducting risk assessments:
- What are the business processes I conduct? (process library)
- What obligations do those processes give rise to? (obligations linked to process)
- What risks to my organization do those processes give rise to? (risks linked to process)
- What controls do I have in place to manage the risk? (linked to risks)
- What controls are specifically required by obligations (linked to obligations)
While there are alternatives, if one of the components change, it is easy to identify other components that may need investigation. For example, if obligations change, you have an immediate list of business processes that may need review, as well as their associated controls.
12. Details of evidence
Re the "what" under control factors - should that not be the control activity undertaken rather than the evidence of the control?
This was just one of the factors from a more comprehensive list. We consider evidence as part of design (can it produce auditable evidence by design) and operation (was the evidence actually recorded and retained correctly after the control was performed). Depending on the type of test(s) chosen, this evidence can support observation / walkthroughs of the actual control being performed.
12. Reasonable versus limited assurance
What is the difference between reasonable assurance and limited assurance? Do you have a webinar on this or material.
This terminology has specific application in the audit field, and is usually expressed over the broader control environment rather than specific controls. Reasonable assurance and limited assurance are often expressed in positive and negative forms, such as:
- Reasonable assurance - “In our opinion the internal control framework is effective”
- Limited assurance “Nothing has come to our attention that suggests the internal control framework is not effective.”
We do not have any specific material that covers this distinction.
13. Controls and frameworks example
Can you provide a quick worked example of the 'linking controls to multiple frameworks' piece?
Let’s use Inventory management from ISO 27001 and map to NIST.
From ISO 27000, A.5.9 is ‘Inventory of information and other associated assets’. This can map to a range of more granular NIST controls, with a couple being:
- ID.AM-01 Inventories of hardware managed by the organisations are maintained
- ID.AM-02 Inventories of software, services, and systems managed by the organization are maintained
When preparing your own control libraries, you can link it across the different frameworks. Then when you are creating your test plans, you can ensure that it covers the specific requirements across all linked frameworks, enabling single tests to cover multiple frameworks.
14. Too much self
Any explanation on the non-standardized "too much self"?
This was in reference to Risk and Control Self-Assessment and traditional approaches. The comment on "too much self” was when different business units or areas use completely independent approaches such as different risk assessment techniques or measures, making it difficult to aggregate or benchmark across the organization.
15. Evidence as a test-type
It’s interesting to see evidence based in the middle of the list of test types (level of effort and level of assurance). Is this not fundamental as part of control walk-throughs to test effectiveness?
This is typically looking for evidence that the control has been performed as a more limited test compared to complete walkthroughs. As such it has limited use for design effectiveness, and is more appropriate to evidence of historical operation. It may provide limited information about whether the control will continue to operate in future.
16. Risk versus compliance revisited
Is compliance a subset of risk management? if yes, then how does that square up with compliance needing to ensure a firm is complying with FCA's regulatory requirements (which include risk management)?
They are separate disciplines, that have overlap, and you can start from either position.
Risk is the effect of uncertainty on objectives. One of those objectives (among others such as profitability, growth etc) should be to comply with applicable regulations. From this perspective, compliance risk can be considered a subset of risk management it’s one of many that the organization faces.
For some organizations (mostly financial services, listed companies and government), having a risk management framework or performing risk management is also an obligation. From this perspective, risk management is just one of the obligations the organization has to meet.
Risk management should be a value adding activity, regardless of whether it is a regulatory requirement. Regulators typically look at risk management from a consumer protection or financial stability perspective, while we advocate for an Enterprise Risk Management approach that considers all of the organization's objectives and risks.
17. Mapping to multiple frameworks
Coming back to control frameworks, why do you choose DORA for Europe which is resilience for financial institutions and then CSRD which is for sustainability, rather than NIS2, which is resilience for essential/important entities?
This was to demonstrate that the challenges of mapping to control frameworks is not just for resilience or cybersecurity, but is a challenge that can be solved across different domains, such as sustainability or ESG reporting.
18. Controls and quality assurance
What is your view of whether quality assurance can be an effective control?
It may depend on the specific application, but typically I would consider quality assurance to be a detective control – it analyses information and then acts if the information meets certain criteria or is beyond a threshold. In a contact center environment, monitoring calls for quality assurance may be a late detective control. Quality assurance over documents early in a customer engagement may identify issues before there is any impact, and may be an early detective control.
Quality assurance can be an effective control, but is usually alongside other controls to manage the related risks. Said another way, even when performing effectively, alone it may not have a large effect on the risk.
19. Smaller organizations and the 3 Lines model
For smaller organizations, what are your thoughts on the 1b approach to risk and compliance and having the 1b team reporting to the 2nd LoD, doing the controls testing for the 1st LoD?
I assume ‘1b’ refers to something like ‘Line 1.5’ in the 3 Lines Model. If those people report to Line 2 (independent risk or compliance teams) it may reduce accountability in the 1st Line for ensuring their controls are working effectively. However, the reason why it might be considered is that for smaller organizations it may be challenging to have the skillset for controls testing across the entire 1st Line.
This model can be a temporary measure as you develop your risk maturity and grow resources. The alternative is having that small team report to someone directly in Line 1 that then floats across different departments to complete testing. This will largely depend on organization structure and leadership to ensure it is housed appropriately and has the correct mandate.
Next steps for your organization
In Protecht’s Control frameworks and compliance: Bridging the gap to risk management webinar, we explored how you can build an effective controls management process that aligns with controls framework and provides assurance to compliance and risk professionals alike.
Protecht ERM’s latest controls solution is designed to help you implement a streamlined and compliant controls program that integrates with your broader risk management processes. Find out more in our Build a robust controls program with Protecht ERM webinar:
