Element 9: Reviewing a Risk Management Approach

This information sheet is intended to assist Commonwealth officials at the Specialist and Executive levels understand:

• when to review a risk management framework
• selecting the responsibility, scope and method for a review
• how to conduct a review.

A risk management framework sets the foundations and organisational arrangements for designing, implementing, monitoring, reviewing and continually improving risk management capability. Undertaking a periodic review to assess the effectiveness of an entity’s risk management framework is necessary to ensure that the framework continues to evolve and meet the needs of the entity.

The exact timing and frequency of a review will be dependent on the nature of the entity’s operations. For instance, if an entity has experienced significant change, or its exposure to risk has increased, the framework may need to be reviewed more frequently. Conversely, if an entity’s risk management framework has been in place for a number of years, and its operations are relatively stable, the review cycle could be conducted less frequently (for example once every 2 years).

The identification of an increasing number of ‘near misses’ and incidents can, over time, indicate that not only is an entity’s management of individual risks unsatisfactory, but that the framework itself may require review.

An annual review may be conducted through internal or external audit that covers the compliance and effectiveness of the framework. A comprehensive review of the appropriateness, effectiveness and adequacy of the risk management framework can be undertaken up to every 3 years.

An entity’s risk function, or the role that is tasked with risk management in an entity, is well placed to complete a review of the risk management function. However, broad consultation may be undertaken across the entity to ensure the risk management framework, and the risk function are meeting expectations. An effective risk framework supports decision making and key processes, and should not be a seen as an outcome in its own right.

An independent review of the risk management framework can also be useful. This provides the risk function or designated risk role with a fresh perspective, including challenging current norms and practices.

When undertaking a review of the risk management framework, it is important to determine if it has been appropriately communicated and tailored to the entity allowing risks to be:

• efficiently and effectively identified and appropriately assessed,
• considered in the context of the entity’s objectives and other business processes,
• adequately treated and controlled where relevant, with residual exposures understood, and
• effectively and regularly monitored and reviewed by management, executives and the board.

There are several approaches to reviewing a risk management framework, some of which more broadly include:

• An annual compliance review through internal or external audit - These explore particular key elements of the risk management framework in depth or on a rotational basis. This could involve conducting a risk management maturity assessment against the Commonwealth Risk Management Policy, international standards and peer entities.
• A comprehensive review undertaken by an experienced reviewer and undertaken at least every 3 years that would be reported to the entity’s Audit Committee and Risk Committee (if applicable). It could include a comparison of the entity’s current practice against any identified better practice through the use of benchmarking performance against other entities. The review may draw upon the entity’s internal resources, such as internal audit reports and other performance data.

The review approach is dependent on the entity’s specific circumstances including:

• the amount of change in an entity’s operations or operating conditions,
• current maturity of the risk management program,
• complexity of the entity’s operations,
• number of near misses, and
• whether risk events have materialised over a 12 month period.

These are the factors that ultimately determine whether a high level or detailed review of the risk management framework is conducted. For example, an entity which has previously been told it has a mature risk management framework but has had a significant organisational restructure, would most likely need a detailed review of its risk management framework to ensure it meets its new needs. The review of an entity’s risk management framework, program and practice generally occurs at 3 levels.

Level 1 – Regular checking and monitoring

The first line of responsibility for managing risk is the day-to-day decisions of officials at all levels. Accordingly, this is where the first line of review also lies. Individuals will choose to accept or reject risks on a given day for a variety of reasons – some appropriate and informed, and some not. A process of ongoing discussion about risk, and work group and peer moderation is important to ensure a consistent approach.

Relevant issues for consideration include:

• the accuracy and completeness of the risk register
• whether the consequences and impact levels of individual risks are still relevant and
• the effectiveness of controls and treatments.

Level 2 – Management review

Management review of processes, systems and controls constitutes the next level of review. To fulfil this role effectively, managers are encouraged to understand the context, objectives and business of the entity, its risk management framework, and its risk appetite and tolerances. These reviews will be most effective when they are regular, seen as routine, and undertaken on a periodic basis. Reviews may be planned to target high risk processes, but also sample broadly across the entity and its service providers. Where issues are identified, determine if they are specific to an individual risk or risk decision maker, or systemic in the entity.

Once determined, the issue can be addressed with findings and corrective actions documented. This will work towards building risk management capability and confidence in an entity’s risk management system and approach.

Level 3 – Independent review

Independent reviews can provide a level of assurance that a comprehensive risk management framework and process is in place and implemented effectively. They do not need to be undertaken by an independent auditor or consultant, they can be overseen by a SME or official from inside the portfolio of an entity who is independent from the framework, program or project.

Independent review also brings a fresh perspective, and can identify where an entity’s framework lacks alignment with its organisational objectives, where there are instances of non-compliance, and also where there opportunities for improvement in processes. Independent reviews can be useful in identifying opportunities to enhance consistency across the entity including more effective ways of managing similar risk, or categories of risk, from an entity-wide perspective.

The following planning activities have been outlined below as a guide to completing a review of an entity’s risk management framework. An entity may decide that it does not need to complete all activities based on the nature of their entity’s operations (for example, an entity which is relatively small and has staff members assuming multiple roles, may not require a detailed project plan).

You can also use a review of your risk management framework as an opportunity to engage your senior executives on risk and to encourage them to think how changes can be more fully leveraged to achieve better business outcomes.

A review into an entity’s risk management framework may take a lessons learned approach to being able to identify and assess against the following outcomes:

• The framework remains appropriate for the entity and its risk profile
• The framework has been consistently and effectively applied throughout the entity
• There are appropriate procedures, processes and controls in place to ensure that the framework addresses and new or emerging risks
• The framework is effective in providing appropriate, effective and timely risk-information
• The framework is aligned to and works towards embedding risk in the decision-making activities of the entity

Testing alignment with the Commonwealth Risk Management Policy (RM Policy)

The elements of the RM Policy are tested in the Comcover Benchmarking Survey.

• Consider using your survey results to determine where effort may be required to better align your risk management framework with the RM Policy.
• Map your entity’s existing risk framework to the elements of the RM Policy to understand where your entity currently stands.
• Systematically review your existing documentation and systems against the requirements of the RM policy.
• If areas of your risk framework need to be further developed to align with the RM Policy, develop an action plan and seek senior executive and audit and risk committee commitment to oversee the review.

Sample questions to ask when reviewing a risk management framework

The following questions have been included to guide entities in reviewing their risk management framework. This is not an exhaustive list and the questions may differ depending on the nature of the entity’s operations.

• Is a common definition of risk, which addresses both threats and opportunities, used consistently throughout the entity?
• Are the key roles, responsibilities and authorities relating to risk management clearly articulated and followed within the entity?
• Do the governing bodies (for example, Boards, Audit Committees, Risk Committees, Management Committees) have appropriate transparency and visibility into the entity’s risk management practices in order to discharge their responsibilities for oversight?
• Does the risk function’s position in the entity enable direct access to the executive management team?
• Has your entity defined relevant risk categories which enable risks to be aggregated, analysed and reported upon?
• Do your entity’s risks align to its organisational objectives?
• Does your entity have a clear approach for analysing and evaluating risk?
• Has your entity defined its risk appetite?
• Does its risk appetite enable decisions to be made that reflect the entity’s attitude towards risk, that is, what is acceptable and unacceptable?
• Does your entity have a regular reporting cycle where risk information is incorporated for management review and attention?
• Is there a process which identifies, assesses and treats risks for all key activities (for example, projects, programs, policy development and business processes)?

This information sheet is intended to assist Commonwealth officials at the Specialist and Executive levels understand:

• Factors that could cause a change in risk profile
• The implications of a change in risk profile on an entity’s risk management framework.

A risk profile is a description of any set of risks. The set of risks can contain those that relate to the whole entity, part of the entity, business unit, program, project or as otherwise defined.

An entity’s risk profile can contain risks of different natures. Some of these may be managed at an enterprise level and represent the most significant risks exposures of the entity, others will be managed within business units and represent more focused concerns

Any risk profile requires continual maintenance and should be updated to reflect changes in the internal or external context or when a risk event(s) materialises in an unexpected manner. Where changes in an entity’s risk profile occur, the risk management framework should be reviewed to ensure it is up to date and aligns with the entity’s risk profile

An entity’s risk profile may change due to changes in the internal or external context.

• Internal context - includes changes in an entity’s objectives, resources, capabilities, information systems, operating model, decision making processes, values and culture, policies, perceptions and governance structure.
• External context - includes changes in an entity’s operating environment (for example, cultural, legal, financial, technological, geopolitical etc.), external stakeholder perceptions, and key drivers and trends that have an impact on the objectives of the entity.

Analysis of any changes to an entity’s internal and external context may result in the following updates to an entity’s risk profile:

• Identification of new risks (for example, at the strategic, operational or project level) that change the overall exposure of the entity
• Movements in the effectiveness of key controls that change the overall exposure of the entity
• Materialisation of emerging risks or the identification of new emerging risks
• Worsening risk exposures (for example, increases in the likelihood and/or consequence of key risks)
• Review of who owns the risk(s) and how they are reported.

Undertaking a review to assess the effectiveness of an entity’s risk management framework following a change in risk profile is necessary to ensure that the framework continues to evolve and meet the needs of the entity.

Examples of areas of the risk management framework that may need to be updated due to a change in an entity’s risk profile include:

• Likelihood and consequence descriptors - are the descriptors appropriate given changes in the entity’s risk profile? Do certain thresholds (for example, financial consequences) need to be updated?
• Risk appetite - has the entity’s risk appetite and tolerance levels changed due to changes in the internal and external context (for example, technological advancements may enhance the entity’s appetite to be more innovative)?
• Risk severity matrix or heatmap - does the structure of the entity’s risk heatmap and severity levels reflect the considered risk appetite of the entity?
• Risk categories - are the entity’s risk categories still relevant? Should they be changed or aggregated differently given changes in the entity’s risk profile?
• Risk escalation and reporting cycle - is the current risk escalation and reporting cycle appropriate given change in the entity’s risk profile and overall risk exposure (for example, are more frequent reporting cycles and/or different accountabilities required given the change in risk exposure?)

This information sheet is intended to assist Commonwealth officials at the Specialist and Executive levels understand:

• What Key Performance Indicators (KPIs) are and their benefits in relation to monitoring effectiveness when reviewing an entity’s risk management framework
• Types of KPIs that can be used to support the review of an entity’s risk management arrangements
• Practical tips on developing, monitoring and reporting risk management KPIs

Key Performance Indicators (KPIs) are measures of progress toward an intended result that can help focus attention on areas that matter most. KPIs can be used to:

• measure the performance of its risk management framework and identify opportunities for improvement
• obtain assurance that the framework is designed to support critical business processes and risks
• identify gaps in processes that contained within an entity’s risk management framework (for example, risk assessments, mitigation and control activities) 
• monitor resources (for example, budget, capability etc.) allocated to support the risk management framework.

The nature and characteristic of KPIs may differ depending on what aspect of a risk management framework is measured or monitored for effectiveness. KPIs can be:

• Qualitative - assess subjective characteristics (for example, risk owner’s confidence in assessing key risks)
• Quantitative - assess objective characteristics and looks at hard data (for example, number of risks identified or closed)
• Lagging - measures that look back at what the entity has already achieved
• Leading - measure that look forward at the entity’s future outcomes and events.

It is critical that KPIs are tailored to the entity’s context and developed to improve the effectiveness of the risk management framework. The approach below outlines the key steps to ensure that each indicator is specific, measurable and fit-for-purpose to measure and review the effectiveness of an entity’s risk management framework.

1. Identify the entity’s risk management objectives and intended outcomes

The purpose and objectives of the entity’s risk management framework will determine what needs to be measured and ensures linkage between the KPIs and the entity’s overall risk management strategy.

2. Identify data needs and availability

Once the objectives have been identified, consider what direct and indirect measures can be used to assess performance against each intended outcome. Identify what metrics or data will be required for each measure and compare against existing data to determine if there are any gaps. This will help narrow down the potential measures identified to develop the most appropriate KPIs for the entity.

3. Developing indicators

Focus on indicators that can track changes in the entity’s risk management framework and aim to include a mix of leading and lagging indicators, with consideration of the existing data available as identified in step 2.

Characteristics of good KPIs include:

• They are capable of being measured
• They provide objective and quantitative evidence
• They provide the opportunity to undertake trends analysis and compare performance over time
• They track efficiency, effectiveness and quality.

The entity should also outline the trigger levels and thresholds that determine the need for review or escalation.

It is important to determine the methodology, frequency and ownership of these indicators. These refer to:

• How you will collect the data
• When and how often the data will be collected
• Responsibility assigned to own, collect and analyse the relevant data.

The successful implementation of a risk management framework is underpinned by effective monitoring, review and evaluation of applicable KPIs. Effective monitoring includes collecting timely (that is, quarterly, bi-annually or annually) and relevant information that allows progress to be tracked. Performance should be tracked in a deliberate and systematic manner.

Entities should ensure that monitoring and performance reporting on the effectiveness of their framework meet the needs of stakeholders. The measurement of results are used to inform those enhancements required to the framework including required resourcing and the prioritisation of activities.

Whenever there is a change in the entity’s risk management framework, the entity should review and update its KPIs to ensure they remain relevant and aligned to the desired performance of the framework.

The number of risks with negative consequences realised over a 12 month period

• Tracks and measures the effectiveness of the framework. It could also be an indicator of:
  • Effectiveness of implementation
  • Effectiveness of escalation processes
  • Sufficiency in information/knowledge sharing
  • Staff engagement
  • Effectiveness of incentives, rewards or recognitions
  • Adequacy and quality of training and induction programs.

Risk training completion rate (or an annual risk culture survey, if relevant)

• Measures staff engagement and awareness of the entity’s risk management practices to improve implementation of the risk management framework. It may also be an indicator of: 
  • Staff engagement and interest in risk management
  • Quality of communications
  • Quality and accessibility of training materials and resources
  • Management involvement
  • Adequacy of time allocated for staff development
  • Risk culture.

The number of overdue mitigation actions

• Assesses the effectiveness of current mitigating strategies and escalation processes. It may also be an indicator of: 
  • Effectiveness of escalation processes
  • Adequacy of risk monitoring
  • Efficiency of resource allocation
  • Effectiveness of implementation
  • Sufficiency in information/knowledge sharing.

The number/percentage of incomplete or incorrect risk assessments

• Evaluates compliance with the process to identify and assess risks as defined in the risk management framework. This can be measured by reviewing completed risk assessments (for example, in a risk register) and identifying errors such as the number of risks with incomplete fields or incorrectly calculated risk ratings. It may also be an indicator of: 
  • Effectiveness of implementation
  • Effectiveness of risk management training and guidance materials
  • Adequacy of risk monitoring
  • Management involvement.

Ensure that KPIs are well understood by management and relevant personnel
Critical that stakeholders understand how the indicators measure progress towards achieving the objectives of the entity’s risk management framework.

Identify the most effective method or channel to communicate KPIs
Avoid excessive reporting of KPIs (that is, long reports full of numbers, tables, etc.). Use graphics and visual representations to communicate insights and trends or highlight issues in an engaging way.

Establish a mechanism to regularly review KPIs to ensure that they remain effective and relevant 
Constantly review whether the KPIs developed still measure the effectiveness of the entity’s risk management framework. Remove or add as necessary.