The Relative Effectiveness of widely used Risk Controls and the Real Value of Compliance

Abstract

As the volume of cyber-attacks continues to rise and also the levels of harm suffered from them, it is becoming critical that organisations can demonstrate that reasonable efforts are being undertaken to reduce cyber-risk. However, the risk responses and controls typically viewed as necessary, and even essential, by the professional and expert community are generally not underpinned by any framework that facilitates rigorous reasoning, qualification or quantification of the benefits resulting from their deployment. This means that the real value of compliance, or the variability of compliance, to risk-control standards is not well reasoned or measurable in any scientific, unambiguous or verifiable sense. This further means that methods used to manage, mitigate or transfer the risk by stakeholders across the information security and cyber risk landscape` are existing increasingly in isolation. Our study here has shown that a more rigorous risk valuation and risk management environment can only fully exists where transparent and effective collaboration between stakeholders exists. Only through such collaboration can risk be fully understood, modelled, valued and thus managed. In this project we explore the value of risk controls to security posture, and so the value of compliance to standards and frameworks prescribing these controls. Our approach in this report centres on the effectiveness of controls, with particular emphasis on determining the residual risk for each control. The residual risk may occur due to inherent vulnerabilities that controls have against specific attack vectors or due to implementation practices. We provide details on how residual risk, a critical factor for determining the effectiveness of controls, may be assessed for each risk control. This report examines the effectiveness through a number of lenses: assets and their attack-surface (which are determined through a number of risk dimensions, with the most important being the Bespoke or Common dimension) and the ability of controls to protect them; controls in the context of inherent and known vulnerabilities; the capability of threats in compromising the effectiveness of controls; and the role controls play in how cyber-harm manifests and propagates. These lenses offer a unique insight into how key organisational contexts can be impacted by the effectiveness of controls. This knowledge is also used to extend our work and the model that we have created to map the relationship between risk controls and organisational concerns such as assets, cyber-value-at-risk (cyber-VaR) and cyber-harm. Our main contributions in this report can be summarised as follows: • An extension of our model that defines the associations between risk controls on the one side and assets, cyber-VaR and the different types of cyber-harm which may occur in a typical organisation on the other. In particular, we have significantly extended our research on the relationships between these components, which incorporates general relationships, relationships across the three main levels, inter-dependencies between levels, links to controls and control effectiveness. This has also involved the provision of a reasoning approach and syntax whereby the associations between the various enterprise components could be defined. This would allow organisations to have a better understanding of what harms may be caused to which assets (or set of assets) and how those harms could trigger other harms, and eventually increase the cyber-VaR. Using our proposed reasoning method, we also considered how controls as implemented could protect assets, reduce or limit cyberharm, and impact cyber-VaR. This links directly to the control’s effectiveness, as different controls have different influences on these variables. 3 | P a g e • An iniial validation of our model through interviews and focus groups with industry professionals in the areas of cybersecurity and cyber-insurance. To comment briefly on the findings from this stakeholder engagement, we found that: 1. There appears to be a clear understanding of what critical assets to organisations are and these assets are all successfully covered in our model. Furthermore, organisations determine critical assets based on their importance to key business processes, requirements from legislation and regulators, and harms that may result to the organisation if the asset is compromised. 2. The model was able to capture a vast majority of the harms that professionals identified as potential impacts from cyber-attacks. However, professionals generally had not considered the full range of cascading harms that can result from attacks, nor did they have well-defined metrics for estimating or measuring harms. 3. While organisations do apply controls to address specific risks that they face, other key motivators for selecting certain controls include the requirements placed by regulatory bodies, legislation and broad concerns facing the industry (e.g., new types of attackers). Most importantly, many organisations do not have a good understanding of how to measure the effectiveness of controls especially on a realtime basis. • Focusing specifically on the topic of the relative effectiveness of risk controls, our analysis of the literature (academic and industry-related) and the interaction we have had with industry professionals has provided little scientific evidence to suggest that there exist clear ways to measure effectiveness. To validate the utility of our model therefore and the effectiveness of controls more broadly, we have outlined several data requirements at each level of the model. Data is crucial as it will enable users of the model to reason about numerous aspects including the links between certain assets and cyber-harms (e.g., typical harms that result from certain assets), the likely propagation paths of harms (e.g., specific harms that are likely to result due to other harms), the probability distributions that allude to the likelihood of particular losses, and effectiveness of risk controls. These requirements are outlined and followed by an explanation of how they would be applied in the use of the model. • The following actions should be considered in order to further this line of research: 1. Implement a prototype software tool of the model proposed, which would be capable of determining the potential range of impacts of a risk control upon exposure to harm. This might be developed with a selection of estimate probability distributions based upon knowledge in the community, and with which it would be possible to test the sensitivity of results in a range of scenarios. 2. Design a methodology for learning the impacts of risk controls within an organisation using software sensors with an organisation’s infrastructure. This would enable the collection of data to establish the probability distributions required by the Model (in 1 above) through aggregation of results across multiple organisations and identification of general patterns. This approach would have the added benefit of allowing organisations to consider results tailored to their specific operations. 3. Additionally, further consideration is required of how different datasets may be linked to provide quantitative evidence on how effective controls are. This new approach should take into consideration the interdependency of controls how 4 | P a g e effectiveness of the ecosystem of controls may change if certain controls are not present. Further exploration of historic data is required to identify features that will be abstract enough to provide useful information regarding the likelihood of an attack, even when the data is considered obsolete (i.e. on systems which are not used any more). 4. The approach should be extended to address other classes of harm, from natural disaster or accidental insider actions (for example, where our research in other projects leads us to believe that current risk controls are often inadequate). 5. We should also consider expanding the findings from the qualitative research. A possible next step could be to conduct large scale questionnaires, focusing on which controls are widely accepted in the industry and what metrics are used to determine their effectiveness. Additionally, interviews and focus groups could take place to emphasise on how the interconnection of assets may change the way organisations perceive assets, as well as identifying how cascading harm may occur and which types of harm are triggered. Interviewing lawyers will shed light into how recent developments in legislation may influence the way organisations reason about controls and whether cyber-insurance will become a norm, enabling insurers to suggest a set of desirable controls to hedge risk 6. Specific research should be conducted into the relationship between harm, and an assets level of digitisation. We need to know if it is the case that the level of digitisation has a consequence for the likelihood of susceptibility to successful cyberattack, and the potential for harm and cascading harms within an organisation. New controls might be suggested. 7. Specific research should be conducted into the value of unpredictability in control usage as a mechanism for improving cyber-defenses to reduce cyber-harm. 8. We still need to consider how aggregation and systemic risk affect propagation of harm and how potential for such affects the decisions that organisations make. In particular we should focus on harm propagation across organisations who share common technologies, harm from unavailability of web-services and impact on business interruption.