Measuring the Effectiveness of Your BCM Programme through Measuring Resilience

Transcription

1 Measuring the Effectiveness of Your BCM Programme through Measuring Resilience Amy V. Stephenson Abstract: The purpose of this paper is to discuss how measuring organisational resilience over time can help to evaluate the effectiveness of business continuity management (BCM) programmes. The paper draws on an empirical study which developed a tool for measuring organisational resilience and presents an applied model and case study material. Key words: Organisational Resilience; ; Adaptive Capacity Introduction Organisations environment is increasingly complex and interdependent. Not only has the speed of change increased, but change also spreads across networks, supply chains and industry sectors to reach further than ever before. Investment choices on one side of the world affect trading on the other. In addition to events on a global and regional scale, local emergencies and crises such as power failures, staff disputes, and localised flooding can affect organisations ability to operate. These trends have been accompanied by a rise in the number of organisations using BCM strategies and programmes over the last few years. Despite the rise in the adoption of BCM programmes, very few organisations evaluate whether their BCM strategies are effective. Those organisations that do attempt to evaluate their BCM programmes focus on the process itself, and not whether their programmes are effective in producing resilience. This paper demonstrates how measuring resilience as an outcome, can enable organisations to evaluate the effectiveness of their BCM programmes and strategies. Why Measure Resilience to Evaluate Business Continuity? Measuring the business continuity process or benchmarking the components of the business continuity lifecycle often contributes towards the management of a BCM programme. However simply performing a task or completing a required step, does not mean that it is effective in producing its intended outcome. Measuring the process or lifecycle to evaluate the effectiveness of BCM programmes, encourages a tick-box mentality and only really measures whether the organisation is talking the talk when it is walking the walk that is important. Resilience, an organisation s ability to survive and potentially even thrive in times of crisis (Seville, et al., 2008, p. 18) is an organisational goal, and organisational capabilities such business continuity management (BCM) are tools which organisations can use to achieve greater resilience (BCI, 2006; Gibson & Tarrant, 2010). Evaluating BCM through measuring resilience is important for three key reasons: The need to evaluate the effectiveness of BCM programmes up, down and across

2 The need to evaluate the effectiveness of BCM programmes in producing or increasing resilience The need to identify where BCM programmes are performing effectively and where they are falling short Organisations often rely on information from senior managers or a BCM practitioner to evaluate the effectiveness of BCM programmes. However this does not measure whether programmes are effective, it only measures senior management s prioritisation and knowledge of BCM. To measure the effectiveness of BCM programmes in producing or increasing resilience, organisations must assess whether their BCM policies, plans and strategies have been absorbed and embedded across the whole organisation up, down and across. It is therefore important that as many members of staff as possible take part in the evaluation so that results are representative of the organisation as a whole. If BCM is a tool for producing or increasing resilience, then its effectiveness is the extent to which it achieves that goal. As organisations continue to develop BCM programmes they begin to ask, how much resilience is our BCM programme buying us? This highlights the challenge of trying to justify and allocate resources to BCM where there are no metrics to measure whether or not the money is being well spent. Cerullo and Cerullo (2004, p. 71) support this when they argue that many executives view the BCP as a way to spend money with little, if any, return on the investment. Herbane (2004, p. 451) identifies two banks who despite having BCM structures in place, have yet to demonstrate substantive benefits from their investments. Although many BCM professionals use common methods and a structured approach, BCM is not a one size fits all discipline. Measuring organisations resilience will help to evaluate the effectiveness of BCM programmes by demonstrating the link between organisations BCM programmes and their resilience. With limited resources and a plethora of BCM techniques and tools to apply, organisations need to know which ones are working for them, and which ones are not. Measurements of resilience can enable organisations to answer key questions including which of our BCM strategies is affecting our resilience and what are the gaps in our resilience that could be addressed through new or re-directed BCM investment. How can Resilience be measured? Stephenson, Seville et al. (2010) present a model and tool for measuring organisational resilience based on an empirical study of New Zealand organisations. During the study researchers and practitioners developed a suite of proposed indicators based on literature and a pilot study. In total 68 organisations of different sizes and sectors measured and benchmarked their resilience using the indicators, which were then refined through factor analysis. The model measures resilience using thirteen indicators grouped within two factors; planning and adaptive capacity, which can be seen in Table 1. Each of the indicators is measured through a survey tool which is completed by senior managers and staff from all levels and hierarchies of the organisation. The organisation s results identify their strongest and weakest resilience indicators as well as their scores for planning, adaptive capacity and overall resilience. Further discussion of the development of this model is available in Stephenson, Vargo et al. (2010).

3 Table 1: Model of Organisational Resilience Adaptive Capacity Proactive Posture Leadership Strategies Staff Engagement & Involvement Participation in Exercises Situation Monitoring & Reporting Recovery Priorities Minimisation of Silos External Resources Internal Resources Decision Making Innovation & Creativity Information & Knowledge (Stephenson, Seville, et al., 2010, p. 19) The research also identified links between these resilience indicators and organisations profitability, cash flow and return on investment. However further research and is needed to determine the extent to which resilience is manifested, for example through faster recovery times, or reduced losses for businesses; this research is currently in progress as a follow-up study within New Zealand. Evaluating the Effectiveness of BCM through Measuring Resilience Figure 1 shows a simple 4-step process that can be used to evaluate BCM programmes at various stages in the BCM lifecycle through measuring resilience. The first step of the process involves measuring the resilience baseline to provide an initial benchmark of how resilient the organisation was before the BCM programme or before a particular BCM investment was made. This involves using a tool such as the one described in this paper to measure the organisation s resilience. During the second step the organisation engages in BCM by initiating a BCM programme or investment. The third step of the process involves measuring the organisation s resilience once the BCM action has been taken to provide a comparative assessment of the organisation s resilience which will reflect any changes since the initial measurement. During the fourth step the organisation compares it s initial and current resilience scores to evaluate how effective the BCM action has been in producing or increasing resilience. Figure 1: Evaluating BCM through Measuring Resilience 1 Measure Resilience Baseline Other elements that 4 Evaluate and 2 Compare BCM Resilience Action affect resilience such as competition, legislation, crises etc. BCM Action 3 Measure Resilience Improvement

4 Organisations can choose when to measure their resilience and when to evaluate BCM actions. For some organisations, regular measurement will enable them to continuously evaluate and adjust their BCM programmes and strategies and their contribution to the organisation s resilience. Other organisations may prefer to only measure their resilience improvement upon completion of a particular project or exercise. The important thing is that the measurement should not be a burden but should be completed to provide information to feed into decision making about future BCM strategies and investments. A caveat on this process is that BCM programmes and investments are not the only element that may have an effect on an organisation s resilience. This is demonstrated through Table 2 which summarises some of the elements that could affect organisations resilience; business continuity is just one of these elements. It is important to consider alternative sources of change during the comparison, such as those in Table 2, which may be responsible for changes in the organisation s resilience. Table 2: Elements that can affect Organisations Resilience Source Elements Gibson and Tarrant (2010) Organisational capabilities such as business continuity and risk management Crichton et al. (2009) Crises and emergencies Granatt and Paré- Increased or decreased competition Chamontin (2006) Christopher and Peck The resilience of supply chains, and organisational relationships (2004) and interdependencies across networks Dalziell and McManus Changes in legislation, e.g. legislation requiring lifelines (2004) organisations (electricity, water, telecommunications etc.) to ensure continuity of service during and after crises and emergencies Note: Sources are listed in date order. Case Study For the purposes of demonstrating how the effectiveness of BCM programmes and strategies can be evaluated using the process outlined above, this paper presents a case study of a small printing business. The printers were one of the organisations that took part in Stephenson et al. (2010) study which used the indicators shown in Table 1 to measure the resilience of 68 organisations in Auckland, New Zealand. The case study is presented using a mixture of real and fictitious data which has been arranged for this paper. It uses the example of an organisation that has gone from having no BCM, to initiating a BCM programme. However the same process could be applied to any organisation at any stage in the BCM lifecycle, for example to evaluate the effectiveness of new resources or the impact of a large BCM exercise or simulation. Figure 2 provides a graph of the baseline resilience results achieved by the printers in the Auckland study. In addition the printers achieved a score of 44% for the planning factor, 50% for adaptive capacity and 47% for their overall resilience which can be characterised as very poor. Figure 2 shows that the printing business has some resilience strengths including their recovery priorities and staff engagement and involvement. A score of 75% for the recovery priorities indicator suggests a good awareness and understanding of the critical functions that would need to be restored following an emergency or crisis. The printing business s score of 75% for staff engagement and involvement is a significant strength and suggests that staff are an integral part of the organisation s response to unexpected disruption.

5 Figure 2: A Printing Organisation s Scores for the Indicators of Organisational Resilience % 90.00% 80.00% 70.00% 75.00% 75.00% 60.00% 57.14% Score 40.00% 30.00% 31.25% 33.33% 35.00% 20.00% 16.67% 10.00% 0.00% Proactive Posture 0.00% Strategies Participation in Exercises Recovery Priorities External Resources Leadership Staff Engagement & Involvement Situation Monitoring & Reporting Minimisation of Silos Internal Resources Decision Making Innovation & Creativity Information & Knowledge Indicators of Organisational Resilience Adaptive Capacity

6 The printing business also has significant resilience weaknesses including their planning strategies, innovation and creativity, and proactive posture. A score of 0% for the planning strategies indicator suggests that the organisation has done no formal planning and very little informal planning, preferring instead to rely on their ability to adapt when a crisis occurs. The organisation s lack of impetus on planning is also reflected in their score of 31% for the proactive posture indicator. A low score for proactive posture suggests that the organisation does not prioritise resilience, is not proactive in managing crises or potential sources of failure, and is not actively participating in discussions with others in its industry sector. The printing business also achieves a very poor score of 17% for the innovation and creativity indicator. Innovation and creativity is a critical component of organisations ability to adapt, and may enable organisations to create advantage over less adaptive competitors in both business-as-usual and crisis situations. Figure 3 provides a comparison between the printer s resilience baseline scores as shown in Figure 2, and scores which they could achieve as they implement a new BCM programme. In this example the printing organisation has initiated a BCM programme and has focused on the management practices identified by the BCI Good Practice Guidelines (Business Continuity Institute, 2010). These include policy and programme management and embedding BCM into the organisation s culture. The organisation has also started to address the technical practice of understanding the organisation. Activities that the printing business has engaged in include: Development of a BCM policy document Defining roles and responsibilities Analysing gaps in training and awareness Scoping and initial stages for a business impact analysis Increased involvement in industry groups While the scores shown on Figure 3 suggest a BCM programme that is still in its infancy they also demonstrate significant improvements over previous scores. The most significant improvement demonstrated by this example is the planning strategies indicator for which the organisation increased its score from 0% to 32%. This results from a significant improvement in the organisation s formal planning and accompanies an increase of 28% in their proactive posture score. The increase in the printer s proactive posture score suggests that the organisation s efforts to establish a BCM programme have resulted in increased prioritisation of resilience as a goal and an increase in resources allocated to resilience. While the printers have not yet completed any exercises they are aware of the need to test and validate their business continuity plans and staff are confident that this is a priority. As a result, their score for the participation in exercises indicator has only marginally risen (an increase of 6%). The majority of the other indicators have responded in the same way, achieving a higher score where formalised planning and communication across boundaries comes to the fore, and remaining roughly the same where the organisation has done little to change its approach e.g. recovery priorities. Comparison and discussion of the organisation s resilience scores before and after BCM identifies the differences that may be attributed to the organisation s BCM programme. In the example of the printing business, the development of a BCM policy, and the definition of roles and responsibilities has increased the organisations resilience through increased awareness and formal planning.

7 Figure 3: Comparison of a Printing Organisation s Scores for the Indicators of Organisational Resilience Before and After Initiating a BCM Programme % 90.00% 82.10% 80.00% 70.00% 75.00% 75.60% 75.00% 68.50% 75.30% 60.00% 59.30% 56.10% 57.14% 56.00% Score 40.00% 30.00% 31.25% 32.40% 42.30% 33.33% 35.80% 25.00% 35.00% 41.80% Before BCM After BCM 20.00% 16.67% 10.00% 0.00% Proactive Posture 0.00% Strategies Participation in Exercises Recovery Priorities External Resources Leadership Staff Engagement & Involvement Situation Monitoring & Reporting Minimisation of Silos Internal Resources Decision Making Innovation & Creativity Information & Knowledge Indicators of Organisational Resilience Adaptive Capacity Note: The After BCM data used in this example is fictitious although it reflects the analysis that is possible using the Resilience Measurement Tool as part of the evaluation of a BCM programme.

8 The comparison also shows that despite the printer s efforts, their score for the external resources indicator has slightly decreased. The external resources indicator investigates whether the organisation is aware of its resource dependencies and has thought about what external resources it might need to access in a crisis. The organisation s lower score for this indicator is a result of higher awareness among staff about their resource dependencies which has corrected over-optimistic baseline data. Overall, the comparison of the organisation s resilience scores demonstrates that their current BCM actions are effective in increasing the organisation s resilience. Conclusion Business continuity management is a tool that organisations can use to produce or increase their resilience. However organisations cannot continuously manage and invest in BCM programmes without feedback on their effectiveness. The majority of organisations evaluate the effectiveness of BCM based on information about the process of BCM. Instead this paper provides a starting point for evaluating the effectiveness of BCM programmes, strategies and investments based on the extent to which they contribute to organisations resilience. This paper has demonstrated how measurements of an organisation s resilience before and after initiating a new BCM programme, strategy or investment can be used to evaluate their effectiveness. In the example of the printing organisation discussed in this paper the comparison of their scores before and after initiating a BCM programme showed that overall their programme was effective in increasing the organisation s resilience. References BCI (2006). Business Continuity Management, Part 1: Code of Practice: Business Standards Institute. Business Continuity Institute (2010). The BCI Good Practice Guidelines. London: BCI. Cerullo, V., & Cerullo, M. J. (2004). Business Continuity : A Comprehensive Approach. Information Systems Management. Christopher, M., & Peck, H. (2004). Building the Resilient Supply Chain. The International Journal of Logistics Management, 15(2), Crichton, M. T., Ramsay, C. G., & Kelly, T. (2009). Enhancing Organizational Resilience Through Emergency : Learnings from Cross-Sectoral Lessons. Journal of Contingencies and Crisis Management, 17(1), Dalziell, E., & McManus, S. (2004). Resilience, Vulnerability and Adaptive Capacity: Implications for System Performance. Paper presented at the International Forum for Engineering Decision Making. Gibson, C. A., & Tarrant, M. (2010). A Conceptual Models' Approach to Organisation Resilience. Australian Journal of Emergency Management, 25(2), Granatt, M., & Paré-Chamontin, A. (2006). Cooperative Structures and Critical Functions to Deliver Resilience Within Network Society. International Journal of Emergency Management, 3(1), Herbane, B., Elliot, D., & Swartz, E. M. (2004). Business Continuity Management: Time for a Strategic Role? Long Range, 37, Seville, E., Brunsdon, D., Dantas, A., Le Masurier, J., Wilkinson, S., & Vargo, J. (2008). Organisational Resilience: Researching the Reality of New Zealand Organisations. Journal of Business Continuity and Emergency Management, 2(2),

9 Stephenson, A. V., Seville, E., Vargo, J., & Roger, D. (2010). Benchmark Resilience: A Study of the Resilience of Organisations in the Auckland Region Christchurch: Resilient Organisations Research Programme. Stephenson, A. V., Vargo, J., & Seville, E. (2010). Measuring and Comparing Organisational Resilience. Australian Journal of Emergency Management, 25(2),