Creating Safety by Engineering Resilience

Transcription

1 Creating Safety by Engineering Resilience How to be prepared to be surprised emergence - people adapt essentials FBC pressure sacrifice judgments enhancing resilience learning from incidents tradeoffs cross-checks 4 I s of safety organization

2 Paradigm shift on safety management the future seems implausible, the past incredible Create Foresight Anticipate the changing shape of risk, before failure or harm occurs

3 TMI Anomaly Response / Brittle Machines Behind Human Error Adaptation / Hindsight HRO s Automation Surprises Bridging GAPS

4 Doing things safely is ongoing, but strategies sensitive to paths toward failure, but partial awareness of potential paths since world is changing, potential paths are changing coping strategies may be weak or mistaken over-confidence missing side effects of change invest in monitoring changing paths and potential

5 C/S/E/L Patterns in Cognitive Work Transformation and Adaptation The law of stretched systems: every system is stretched to operate at its capacity; as soon as there is some improvement, for example in the form of new technology, it will be exploited to achieve a new intensity and tempo of activity. Cognitive Systems Engineering Laboratory:

6 C/S/E/L Patterns in Cognitive Work Patterns of Reverberations Much of the equipment deployed... was designed to ease the burden on the operator, reduce fatigue, and simplify the tasks involved in operations. Instead, these advances were used to demand more from the operator. Almost without exception, technology did not meet the goal of unencumbering the personnel operating the equipment... systems often required exceptional human expertise, commitment, and endurance. there is a natural synergy between tactics, technology, and human factors... effective leaders will exploit every new advance to the limit. As a result, virtually every advance in ergonomics was exploited to ask personnel to do more, do it faster and do it in more complex ways.... one very real lesson is that new tactics and technology simply result in altering the pattern of human stress to achieve a new intensity and tempo of operations. Cordesman and Wagner, 1996, p.25 edited to rephrase domain referents generically Cognitive Systems Engineering Laboratory:

7 The operator s job is to make up for the holes in designers work (Rasmussen, 1981)

8 Law of Fluency: Well -adapted cognitive work occurs with a facility that belies the difficulty of the demands resolved and the dilemmas balanced.

9 1. Over / under adapting when events challenge plans in progress. 2. Over-simplifications to cope with knowledge demands of complexity. 3. Shifts in technological capacity are exploited to transform systems law of stretched systems. 4. Incremental improvement in design leads to creeping complexity. 5. Gap filling -- the job of operators at the sharp end of systems is to make up for holes in designers/managers work. 6. Mis-design squeezes the sharp end exacerbating double binds and bottlenecks.

10 Images of Resilience, of Brittleness through an aggressive and innovative programme of cost cutting on its P36 production facility.

11 Images of Resilience, of Brittleness Space Mars series 1999, Ariane 501, Columbia, but mission control Health Care MAR knockout (IL), chemotherapy misadministration (OH), automatic dispensing (FL), but liver transplant Process control Tokai-mura criticality, P-36, but humulin manufacture Eli Lily

12 Incidents: stories of resilience or brittleness? assess adaptive capacity by observing how it responds to disruptions or challenges cross-checks in health care MAR knockout at hospital Israeli handling of victims of bus bombings Global hawk 1999 mishap MCC anomaly response

13 Patterns Across Cases all episodes include system adapting to disruption (e.g., decompensation pattern) shows how the system adapts sources for adaptation and boundary conditions disturbance, disruption and changes that challenge limits of its capacity to adapt. discriminate buffers from inefficiencies recognize drift under FBC pressure

14 cross-level interactions Downward, resilience is affected by how organizational context creates or facilitates resolution of pressures/goal conflicts/ dilemmas, for example, mismanaging goal conflicts or poor automation design can create authority-responsibility double binds for operational personnel. Upward, resilience is affected by how adaptations by local actors in the form of workarounds or innovative tactics reverberate and influence more strategic goals and interactions (e.g., workload bottlenecks at the operational scale can lead to practitioner workarounds that make management s attempts to command compliance with broad standards unworkable.

15 FBC Pressure: faster, better, cheaper Run up to Columbia

16

17 Boundary diagrams

18

19 Silver bullets or goal tradeoffs? Health Care Should Be: Safe avoiding injuries to patients from the care that is intended to help them. Effective providing services based on scientific knowledge to all who could benefit and refraining from providing services to those not likely to benefit (avoiding underuse and overuse, respectively). Patient-centered providing care that is respectful of and responsive to individual patient preferences, needs, and values and ensuring that patient values guide all clinical decisions. Timely reducing waits and sometimes harmful delays for both those who receive and those who give care. Efficient avoiding waste, including waste of equipment, supplies, ideas, and energy. Equitable providing care that does not vary in quality because of personal characteristics such as gender, ethnicity, geographic location, and socioeconomic status.

20

21 A common expression from military decision making: No plan survives contact with a disaster-in-the-making. our experience [is] that every response is totally different and causes unforeseen problems or opportunities. We have never gone to an actual response and used the equipment the way we thought we would. (Murphy & Burke, 2005, p. 4) How to be Prepared to be Surprised?

22 Potential for surprise is related to the next anomaly or event that practitioners will experience and how that next event will challenge pre-developed plans and algorithms in smaller or larger ways. To assess potential for surprise in a setting, ask how the above generalization applies? how do plans survive or fail to survive contact with events? search for the kinds of situations and factors that challenge the textbook envelope

23 Balancing Production/Safety Stories of Sacrifice decisions Help organizations decide when to relax production pressure to reduce risk Extra investment in safety is most needed when least affordable

24 Sacrifice Judgments

25 Sacrifice Judgments

26 Resilience: the ability to recognize and adapt to handle unanticipated perturbations that call into question the model of competence, and demand a shift of processes, strategies and coordination. textbook competence envelope: designed for uncertainties unanticipated perturbations: situations that challenge the envelope risks of under- or over- adaptation miss side effects of change operating riskier than they want or realize fundamental surprise

27 Stress-Strain Plot

28 Demand-Stretch Space Decompensation Pattern

29 Xtra sub-regions S

30 Restructuring Restructuring

31 Mis-Calibration sys actually operating, here Restructuring imagine sys is operating, here interpret incidents as degraded operating pt.

32 Fundamental Gap The gap between system as imagined and system as operated Mismatch of distant images of work and actual work practices is inevitable How to make visible and bridge the gap dynamically Effective organizations search out evidence to close this gap, respecting sharp end practice.

33 Dilemmas of Safety Organizations: 4 I s cold water and an empty gun to revise and reframe the organization s assessment of the risks it faced and the effectiveness of its countermeasures against those risks as new evidence accumulates. to monitor the organization s model of itself - the risk that the organization is choosing to operate nearer to safety boundaries than it realizes. to monitor risk continuously throughout the lifecycle of a system, so as to maintain a dynamic balance between safety and the often considerable pressures to meet production and efficiency goals.

34 The 4 I s of Safety Organizations: Independent, Involved, Informed, and Informative provide an independent voice that challenges conventional assumptions about safety risks within senior management. have constructive involvement in targeted but everyday organizational decision making (for example, ownership of technical standards, waiver granting, readiness reviews, and anomaly definition). actively generate information about how the organization is actually operating and the vectors of change that influence how it will operate. use information about weaknesses in the organization and the gap between work as imagined and work as practiced in the organization to reframe and direct interventions.

35 Enhancing Resilience: diagnosis and design ~ calibration: monitoring the gap between distant images of work and actual work? ~ reluctance to make sacrifice judgments? ~ learning from incidents? ~ how cross-checks work? ~ monitor/manage hidden entanglements (side effects problem) ~ reduce, reveal, focus heuristics for taming complexity

36 Re-framing & Broadening Checks

37 All joint systems are adapted to the potential for surprise in their fields of practice how do plans survive or fail to survive contact with events. Resilient systems are prepared to be surprised: set up to re-frame and re-conceptualize assessments of how well plans/models/ automata fit particular situations to be handled.

38 Engineering Resilience? (1) measure: distinguish sources of resilience from inefficiencies? (2) how to deploy resources for resilience at the right place and time (like reserves in a battle). (3) tools for organizations to signal how to make tradeoffs in the face of pressure to achieve throughput and efficiency goals,

39 Engineering Resilience? (4) tools for coordinating diverse perspectives, (5) techniques to visualize and anticipate the side effects of change and decisions on risk. (6) provide the 4th parameter for organizations: balance FBC and resilience

40 Provocations on Coping with Complexity Complexity lies in the effects / demands / difficulties spawned when one tries to control processes to reach goals. Complexity is read in the coping strategies, in learning how people have adapted and to what. How do we tell the difference between a useful simplification and an over simplification? When to shift or expand on a simplification heuristic? How do we learn / invent better strategies? Systems engineering - as decomposition/re-composition - has failed to cope with complexity. Incremental design can t deal with complexity; creeping complexity - inadvertent cumulative complexity -will result.

41 Provocations: Coping with Complexity Failures of architecture - architects stories of future effects are usually wrong (e.g. failure rate of large software projects. Adding flexibility to respond to complexity just transfers the work/ risk/difficulties to other stakeholders. Can t out-think complexity. How to stay humble in the face of complexity? How to stimulate constructive (expansive) adaptations? What is quality in a adaptive system? What is well adapted? People are a (or the) competence model for understanding adaptive processes; cellular systems, perceptual systems, learning systems, organizational dynamics, biological systems.

42 adaptive capacity is related to the potential for action in the future when conditions change or new events challenge old models

43