Background and Incidents Denials and Coverups Investigation and Action Technical Causes Organizational Causes Stakeholder R.

Transcription

1 The Therac-25 Alice E. Fischer October 22, 2010 The Therac /34

2 I. Background and Incidents The Therac /34

3 Background The Therac 25 was a Canadian product... A medical linear accelerator used to treat cancer Put on the market in 1983 Preceded by Therac-6 and Therac-20 (pre-1981) Designed by AECL (Atomic Energy Commission Limited) Bimodal use: Electron beam and X-ray modes. Earlier models were not bimodal. Completely dependent on software for operation. Safety analysis excluded an analysis of software. Eleven machines were installed, 6 in Canada and 5 in the U.S. The Therac /34

4 The Cast of Characters (Stakeholders) AECL, the manufacturer and distributor The programmer who wrote the code Eleven medical centers that purchased Therac-25 Several Therac-25 operators 11 Radiation MDs in charge of the treatments 11 Hospitals, medical physicists and technicians Many patients receiving treatments The U.S. Food and Drug Administration Canadian Radiation Protection Bureau (CRPB), later merged into the Bureau of Radiation and Medical Devices The Therac /34

5 The Six Accidents June 1985, Georgia, severe radiation burns (front and back), swelling, great pain, skin sloughing off, shoulder paralyzed. July 1985, Ontario, patient lost hip, then died of cancer. Would have needed hip replacement. December 1985, Yakima WA, burns in a grid pattern. Chronic skin ulcer, pain, necrosis under skin. Damage was repaired surgically, has a minor disability and scarring. March 1986, Texas, patient died from complications after 5 months of pain April 1986, Texas, patient died from overdose 3 weeks later January 1987, Yakima, patient died in April from complications related to overdose. The Therac /34

6 II. Denials and Coverups The Therac /34

7 Marietta Georgia: July, 1985 Kennestone Regional Oncology Center The patient was severely burned and ended up with a paralyzed arm and shoulder. Hospital and technician denied any wrongdoing. They continued to send the patient for more treatments. The patient filed a lawsuit. AECL denied knowledge of incident and said it was impossible for the equipment to operate in the wrong mode. Nobody notified the FDA. Other Therac users were not notified. The Therac /34

8 Hamilton, Ontario: July, 1985 Ontario Cancer Foundation The patient lost a hip, then died soon from an aggressive form of cancer. The hospital contacted AECL. AECL sent a service engineer to investigate. The FDA and Canadian Radiation Protection Bureau (CRPB) were informed. Hospitals were told to visually confirm the treatment parameters until further notice. AECL could not replicate the malfunction. Suspected a microswitch. Looked for problems in the turntable positioning. AECL announced they had improved safety by 5 orders of magnitude. The hospital hired an independent investigator. The Therac /34

9 Canadian Investigation: 1985 As a result of the Hamilton accident, the head of advanced X-ray systems in the CRPB, Gordon Symonds, wrote a report that analyzed the design and performance characteristics of the Therac-25 with respect to radiation safety. Besides citing the flawed microswitch, the report faulted both hardware and software components of the Therac s design. It concluded with a list of four modifications to the Therac-25 necessary for minimum compliance with Canada s Radiation Emitting Devices (RED) Act (which gave government officials power to ensure the safety of radiation-emitting devices). The Therac /34

10 Yakima, Washington: December, 1985 Yakima Valley Memorial Hospital Treatment left burns in a grid pattern, leading to a chronic skin ulcer, pain, necrosis under skin. Damage was repaired surgically, has a minor disability and scarring. Hospital contacted AECL, spoke to a technical support supervisor. AECL replied that the damage could not have been caused by the machine or by an operator. Its claim was supported by 2 pages of details why such an accident was impossible. AECL claimed that there were no other similar incidents. The Therac /34

11 Tyler, Texas: March, 1986 East Texas Cancer Center The patient died from complications after five months of pain. The video monitor was unplugged and the audio unit was broken. The Therac commonly had glitches and delays. After the first treatment, the machine displayed an error code and showed a severe under-dose. The operator (as usual) hit the proceed button. The patient went to the door and pounded on it. He was immediately examined by a physician. The machine was examined next day by AECL, ETCC AECL again claimed there were no other incidents. They thought an electrical glitch had caused the problem. The Therac /34

12 Tyler, Texas: April 1986 East Texas Cancer Center, one month after the previous incident. The patient died 3 weeks later from the radiation overdose. The hospital physicist recognized the seriousness of the problem immediately and gathered information. ETCC took the machine out of service. Called AECL. The physicist and technician documented the procedure. ETCC Informed AECL that speed was necessary to cause the trouble. AECL admitted to previously discovering a cursor-up problem but thought they had fixed it. The FDA was notified, and notified other users. The Therac /34

13 Yakima, Washington: January, 1987 Yakima Valley Memorial Hospital 9 months after the nature of the problem was understood! The patient died in April from complications of the overdose. After 5 or 6 seconds of treatment, the machine paused and displayed an error message that may have disappeared quickly. The operator was not sure. The treatment console displayed no dose. The operator pressed the proceed button. The machine shut down again displaying the error message Flatness. The operator went to check on the patient, who reported burning. Four days later, the skin showed burns in a striped pattern. The Therac /34

14 III. Investigation and Action The Therac /34

15 AECL report: April 1986 The AECL issued a report following their investigation at ETCC The frying sound heard by patients happened when the ion chamber became saturated. Effective immediately, and until further notice, the key used for moving the cursor back through the prescription sequence (i.e. cursor UP, inscribed with an upward pointing arrow) must not be used for editing or any other purpose. To avoid accidental use of this key, the key cap must be removed and the switch contacts fixed in the open position with electrical tape or other insulating material. Disabling this key means that, if any prescription entered is incorrect, then a Reset command must be used and the whole prescription reentered. The Therac /34

16 FDA Response: May, 1986 We have reviewed AECL s April 15 letter to purchasers and have concluded that it does not satisfy the requirements for notification to purchasers of a defect in an electronic product. Specifically, It does not describe the defect nor the hazards associated with it. The letter does not provide any reason for disabling the cursor key and the tone is not commensurate with the urgency for doing so. In fact, the letter implies the inconvenience to operators outweighs the need to disable the key. We request that you immediately renotify purchasers. The Therac /34

17 Resolution: July, 1987 The software race condition was understood. AECL agreed to fix it and make a number of other modifications to the machine. Those changes included safety interlocks. Operator error messages were to be written in clear English instead of cryptic codes. Up-to-date documentation was to be provided. The Therac /34

18 IV. Technical Causes The Therac /34

19 Rapid editing of an error. Every incident involved a skilled and experienced Therac operator. The prescription was initially entered incorrectly, and the cursor reached the final position that signals completion of the prescription. This triggers the machine to start the setup process. Then the operator noticed an error and used the up-cursor key to revisit the wrong field and correct it, then used cursor-down to get back to the done position. If the correction task was fast enough, the control process would not be aware that anything had changed. However, the concurrent setup had already begun. The incorrect data was used to set one of the radiation parameters. Later, the corrected data was used to set the other. The Therac /34

20 The second Yakima accident Bad programming practice: It was a one-byte variable ( ) was used for indicating whether the upper collimator was in position. It was incremented every time its parent process was rescheduled. It eventually overflowed and wrapped to 0. A non-zero value indicated setup was not complete. A 0 indicated that the positioning was complete. Instead of incrementing the variable, it should have been set to some fixed non-zero value. The Therac /34

21 Contributing Factors: Design and Construction The system was developed over a span of several years without adequate structure. There was no overall specification and no documented test plan. The model behind the software did not take concurrency and timing issues into account. No redundancy was designed into the system. The code was written by a single developer with no significant software review. There was no software audit trail. Code was written in assembly code (probably avoidable in 1983). The Therac /34

22 Contributing Factors: Code Reuse The software for Therac-25 was ported from the Therac-20 without a new analysis and without consideration of the changes in the hardware. Hardware interlocks on the Therac-20 were removed and replaced by software checks on the Therac-25. The Therac /34

23 Contributing Factors: Concurrency Extensive use was made of concurrent programming with shared variables. Synchronization was not implemented well, so the system was prone to race conditions. The code relied on status flags that were set and reset by multiple subsystems for more than one reason. It used phantom tables for getting dosage values if too low a value was entered. The Therac /34

24 Contributing Factors: Testing, Usability, and Operation Simulation and unit testing were inadequate. The user interface was bad, error messages were cryptic and poorly documented. As a consequence of common software glitches and hiccups, operators were desensitized and ignored error comments. Standard operating procedure was to keep pressing Proceed. The machine was operated with safety systems disabled: broken audio and video. Errors were not reproduced easily because half-trained people did the testing. The Therac /34

25 V. Organizational Causes The Therac /34

26 Problems at AECL No software quality assurance procedures were in place. In a letter to the FDA, AECL erroneously or falsely claimed a lawsuit notification was received in March 1986 instead of November 1985, as was the case. After the second accident (Hamilton), overexposure was blamed on faulty microswitch. AECL misrepresented facts and did not assume responsibility for its product after the first death. AECL failed to replicated the error but, after minor hardware and software modifications, they claimed five orders of magnitude safety improvement. The Therac /34

27 More Problems at AECL AECL ignored Symonds and CRPBs recommendations to modify the data entry procedure. Number of retries allowed changed from five to three. After accident #3, AECL claimed that no operator error or system malfunction could have caused the overexposure. In March, 1986, after the fourth incident, AECL s engineer completed inspections in one day. Their quality assurance manager told the ETCC Physicist that AECL knew of no accidents involving radiation overexposure by the Therac-25. AECL s first recall notification letter to users was inadequate and confusing. The Therac /34

28 Organizational Problems Elsewhere One of the accidents occurred when audio and video monitoring of patients was broken/turned off. The FDA (June 1987) gave the AECL a Class II recall, meaning the risk of serious consequences is remote, or injuries are reversible. Nine months after the problem was diagnosed, another accident happened. The Therac /34

29 VI. Stakeholder Responsibilities The Therac /34

30 Manufacturer of Safety-Critical Equipment Responsibilities: Taking away the hardware locks was a false economy. Train programmers and engineers that safety comes first. Thoroughly check and test software and hardware before release. Use a responsible design, development, and testing process. But if your model is wrong, testing won t detect the problem. One guy. No code review. Were there unit tests? There should have been reviews at different levels, all should have been documented. Keep track of code reviews and equipment design changes. For customers, provide high-quality documentation and descriptions of error conditions. The Therac /34

31 Manufacturer Responsibilities (Continued) Respond in a truthful, competent, appropriate and timely manner to problems. Announce recalls promptly and clearly. Establish good communication among departments within the company. Make sure there are proper checks and balances on responsibilities of departments. Coders, legal staff, managers, complaint desk Follow best practices for customer support and complaints The Therac /34

32 Programmer / Programming Department Responsibilities Use a responsible design and development process including Use peer review during coding. Do code review between versions of the machine. Use modular design. Implement a commit step before using a life-critical machine. Create unit tests and a test suite. Do regression testing over a long period of time. Synchronize the concurrent processes properly. Document and comment the code. Design a user interface that is clear, especially with respect to errors. The Therac /34

33 Hospitals Responsibilities: Make the environment safe for the patients (Yakima, video broken) Don t use the equipment if the safety systems are not functional. (Intercom) Contact the manufacturer in case of problems ( The GA hospital didn t think problem was serious) Make sure that maintenance checks are timely. ( Safety equipment) Follow-up on treatments. (Might have prevented later incidents.) The Therac /34

34 Regulatory Agencies Be prompt if it is a life-threatening problem. Do not minimize the severity of a problem that has caused a death. Issue a warning even if a full investigation is not complete. Follow the rules about recalls. The Therac /34