Brydon Dewitt Jared Schmidt

Transcription

1 Brydon Dewitt Jared Schmidt

2 Types of errors and failures Examples of failures Causes of system failures Laws and regulations Professional responsibility Risks Risk vs. Reward What is acceptable? Who should make such determinations?

3 1. Cause 2. Seriousness of effects o Individual vs large scale effects 3. Application area o Safety-critical applications (e.g. Hospital machines)

4 Errors with billing systems One IL couple was billed for $68 billion in tax penalties because the IRS generated erroneous bills after modifying its billing program. Inaccurate and misinterpreted database information Mistakenly being on the federally sex offender list because you live somewhere where a sex offender previously lived o Better testing methods can make these programming errors avoidable.

5 1) Large population Poor database design 2) Automated processing Inability to recognize special cases 3) Overconfidence in data accuracy Unrealistic or inadequate understanding of risks in a complex system 4) Failure to update and correct data 5) Lack of accountability for errors

6 Modern communications, power grids, medical, financial, retail, and transportation systems rely heavily on computer systems. o o o Communications systems Transportation systems Stock market Adequate planning and backup provisions need to be made in case of failures.

7 Several companies have gone bankrupt after investing in computer systems that failed. A few dozen companies that bought an inventory system called Warehouse Manager which reported incorrect data and did not place orders correctly. Despite receiving numerous complaints from several clients, the company dishonestly said the problems were unique to the customers.

8 Florida voters in 2000 were not able to vote because their names matched those in a database of felons. Help America Vote Act (2002) Authorized $3.8 billion to improve voting systems Voting machines still failed because of common problems: o Insufficient planning, testing, and training o Security issues

9 Many systems are trashed before they ever fully conceived. Require more money and time than anticipated Ford Motor purchasing system ($400 million) Hotel and rental car businesses ($125 million) FBI Virtual Case File system ($170 million) IRS tax modernization plan ($4 BILLION) About $1 trillion spent worldwide on hopelessly inadequate projects.

10 Complex $193 million baggage handling system did not work as planned Delayed the opening of the airport for a year Cost $30mil/month in bond interest and operating costs o Insufficient time for development and testing Denver changed design specifications after the project began.

11 Legacy systems are reliable, but inflexible. Often used by banks, airports, government agencies, and infrastructure services o Implementing a new system would be expensive and disruptive Y2K Problem Two-digit representation of year caused errors in many systems o $308 billion spent worldwide ($416 billion in 2012 USD)

12 Military, power plant, aircraft, medical applications Between , four A320s crashed. o Too much confidence in fly-by-wire autopilot program Traffic control systems o Must work in real time o Many different devices and systems working together

13 Computer systems fail for two general reasons: 1) The task they are intended for is inherently difficult o Errors and ambiguity in program specifications 2) The job is done poorly o Not considering how the system interacts with real users or real world problems o Dishonest reliability or safety estimates to cover up flaws and avoiding expenses

14 Good software engineering techniques at all development stages are important. ACM/IEEE-CS Software Engineering Code of Ethics and Professional Practice ACM Code of Ethics and Professional Conduct (Appendix A) Software engineers who work on safety-critical applications should have special training. Long, careful planning and good management o Discover and modify unrealistic goals during design

15 Well-designed user interfaces can avoid many computer-related problems. Should be consistent Provide clear instructions and error messages Check input to avoid typos AA Flight 965 (1995) 159 deaths o Caused by overconfidence in the system and poor user design

16 Adequate, well-planned testing is critical. Unfortunately, testing is not always thoroughly done because of budget or deadlines. This is foolish, risky, and irresponsible. Beta testing Testing by regular users in a real-world environment NASA s independent verification and validation practice o Testing and software validation done by independent company to find flaws

17 Building fail-safes into systems to avoid errors Especially important when lives and fortunes are at stake Even with extensive testing, there is no guarantee of bug free code. Errors in tests and system recovery routines

18 Lawsuits and settlements Many contracts for business computer systems limit or waive the right for the consumer to recover losses. Fraud and misrepresentation are not part of a contract. Liability and criminal laws in the US are flawed and often abused. Well-designed laws should: 1) Not be so strict that they discourage innovation 2) Provide incentives to produce good systems

19 Licensing agreements typically indicate software is bought as-is. Some agreements also include provision that the vendor may choose the states in which any legal disputes are settled. TWO VIEWS: Agreements are binding contracts Requirements for warranties would raise: Increase price of testing, development, and insurance, hurting small companies Reduce innovation and new software development Software should have mandatory warranties Strict legal requirements for warranties would: Encourage responsibility and lead to better software Protect the consumer from unfair agreements

20 Radiation-therapy machine Made by Atomic Energy of Canada Limited (AECL) Used between 1985 and 1987 Radiation overdoses

21 Cancer patients Family/friends Medical centers Technicians handling the machine AECL FDA

22 Section 1.2 Avoid harm to others. Section 2.5 Give comprehensive and thorough evaluations of computer systems and their impacts, including analysis of possible risks.

23 Utilitarian Rights-based Justice-based

24 Take machines offline until fixed AND provide an alternative treatment Best attempt at satisfying all stakeholders' rights Morally, the right thing to do

25 Therac-25 bypassed rigorous FDA testing o AECL declared pre-market equivalence and was able to market the machines. Strict regulation can keep bad products out of the market. o Strict standards can inhibit progress. o Responsibility would be shifted from the manufacturer to the government. o The regulation processes are expensive and susceptible to bureaucracy.

26 Some feel that software developers should be required to obtain a mandatory license. Economic analyses have shown licenses have no effect on improving quality. Opponents of mandatory licensing argue it violates an individual s negative right to work.

27 Many companies strive to uphold an ethical policy and pay for problems and damages. Intuit has paid interest and penalties for flawed income-tax software. Business pressure can also be a good tool for insistence on quality. o Customer satisfaction is critical to success o Good customer relations maintain a healthy company reputation

28 Are we too dependent on computers? o Computers are tools; we are far better off with them than without. Many failures stem from: o Lack of responsibility o Ignorance o Overconfidence

29

30 Baase, Sara. A Gift of Fire: Social, Legal, and Ethical Issues for Computing and the Internet. Upper Saddle River, NJ: Pearson, Print. Calleam Consulting. Denver Airport Baggage Handling System Case Study Web. Leveson, Nancy. An Investigation of the Therac-25 Accidents. IEEE Computer, Vol. 26, No. 7, July 1993, pp