LESSONS LEARNED: SUBCONTRACTOR SELECTION BIAS

Transcription

1 Wednesday, November 8 8:30 9:30 a.m. LESSONS LEARNED: SUBCONTRACTOR SELECTION BIAS Presented by Rose Hoyle Construction Risk Engineer XL Catlin Cyrus A. Hund Risk and Finance Manager JE Dunn Construction Research suggests that the way people behave under conditions of risk is commonly at odds with normative theories of rational choice. Construction projects offer a rich setting for discussing the challenges of making good decisions under the pressure of risk and uncertainty. This session will use a case study approach to explore the human element of decision-making, relative to subcontractor selection, by describing the strong influence of cognitive bias at different stages of a typical construction project. Finally, we suggest some tools or other methodologies that may be helpful to improve the decision-making process in construction risk management. To print on both sides of the page, set your printer for duplex printing. Copyright 2017 International Risk Management Institute, Inc. 1

2 Rose Hoyle, P.E., CRIS, CLMP Construction Risk Engineer XL Catlin Ms. Hoyle is a construction risk engineer for XL Catlin s construction specialty product, Subcontractor Default Insurance. She brings more than 15 years of practical experience in engineering, construction, and risk management to the XL Catlin North America Construction insurance team. Before joining XL Catlin, Ms. Hoyle developed her expertise in construction management by spending more than a decade in operations with engineering and construction firms, namely Bohler Engineering and Turner Construction. Most recently, she leveraged her varied background and practical experience in the roles of claims and litigation consultant and expert witness on construction-related matters with consulting firm WCD Group. Earning a bachelor of science degree in civil engineering from Rutgers University and a master of science degree in civil engineering, focusing in construction engineering and management, from Stanford University, Ms. Hoyle is a licensed professional engineer in New Jersey, New York, and North Carolina. She earned designations of Leadership in Energy and Environmental Design Accredited Professional, Certified Litigation Management Professional (CLMP), and the IRMI Construction Risk and Insurance Specialist (CRIS ) certification. She now serves as the co-dean and faculty member of the CLM Claims College, School of Construction, and holds the associated credential of Certified Claims Professional. Ms. Hoyle is a New Jersey native and now lives in Greensboro, North Carolina, with her husband and stepdaughter. Cyrus A. Hund Risk and Finance Manager JE Dunn Construction Mr. Hund works on enterprise risk challenges that include subcontractor default risk and captive capital adequacy. He holds a master of business administration degree from Washington University in St. Louis and is a doctoral candidate in entrepreneurship and innovation at the University of Missouri Kansas City. His research has focused on understanding risk taking, risk alignment, decision-making in organizations, and the role of corporate entrepreneurship in competitive performance. 2

3 Lessons Learned: Subcontractor Selection Bias Daniel Kahneman ROSE HOYLE, P.E., CRIS Construction Risk Engineer XL Catlin Subcontractor Default Insurance CYRUS A. HUND Risk and Finance J.E. Dunn Construction 2 3

4 Method Apply work from Behavioral and Decision Science to Construction Risk Decision Making Subcontractor Selection is an especially good domain for decision analysis: 1. Decisions are made frequently 2. Subcontractor defaults can be high in magnitude. 3 Goal Offer a perspective useful for improving how subcontractors are selected. Exercise some problems with the human element of decision making. Explore promising prospects for improving subcontractor selection through better decision making. 4 4

5 Discussion Topics 1. Some Theories of Decision Making Under Conditions of Risk 2. Industry Prequalification & Subcontractor Selection Standards 3. The Case of Decision Making in Subcontractor Selection 4. Engineering Better Decision Support Systems 5 Important Note on References For brevity, all theories/ research is not exhaustively cited in slides Concepts are derived from the theories of many authors; errors in citing research are Cyrus s fault. Complete list of references and suggested reading is provided. 6 5

6 Theories of Decision Making Under Conditions of Risk 7 Risk Theory vs. Decision Maker Behavior Observed behavior is generally inconsistent with optimal behavior in theory! Example: People buy relatively more insurance against high probability, low magnitude risk than low probability, high-risk events. Example: A survey reveals Managers do not always believe risk is positively correlated with expected return! 8 6

7 Complexity of the World Limits Rationality REALITY BOUNDED RATIONALITY 9 A Continuum of Extreme Logic Applied by Decision Makers LOGIC OF CONSEQUENCES LOGIC OF APPROPRIATENESS Self-Interested (boundedly rational) individuals Fixed Preferences, Goals Fixed Identities (as in Roles) Behavior determined by calculation of alternatives Decision Makers match situation, roles, and social rules Follows rules that govern appropriate behavior for a given role Multiple Identities (Roles) Rules are institutionalized by social practice over time 10 7

8 Rule Following and Heuristics are not ALL bad! Rules / Heuristics developed over time embrace intelligence of experience Heuristics makes it easy to get things mostly right especially when decisions are as complex as they are in the construction industry 11 8

9 Industry Prequalification & Subcontractor Selection Standards What are the decisional guidelines that have been developed as bestpractices by the industry? 12 Cornerstones to Subcontractor Selection Best Prequalification Practices 1 Number Crunching 2 Ops Strength 3 Safety First 4 Quality Time 5 Reputation and Track Record 13 9

10 Cornerstones to Subcontractor Selection 1 Best Prequalification Practices Number Crunching FINANCIAL REVIEW SURETY BONDING CREDIT HISTORY Number Crunching should be conducted and/or reviewed by qualified individuals such as CFO, Controller, Accountant, etc. LEGAL 14 Cornerstones to Subcontractor Selection Best Prequalification Practices 2 Ops Strength Operational Depth Operational Specificity Operational Capacity Ops Strength parameters should be performed and/or reviewed by qualified individuals such as purchasing and operations staff. 2016, XL Group Ltd. All rights reserved. 15

11 Cornerstones to Subcontractor Selection 3 Best Prequalification Practices Safety First Does this subcontractor s approach to Safety align with yours? Safety parameters should be performed and/or reviewed by qualified individuals such as safety and/or risk management staff. 16 Cornerstones to Subcontractor Selection 4 Best Prequalification Practices Quality Time Does this subcontractor s approach to Quality align with yours? 1. Corporate Quality Management 2. Field Quality Management 2016, XL Group Ltd. All rights reserved. I MAKE YOUR WORLD GO 17 11

12 Cornerstones to Subcontractor Selection 5 Best Prequalification Practices Reputation and Track Record Post-bid / Pre-award interview Post-performance reviews (internal) Reference checks (external) Rumor Mill 18 12

13 The Case of Decision Making in Subcontractor Selection a case study example 19 Project Life Cycle PRECON PHASE Bid Day SUB-SELECTION PHASE Break Ground CONSTRUCTION PHASE POST- CONSTRUCTION PHASE Project Completion Internal References External References RMP Assignment Approval Checks/Balances Sub Mobilizes Sub Completes Scope PQ Bid List Bid Spread Analysis PQ Updates Descope Mtgs Award Recommendations Award Sub Performs Work Project Life Cycle Timeline 20 13

14 Project Life Cycle PRECON PHASE Bid Day SUB-SELECTION PHASE Break Ground CONSTRUCTION PHASE POST- CONSTRUCTION PHASE Project Completion Internal References External References RMP Assignment Approval Checks/Balances Sub Mobilizes Sub Completes Scope PQ Bid List Bid Spread Analysis PQ Updates Descope Mtgs Award Recommendations Award Sub Performs Work Project Life Cycle Timeline Decision Nodes = Opportunities for Bias Adverse Risk Path Default Situation 21 Example: Timeline of Decision Making Subcontractor Selection 22 14

15 Example 1: Unrealistic Optimism + Denial of Risk Estimates DENIAL OF RISK 23 Example 1: Unrealistic Optimism + Denial of Risk Estimates THE DATA: JE Dunn experiences a nonzero loss 1 in every $140M backlog. Distribution of Magnitude is WIDE YET: I ve NEVER had a subcontractor default on us in the 10 years I ve been building. Be we can control the risk! Case Study They ve been in business for 50 years! They won t have financial problems on this job. but they re one of the biggest in the industry! 24 15

16 Example 2: Limitation of Market Knowledge LIMITATION OF MARKET KNOWLEDGE DENIAL OF RISK 25 Example 2: Limitation of Market Knowledge Does not seem possible to estimate all opportunities, their risks, and alternatives Backlogs, manpower change Project timing is typically uncertain Case Study We know ALL of the best, qualified subcontractors in a market - Contractor to Owner 26 16

17 Example 3: Incentives, Goals and Authority LIMITATION OF MARKET KNOWLEDGE INCENTIVES, GOALS AND AUTHORITY DENIAL OF RISK 27 Example 3: Incentives, Goals and Authority Does every actor in a group seek to optimize shareholder value? When there are multiple identities, which values are emphasized? Case Study 1 Number Crunching 2 Ops Strength 3 Safety First 4 Quality Time 5 Reputation and Track Record Subcontractor A Subcontractor B Subcontractor C 28 17

18 Example 4: Anchoring + Problem of Satisficing LIMITATION OF MARKET KNOWLEDGE INCENTIVES, GOALS AND AUTHORITY DENIAL OF RISK ANCHORING, PROBLEM OF SATISFICING 29 Example 4: Anchoring + Problem of Satisficing Case Study 30 18

19 Example 5: Belief in a Causal Basis of Events LIMITATION OF MARKET KNOWLEDGE INCENTIVES, GOALS AND AUTHORITY DENIAL OF RISK ANCHORING, PROBLEM OF SATISFICING BELIEF IN A CAUSAL BASIS OF EVENTS 31 Example 5: Belief in a Causal Basis of Events Signs of Subcontractor Default and the Illusion of Control Example: Contractor displays signs of financial and operational stress Lack of manpower Suppliers and lower tiers calling with unpaid bills Case Study We will get them through. We will control their performance with a risk plan -Project Manager 32 19

20 Example 5: Belief in a Causal Basis of Events A Note on History and Decision Making How History Frames Decisions Great Outcomes We made great decisions! Imply Order and direction of events = Virtue Rewarded! Decision process demonstrably effective Mistakes, Poor Outcomes Decision makers: Emphasize External, unknowable factors Decision wasn t implemented well enough Stakeholders: What makes sense in hindsight was predictable! 33 Example 6: Prospect Theory and Subcontractor Default LIMITATION OF MARKET KNOWLEDGE INCENTIVES, GOALS AND AUTHORITY DENIAL OF RISK ANCHORING, PROBLEM OF SATISFICING BELIEF IN A CAUSAL BASIS OF EVENTS PROSPECT THEORY + SUB DEFAULT 34 20

21 Example 6: Prospect Theory and Subcontractor Default Positive (or Non-negative) Outcomes: Bids Come in Favorably Chance to save: A: 50% +$2M; 50% + $1M B: 50% +$4M; 50% +$0M C: 25% +$9M; 75% +$0M Risk Aversion! All Negative Outcomes: Subcontractor Replacement Bids You will lose: A: 100% -$3M B: 50% -$2M ; 50% -$5M C: 50% -$1M ; 50% -$6M Risk Seeking! Case Study 35 Example 6: Prospect Theory and Subcontractor Default A Subcontractor Defaults, and we are biased by how the problem is framed (Prospect Theory) Generally, we hate losing more than we like winning! 36 21

22 Example 6: Prospect Theory and Subcontractor Default A Note on Managerial Attitudes Toward Risk When surveyed, most managers reported that risk could be better defined in terms of an amount that is expected to be lost. Why is only associated with negative outcomes? 37 Example 6: Prospect Theory and Subcontractor Default LIMITATION OF MARKET KNOWLEDGE INCENTIVES, GOALS AND AUTHORITY DENIAL OF RISK ANCHORING, PROBLEM OF SATISFICING BELIEF IN A CAUSAL BASIS OF EVENTS PROSPECT THEORY + SUB DEFAULT SUCCESS-INDUCED BIAS 38 22

23 Example 7: Success Induced Bias; Problem of Risk Taking + Success When we have a positive experience, we tend to: Give ourselves credit for execution of our intentions See history as a causal chain of events Update our expectations for future performance and confidence in our decisions. We don t always: Attribute the correct causality of the positive event See outcomes as a draw from a probability distribution Case Study 39 23

24 Engineering a Better Approach to Decision Making Under Risk 40 One Diagnosis Heard from an SDI carrier: Increases in Subcontractor Defaults are due to a lack of proper execution of prequalification procedures, and lack of claims-prevention culture

25 Challenges of Building Decision Systems LOGIC OF CONSEQUENCES LOGIC OF APPROPRIATENESS We cannot expect to completely resolve decisional problems arising from socially embedded rule following or very (mentally) expensive cognitive odysseys that these logics follow but there do appear to be some opportunities for improvement. 42 An Update on the Decision Heuristics Standards haven t significantly changed since SDI inception Need to perform some investigation / review on decision guidelines Let s Discuss: Industry-wide investigation of prequal vs claims data can we (causally) explain subcontractor defaults? 43 25

26 Information Gathering Systems Regardless of what decisional processes are functioning, more (and more accurate) information would support decision makers. Information Gathering for Qualifying subcontractors seems to be a problem. 44 Clear Definitions, Awareness Needed Kahneman: Firms [contractors] are decision Factories 1. Be self-aware of the biases that naturally affect us. 2. Develop a language to diagnose problems. 3. Apply quality control measures in our decision process at different stages of decision making. 4. Ensure decisions are evaluated based on the quality and logic applied, not simply to outcomes. Develop quality control in decision making, like any good manufacturing process would have

27 46 27

28 References Cyert, R. M., & March, J. G. (1963). A Behavioral Theory of the Firm (2nd ed.). Englewood Cliffs, NJ. Daniel Kahneman, A. T. (1979). Prospect Theory: An Analysis of Decision under Risk. Econometrica, 47(2), Eisenhardt, K. M. (1989). Agency Theory: an Assessment and Review. Academy of Management Review, 14(1), Kahneman, D. (2003). MAPS OF BOUNDED RATIONALITY: A PERSPECTIVE ON INTUITIVE JUDGMENT. Prize Lecture, (December), Kahneman, D. (2015). Thinking, fast and slow. New York: Farrar, Straus and Giroux. Kahneman, D., & Frederick, S. (2002). Heuristics of Intuitive Judgment: Extensions and Applications. Heuristics of Intuitive Judgment: Extensions and Applications, Kahneman, D., & Tversky, A. (1979). Prospect Theory: An Analysis of Decision under Risk. Econometrica, 47(2), Langer, E. J. (1975). The illusion of control. Journal of Personality and Social Psychology, 32(2), March, J. G. (2014). Primer on decision making: how decisions happen. New Jersey: Free Press. March, J. G., & Shapira, Z. (1987). Managerial Perspectives on Risk and Risk Taking. Management Science, 33(11), Olson, B. J., Parayitam, S., & Bao, Y. (2007). Strategic Decision Making: The Effects of Cognitive Diversity, Conflict, and Trust on Decision Outcomes. Journal of Management, 33(2), Penrose, E. T. (2013). The theory of the growth of the firm. Mansfield Centre, CT: Martino Publ. Scopelliti, I., Morewedge, C. K., Mccormick, E., Min, H. L., Lebrecht, S., & Kassam, K. S. (2015). Bias blind spot: Structure, measurement, and consequences. Management Science, 61(10), Slovic, P., Finucane, M., Peters, E., & Macgregor, D. G. (2002). Rational actors or rational fools: implications of the affect heuristic for behavioral economics. Journal of Socio-Economics, 31, Slovic, P., Fischhoff, B., & Lichtenstein, S. (1982). Why Study Risk Perception? Risk Analysis, 2(2), Slovic, P., & Peters, E. (2006). Risk Perception and Affect. Current Directions in Psychological Science, 15(6), Tversky, A., & Kahneman, D. (1975). Judgment under uncertainty: Heuristics and biases. Utility, Probability and human Decision Making. Netherlands. Von Neumann, J., & Morgenstern, O. (2007). Theory of games and economic behavior: sixtieth- anniversary edition. Princeton: Princeton University Press. 47