Modeling Adversary Beliefs and Motivations

Transcription

1 CREATE Research Archive Current Research Project Narratives 2013 Modeling Adversary Beliefs and Motivations Heather Rosoff University of Southern California, Richard John University of Southern California, Follow this and additional works at: Part of the Applied Behavior Analysis Commons Recommended Citation Rosoff, Heather and John, Richard, "Modeling Adversary Beliefs and Motivations" (2013). Current Research Project Narratives. Paper This Article is brought to you for free and open access by CREATE Research Archive. It has been accepted for inclusion in Current Research Project Narratives by an authorized administrator of CREATE Research Archive. For more information, please contact

2 National Center for Risk and Economic Analysis of Terrorism Events University of Southern California Modeling Adversary Beliefs and Motivations October 1, 2012 to September 30, 2013 Heather Rosoff Richard John University of Southern California "This research was supported by the United States Department of Homeland Security through the National Center for Risk and Economic Analysis of Terrorism Events (CREATE) under Cooperative Agreement No ST-061-RE0001. However, any opinions, findings, and conclusions or recommendations in this document are those of the authors and do not necessarily reflect views of the United States Department of Homeland Security or the University of Southern California." Cooperative Agreement No ST-061-RE0001 Department of Homeland Security December 31, McClintock Avenue, RTH 314 ~ Los Angeles, CA ~ (213) ~

3 ABOUT CREATE Now in its tenth year of operation, the National Center for Risk and Economic Analysis of Terrorism Events (CREATE) was the first university-based Center of Excellence (COE) funded by University Programs of the Science and Technology (S&T) Directorate of the Department of Homeland Security (DHS). CREATE started operations in March of 2004 and has since been joined by additional DHS centers. Like other COEs, CREATE contributes university-based research to make the Nation safer by taking a longer-term view of scientific innovations and breakthroughs and by developing the future intellectual leaders in homeland security. CREATE's mission is to improve our Nation's security through research and development of advanced models and tools to evaluate risks, costs and consequences of terrorism and natural and man-made hazards and to guide economically viable investments in homeland security. We are accomplishing our mission through an integrated program of research, education and outreach that is designed to inform and support decisions faced by elected officials and governmental employees at the national, state, and local levels. We are also working with private industry, both to leverage the investments being made by the Department of Homeland Security in these organizations, and to facilitate the transition of research toward meeting the security needs of our nation. CREATE employs an interdisciplinary approach merging engineers, economists, decision scientists, and system modelers in a program that integrates research, education and outreach. This approach encourages creative discovery by employing the intellectual power of the American university system to solve some of the country s most pressing problems. The Center is the lead institution where researchers from around the country come to assist in the national effort to improve homeland security through analysis and modeling of threats. The Center treats the subject of homeland security with the urgency that it deserves, with one of its key goals being producing rapid results, leveraging existing resources so that benefits accrue to our nation as quickly as possible. By the nature of the research in risk, economics, and operations, CREATE serves the need of many agencies at the DHS, including the Transportation Security Administration, Customs and Border Protection, Immigration and Customs Enforcement, FEMA and the US Coast Guard.. In addition, CREATE has developed relationships with clients in the Offices of National Protection and Programs, Intelligence and Analysis, the Domestic Nuclear Detection Office and many State and Local government agencies. CREATE faculty and students take both the long-term view of how to reduce terrorism risk through fundamental research, and the near-term view of improving the cost-effectiveness of counterterrorism policies and investments through applied research. In 2011, the University of Southern California (USC) and a team of 23 partner institutions were awarded a new 5-year Cooperative Agreement resulting from a recompetition of the Center s charter. This annual report covers the third year under Cooperative Agreement 2010-ST-061-RE0001, the ninth year overall of CREATE s DHS funding, from October 2012 to September McClintock Avenue, RTH 314 ~ Los Angeles, CA ~ (213) ~

4 Table of Contents 1. Modeling Terrorist Beliefs and Motivations Overview Research Accomplishments Project 1. Model Simplification Project 2. Model Validation Project 3. APM Model Validation Research Products Publications Presentations Research Transition Education and Outreach Products Outreach Modeling Terrorist Beliefs and Motivations Overview The objective of this research is to further develop our pioneering approach to adversary threat assessment through the construction of random utility models of terrorist preferences. This work builds on previous research efforts (Rosoff, 2009; Rosoff & John, 2009) that have used decision analysis models and elicitation methods to: (1) construct of a value tree for a terrorist leader or organization using value focused thinking (VFT), (2) construct a random multi-attribute utility model (RMAUM) capturing trade-offs among conflicting objectives and single-attribute utility functions representing risk attitudes of terrorist leaders, (3) construct probability distributions capturing key uncertainties for terrorist leaders (e.g., attack success) and uncertainties in the utility function parameters provided by adversary experts. Understanding the objectives and motivations that drive terrorist group behavior is critical. Current methods for terrorism risk assessment focus on target vulnerability, terrorist capability and resources, and attack consequence. What many researchers have yet to consider is the influence of terrorist group values and beliefs in deciphering the root cause of their militant behavior. This understanding has the potential to contribute to probabilistic estimates of terrorist threats. During Year 9 we further developed the adversary preference modeling (APM) methodology by (1) continuing to validate the APM through a case study with political/social/advocacy groups PETA (Sea Shepherd) with an action oriented agenda that is driven by specific motives, values, and objectives, and (2). Explored simplified methods used in the elicitation of adversary objectives, values and beliefs, uncertainties and tradeoffs and, (3) testing the approach through the analysis of different terrorist organizations (Al Qaeda and Hezbollah). Research progress and accomplishments made throughout Year 9 are described in the subsequent sections. 2. Research Accomplishments 2.1. Project 1. Model Simplification We explore the use of the Boltzmann equation (Softmax) for simplifying the assessments required to apply APM methodology. This simplification includes construction of the value tree, attribute scale construction, and elicitation of probability distributions for capturing Page 3 of 15

5 uncertainty in the consequence matrix, single-attribute utility function parameters, and trade-off (weight) parameters. When applying the Boltzmann equation, we only needed to elicit median values when applying the APM methodology. Using the median values elicited, expected utility was calculated for each of the alternatives. That is, we worked with a proxy Al Qaeda leader to first develop an objective s hierarchy for Al Qaeda. Subsequently, we elicited median values for the consequence matrix, single-attribute utility function parameters and trade-off parameters. The calculated expected utilities for the proxy Al Qaeda leader can be found in Figure 1. No attack was associated with the highest expected utility. Alternatives E(U) No attack at all (baseline) 0.42 Anthrax Attack - Aerosolized anthrax release 0.38 Chemical Attack - Chlorine Tank Explosion 0.41 Biological Attack - Food Contamination 0.35 Explosive Attack - Bombing Using Improvised Explosive Devices 0.39 Chemical Attack - Blister Agent 0.36 Chemical Attack - Nerve Agent 0.47 Biological Attack - Pneumonic Plague 0.37 Radiological Attack - Radiological Dispersal Devices 0.41 Figure 1. Expected Utility Outputs for Proxy Al Qaeda Leader The calculated expected utilities were then put into the Boltzmann equation in order to get the probability of an alternative being selected. The Boltzmann equation calculates the probability of selection based on the following equation: Boltzman Equation (Softmax) P(a) = e Qt(a) τ n e Qt(a) τ b=1 In this equation the value of Qt(a) is the expected utility value obtained. The temperature parameter (τ) is a variable that is greater than 0. The reason for the temperature being greater than 0 is the fact that theoretically when it is 0, then the probability is equal to the expected utility, which suggests that the decision maker is using pure logic when considering each alternative. The purpose of the Boltzmann equation is to account for the randomness/lack of logic used by the decision maker when selecting an alternative. As the temperature increases, the probabilities reflect more randomness/illogical decision making. As seen in Figure 2, this is observed as the probability of each alternative gets closer and closer to each other as the temperature increases. However, although the probabilities of each alternative get closer to each other when the randomness increases, the ranking of alternatives (best-worst) does not change. The overall benefit of the boltzman equation is that it demonstrates the uncertainty in the proxy s decision making logic without having to elicit uncertainty parameters over all the model inputs. Page 4 of 15

6 P(Base) P(Chemical1) P(expl) P(Anthrax) P(Bio1) P(Chem3) P(Chem2) P(Bio2) P(Radio) Figure 2. Probability of success of each alternative using the Boltzmann equation The above process was carried out with 5 proxies, 3 for Al Qaeda Central, 1 for an Al Qaeda cell, and 1 for Hezbollah. This work was invited for presentation at The First conference on Validating Models of Adversary Behavior in Buffalo/Niagara Falls on June 23-26, In addition, this work was presented at the INFORMS Annual Meeting in Minnesota on October 6-9, This work is currently being written up for a special issue in Risk Analysis on validating models of adversary behavior Project 2. Model Validation Two types of validation analyses were conducted, construct and discriminant validity. Construct validity tests the degree to which two measures that theoretically should be related are, in fact, related. Discriminant validity tests whether measurements supposed to be unrelated are, in fact, unrelated. To assess convergent validity, we compared results across different modeling techniques (Figure 3). Our first analysis, compared probability calculations for Al Qaeda Central using the traditional APM simulation approach to the Boltzmann equation results set at a temperature of.05. We found the most preferred attack, No Attack, to be the same for both modeling techniques. Page 5 of 15

7 P (Attack) Boltzmann Simulation (temp =.05) Alternatives No Attack Chemical Attack - Chlorine Tank Explosion Chemical Attack - Blister Agent Biological Attack - Pneumonic Plague Biological Attack - Food Contamination Explosive Attack - Bombing Using Improvised Explosive Devices Anthrax Attack - Aerosolized anthrax release Chemical Attack - Nerve Agent Radiological Attack - Radiological Dispersal Devices Figure 3. Convergent validity assessment across different modeling techniques for Al Qaeda We conducted a similar analysis for an Al Qaeda cell with the temperature in the Boltzmann calculation changed to.035 (suggesting that the proxy terrorist leader decision maker is more rational/logical in his decision making). As seen in Figure 4, the results similarly showed the most preferred attack to be the same, an explosive attack (bombing using improvised explosive devices). P (Attack) Boltzmann Alternatives Simulation (temp =.035) Explosive Attack - Bombing Using Improvised Explosive Devices Chemical Attack - Blister Agent Biological Attack - Food Contamination Chemical Attack - Chlorine Tank Explosion Radiological Attack - Radiological Dispersal Devices Anthrax Attack - Aerosolized anthrax release Biological Attack - Pneumonic Plague No Attack Chemical Attack - Nerve Agent Figure 4. Convergent validity assessment across different modeling techniques for Al Qaeda Central To assess discriminant validity, we conducted analyses comparing results across different adversaries. We first compared probability of attack success calculations for Al Qaeda, an Al Qaeda cell, and Hezbollah based on the results of the simulation analyses. As seen in Figure 5, two of the three adversaries preferred no attack, while the third (Al Qaeda cell) preferred an explosive attack involving an IED. Page 6 of 15

8 P (Attack) AQ Central AQ Cell Hezbollah Alternatives Simulation Simulation Simulation No Attack Chemical Attack - Chlorine Tank Explosion Chemical Attack - Blister Agent Biological Attack - Pneumonic Plague Biological Attack - Food Contamination Explosive Attack - Bombing Using Improvised Explosive Devices Anthrax Attack - Aerosolized anthrax release Chemical Attack - Nerve Agent Radiological Attack - Radiological Dispersal Devices Explosive Attack (2) - Bombing Using Improvised Explosive Devices 0.02 Figure 5. Discriminant validity assessment across different adversaries We also assessed discriminant validity by comparing the results for 3 adversary values experts with different perspective about the same adversary Al Qaeda Central. Figure 6 shows that one AVE believed Al Qaeda Central preferred no attack, while the second believed they preferred an IED attack, and the third believed the group would prefer an anthrax attack. P (Attack) AQ Central AQ Central AQ Central Alternatives Simulation Simulation Simulation No Attack Chemical Attack - Chlorine Tank Explosion Chemical Attack - Blister Agent Biological Attack - Pneumonic Plague Biological Attack - Food Contamination Explosive Attack - Bombing Using Improvised Explosive Devices Anthrax Attack - Aerosolized anthrax release Chemical Attack - Nerve Agent Radiological Attack - Radiological Dispersal Devices Figure 6. Discriminant validity assessment across AVEs with different perspectives on Al Qaeda Central Overall, our analyses demonstrate convergent validity with respect to different modeling approaches. Both the simulation and SoftMax approach outputs identify the same alternative as the most preferred attack option for Al Qaeda Central and Al Qaeda Cell. In addition, our analyses demonstrate discriminant validity across different adversaries. Despite Al Qaeda (Central and Cell) and Hezbollah having shared objectives, there are distinct differences in attribute definition, matrix values, tradeoffs and risk attitudes that, in turn, result in different preferences for attack alternatives. Lastly, we also demonstrate discriminant validity across different AVEs. There are distinct differences in the way that each AVE perceives the attack alternative preferences of Al Qaeda Central. Page 7 of 15

9 Similar to Project 1, this work was invited for presentation at The First conference on Validating Models of Adversary Behavior in Buffalo/Niagara Falls on June 23-26, In addition, this work was presented at the INFORMS Annual Meeting in Minnesota on October 6-9, This work is currently being written up for a special issue in Risk Analysis on validating models of adversary behavior Project 3. APM Model Validation During Year 9 we built an MAU model based on what could be identified about the PETA online and from second-hand sources. We also interviewed PETA stakeholders to evaluate the accuracy and validity of the originally constructed second-hand model, and hence determining whether the APM modeling approach could be validated. With respect to the construction of the proxy model, basic research (online resources and literature) led to the development of a first set of objectives. PETA s overall goal is improving animals rights. This is characterized by the four core values expressed in their motto - Animals are not ours to eat, wear, experiment on, or use for entertainment, as well as an emphasis on pet spay and neutering, to reduce the numbers of unwanted animals. The four motto objectives are folded into the Minimize Cruelty to Animals sub-objective to avoid model over-specification. PETA must also consider growing its organizational power. Maximize Organizational Power considers income, members and minimizing cost, as well as maximizing new volunteers and influence. Lastly, Perception of Group refers to the group s delicate balancing act of radical positioning and working with the system, and outreach. The twin perception objectives are an attempt to model the tension of PETA s choice of positioning in the political spectrum; they want to bring the discussion to where they are, and have staked out a more radical position than the culture holds regarding treatment of animals. However, they want to be taken seriously; if they are perceived as too far out of the spectrum, then achieving successes inside the system via the traditional methods becomes impossible. The two metrics selected to characterize Perception of Group, maximize media hits and minimize negative press, cover the idea of getting attention but only the right kind of attention. Figure 7 is a graphic depiction of PETA s objectives hierarchy. Page 8 of 15

10 Improve Animal Rights Maximize Organizational Power Max Incoming Money Max New Members Max New Volunteers Maximize Influence Network Minimize Costs Minimize Cruelty to Animals Max Changes to Industry Practices Max Educational Materials Max New Laws Perception of Group Max Media hits Minimize Negative Press Figure 7. PETA Objectives Hierarchy Ultimately, from this working objectives hierarchy we developed the updated hierarchy in Figure 8 after discussions with a proxy from PETA. While he felt that the hierarchy in Figure 7 captured the organization well, following the discussions we simplified the hierarchy to capture the goals of greatest important. The greatest modification is that maximizing organizational power and perception of the group were removed from the hierarchy, as seen in Figure 8. This is largely because while these topics are discussed within PETA, they are not of significant concern or as described by the proxy critical to the success of the organization s goals. Page 9 of 15

11 Change Hearts and Minds of Public End Consumption of Animals Maximize change in corporate policy Maximize change to legislation Maximize educational materials Maximize media attention Maximize number of demonstrations Maximizing online presence/visibility Figure 8. Updated PETA Objectives Hierarchy Developed with Proxy Alternatives as well as metrics and scales were identified for each of the attributes. In addition, estimates of attitudes toward risk and trade-offs among various objectives, Figure 9 is a table that include the attributes and scales for each of the six attributes, as well as the single utility function parameters (SUF) used to capture the risk attitude of the proxy relative to each of the attributes. Range Midpoint SUF Parameters Minimum Maximum Level Utility a b c Maximize change in corporate policy Maximize change to legislation Maximize educational materials 0 1.6e+006 5e e-006 Maximize media attention Maximize number of demonstrations Maximizing online presence/visibility e e e-005 SUF Parameters: if c = 0, U(x) = a + bx, if c # 0, U(x) = a + b(exp(-cx)) Figure 9. Weights for Adversary Value Expert Preference Set Page 10 of 15

12 As suggested in Figure 9, the proxy can have different attitudes toward risk and in turn, may be willing to accept different levels of risk for different attributes. Utility functions are the measurement tool traditionally used to capture an individual s attitude toward risk. The direction of the utility function indicates whether that individual is risk averse, neutral, or seeking. Through the acquisition of certainty equivalents from the proxy, we were able to estimate the nature of their risk attitudes toward each attribute. For example, consider media attention (Figure 10a). The proxy appears to be slightly risk seeking because he perceives 10 media hits perceives to be of equal value to the gamble between his best (15) and worst (0) estimates of the number of desired media hits per campaign. Conversely, consider number of demonstrations in Figure 10b. The proxy appears to be slightly risk averse because he perceives 5 demonstrations to be of equal value to the game between his best (20) and worst (0) estimates of the number of desired demonstrations per campaign. 1 1 Utility Utility Maximize media attention (Media Hits) Figure 10a. Media Attention Maximize number of demonstrations (Number of demonstrations) Figure 10b. Demonstrations Also from the proxy, leaders assess the tradeoffs among the attack attributes. Each proxy rank ordered the attack attributes using swing weights. They assigned one attribute for which the change (swing) from worst to best represented the largest impact for the terrorist leader in terms of the overall objective committing a terror attack. All remaining attributes were assigned a percentage between 0 and 100% to reflect relative desirability of changing (swinging) a score from worst to best on that attribute. Figure 11 includes the 6 attributes, ordered by the median values of the normalized weights assessed with the most weight placed on maximizing media attention and the least weight assigned to maximizing the number of demonstrations. Measure Maximize media attention Maximizing online presence/visibility Maximize educational materials Maximize change in corporate policy Maximize change to legislation Maximize number of demonstrations Weight Figure 11. Swing weights for PETA proxy Page 11 of 15

13 Ultimately from these inputs, the proxy s utility distributions were derived and are presented in Figure 12. The two campaigns with the highest utility are demonstrations and undercover investigations. Alternative Demonstrations Undercover Investigation Online Ad Campaign PRINT Ad Campaign Corporate Campaign Publicity Stunt Policy Enforcement Utility End Consumption of Animals Figure 12. PETA proxy expected utilities across campaigns Given that the swing weights assigned to the six attributes were fairly close for maximizing media attention, online presence and education materials, we conducted some sensitivity analysis to see how the expected utility output might vary given variations in these weights. As seen in Figure 13, we found that undercover investigations, print ad campaigns, corporate campaign, publicity stunt and policy enforcement were not really sensitive to changes in the weight for maximizing online presence. However, if the weight for this attribute is maximized, we find that online ad campaign becomes the preferred campaign followed by undercover investigation. We also see that the demonstrations campaign is very sensitive to variations in this weight; demonstrations is the most preferred campaign when the weight on this campaign is low, but quickly becomes the least preferred campaign as the weight on this attribute increases Utility Online Ad Campaign Undercover Investigation PRINT Ad Campaign Corporate Campaign Publicity Stunt Policy Enforcement Demonstrations Percent of Weight on Maximizing online presence/visibility Measure Figure 13. Sensitivity analysis on weight for maximizing online presence Page 12 of 15

14 In a second sensitivity analysis (Figure 14) we found that changing the weight related to maximizing educational materials did not result in as much change as seen relative to maximizing online presence. Whether or small amount or a significant amount of weight is placed on maximizing educational materials, demonstrations remains the preferred campaign. However, if no weight is placed on maximizing educational materials, undercover investigation becomes the preferred campaign followed by demonstrations and online ad campaigns Utility Demonstrations Corporate Campaign Online Ad Campaign Undercover Investigation PRINT Ad Campaign Publicity Stunt Policy Enforcement Percent of Weight on Maximize educational materials Measure Figure 14. Sensitivity analysis on weight for maximizing educational materials Lastly, the results generated from the research team and the PETA proxy were compared for discriminant validity. Essentially, we are comparing the results for 2 adversary values experts with different perspective about PETA. Figure 15 shows that one AVE believed PETA would prefer pursuing an undercover investigation, while the other preferred organizing demonstrations. This outcome demonstrates discriminant validity across different AVEs, as there are distinct differences in the way that each AVE perceives the campaign alternative preferences of PETA. E(U) of Attack PETA PETA Simulation Simulation Corporate campaign Demonstrations 0.37 Online ad campaign 0.26 Policy Enforcement 0.04 Print ad campaign Publicity stunt Undercover investigation Purchase stock - McDonalds 0.37 Figure 15. Discriminant validity assessment across AVEs with different perspectives on PETA Page 13 of 15

15 All information provided herein is being prepared for presentation at the upcoming Society for Risk Analysis Conference (SRA) in December Subsequently, the material will be written up for journal publication Research Products Research Product Metrics # # of peer-reviewed journal reports published 0 # of peer-reviewed journal reports accepted for publication 0 # of non-peer reviewed publications and reports 1 # of scholarly journal citations of published reports 19 # of scholarly presentations (conferences, workshops, seminars) 5 # of outreach presentations (non-technical groups, general public) Publications John, R. S. & Rosoff, H. (2011). Modeling effects of counterterrorism initiatives for reducing adversary threats to transportation systems. Journal of Homeland Security, Proceedings of the 2011 DHS Science Conference 5th Annual University Network Summit, focused on Catastrophes and Complex Systems: Transportation, Washington, D.C., March 30 April Presentations John, R.S., Rosoff, H. & Cui, J. Combined MAU and SOFTMAX Modeling of Uncertainty in Adversary Behavior. Decision Analysis Society sponsored presentation at the annual meeting for the Institute for Operations Research and Management Science (INFORMS), Minneapolis, MN, October 6-9, Rosoff, H. and John, R.S. Validation of Adversary Utility Assessment by Proxy. Decision Analysis Society sponsored presentation at the annual meeting for the Institute for Operations Research and Management Science (INFORMS), Minneapolis, MN, October 6-9, Rosoff, H. and John, R.S. Contrasting Decision Models for Different Adversary Groups. Multiple Criteria Decision Making, sponsored presentation, Institute for Operations Research and Management Science (INFORMS), Minneapolis, MN, October 6-9, John, R.S. & Rosoff, H.R. Simplifying and Validating Adaptive Adversary Modeling. The First conference on Validating Models of Adversary Behavior, Buffalo/Niagara Falls, June 23-26, John, R. S. & Rosoff, H. Modeling and exploiting adversary risk attitudes and trade-offs among conflicting objectives. Joint Decision Analysis Society and Emergency Management sponsored presentation at the annual meeting for the Institute for Operations Research and Management Science (INFORMS), Phoenix, Az., October 14-17, Rosoff, H. & John, R. S. Modeling effects of counterterrorism initiatives on reducing adversary threats. Multiple Criteria Decision Making sponsored presentation at the annual meeting for the Institute for Operations Research and Management Science (INFORMS), Phoenix, Az., October 14-17, Page 14 of 15

16 3. Research Transition Both the TSA and FEMA have expressed an interest in this work. Meetings are scheduled with both departments in early December 2013 in Washington D.C. 4. Education and Outreach Products Education Initiatives (Please detail below) # # of students supported (funded by CREATE) 1 # of students involved (funded by CREATE + any other programs) 1 # of students graduated NA # of student theses or dissertations NA # of contacts with DHS, other Federal agencies, or State/Local (committees) 2 # of existing courses modified with new material NA # of new courses developed NA # of new certificate programs developed NA # of new degree programs developed NA Funded by CREATE Xi Zheng, Undergraduate Student, USC 5. Outreach In meetings scheduled for December, CREATE researchers plan to establish more direct lines of communication with the TSA and FEMA. Page 15 of 15