Computational Theory and Cognitive Assistant for Intelligence Analysis

Transcription

1 Computational Theory and Cognitive Assistant for Intelligence Analysis Gheorghe Tecuci, David Schum, Mihai Boicu, Dorin Marcu, Katherine Russell Learning Agents Center, George Mason University The Sixth International Conference on Semantic Technologies for Intelligence, Defense, and Security STIDS Fairfax, VA, 17 November 2011

2 Overview Computational Theory of Intelligence Analysis Hypotheses Analysis with TIACRITIS Cyber Insider Threat Discovery and Analysis Future Research and Development Discussion 2

3 Computational Theory of Intelligence Analysis Aristotle Galileo Newton Locke Whewell Peirce Wigmore Hypothesis Likelihood of Hypothesis Evidence marshaling What hypothesis would explain these observations? (Abduction: O possibly H) Hypothesis-driven collection What evidence is entailed by this hypothesis? (Deduction: H necessarily E) Multi-INT fusion What is the likelihood of the hypothesis? (Induction: E probably H) Observations Evidence in search of hypotheses Hypotheses in search of evidence New Evidence Evidentiary tests of hypotheses Implemented in TIACRITIS, a Disciple and web-based cognitive assistant that supports analysts in coping with the astonishing complexity of intelligence analysis. 3

4 Discovery of Evidence, Hypotheses, and Arguments H 5 : A dirty bomb will be set off in the Washington DC area H 4 : build dirty bomb Abduction H 3 : stolen by terrorist organization H 2 : stolen with truck H 3 : stolen by competitor H 2 :misplaced H 3 : stolen by employee H 2 : used in project TIACRITIS Hypothesis-driven collection H 11 : was in warehouse Deduction H 1 : cesium-137 canister is missing from warehouse H 12 : is not in warehouse H 13 : was not checked-out from warehouse H 11 : almost certain Multi-INT fusion H 1 Induction H 12 : very E very H 13 : very H 1 : missing H 1 : not missing E: Article on cesium-137 canister missing What hypothesis would explain this observation? Evidence in search of hypotheses P Possibly Q Security camera showing a person loading a container into an U-Haul panel truck. Ralph, the supervisor of the warehouse, reports that the cesium-137 canister is registered as being in the warehouse, that no one at the XYZ Company had checked it out, but it is not located anywhere in the hazardous materials locker. He also indicates that the lock on the hazardous materials locker appears to have been forced. Assuming that this hypothesis is true, what other things should be observable? Hypotheses in search of evidence P Necessarily Q What is the likelihood of the hypothesis based on the available evidence? Evidentiary tests of hypotheses P Probably Q 4

5 H 5 : A dirty bomb will be set off in the Washington DC area H 4 : build dirty bomb H 3 : stolen by terrorist organization H 2 : stolen with truck very H 1 : missing very E: Article on cesium-137 canister missing H 2 :misplaced H 3 : stolen by competitor H 2 : used in project TIACRITIS H 21 : cesium-137 canister missing from warehouse H 3 : stolen by employee Truck entered company H 2 : cesium-137 canister stolen with truck very The record, made by Sam, security guard at the XYZ Company, that a panel truck bearing Maryland license plate number MDC-578 was in thexyzparkingareaon the day before the discovery of the missing cesium-137 canister. almost certain H 22 : missing canister stolen with truck Cesium-137 canister stolen from locker very Scenario: Truck entered company, canister stolen from locker, canister loaded into truck, truck left with canister. almost certain The lock appears to have been forced. Locksmith Clyde s reports that the lock was forced. Multi-INT fusion Hybrid spiral reasoning Learning analytic expertise Hypothesis-driven collection very Cesium-137 canister loaded in truck Heuristic power of evidence-suggested scenarios Truck left with canister very very Security camera showing a person loading a container into an U-Haul panel truck. 5

6 Analyst s Cognitive Assistant: Disciple/TIACRITIS Analytic Assistance Supports intelligence analysts with evidence marshaling and hypotheses generation, hypothesis-driven evidence collection, multi-int hypotheses testing, collaboration with other analysts and experts, and intelligence sharing. Learning Rapidly acquires and maintains analytic expertise which currently takes years to establish, is lost when experts separate from service, and is costly to replace. Textbooks, Courses, Case Studies Tutoring Helps new student analysts learn the critical thinking skills for evidence-based hypotheses generation and analysis, through a hands-on approach Introduction to Intelligence Analysis: A Hands-on Approach 13 case studies 4 course versions A Practicum in Evidence Marshaling and Argument Construction In preparation complex case studies Modeling the Behavior of Violent Extremists 24 case studies 5 course versions 6

7 Overview Computational Theory of Intelligence Analysis Hypotheses Analysis with TIACRITIS Cyber Insider Threat Discovery and Analysis Future Research and Development Discussion 7

8 Illustration of the Use of TIACRITIS Hypothesis Formulation 1. Analyst formulates the hypothesis analysis problem in English 2. Analyst the selects objects and actors 3. TIACRITIS learns reusable patterns Assess whether a?o1 was stolen from the?o2 with the?o3. 4. Learned patterns speed-up future analyses

9 Hypothesis Decomposition 1. Analyst and TIACRITIS decompose the initial problem down to the level of elementary hypotheses to be evaluated based on evidence H 2 : cesium-137 canister stolen with truck H 21 : cesium-137 canister missing from warehouse Truck entered company H 22 : missing canister stolen with truck Scenario: Truck entered company, canister stolen from locker, canister loaded into truck, truck left with canister. Cesium-137 canister stolen from locker Cesium-137 canister loaded in truck Truck left with canister 2. TIACRITIS learns reasoning patterns from decompositions defined by analyst 3. TIACRITIS may suggest reformulations to reuse reasoning patterns 4. TIACRITIS may suggest decompositions

10 Evidence Collection 2. Analyst associates search criteria with elementary hypotheses 1. Elementary hypotheses to be evaluated based on evidence 3. Search engines are invoked to identify relevant evidence 10

11 Evidence Representation and Use Analyst collects evidence items and associates them to hypotheses 11

12 Automatic Analysis of Elementary Hypotheses TIACRITIS automatically generates the evidence-based analysis 12

13 Drill-down Assessment 2. TIACRITIS computes the inferential force on elementary hypotheses 1. Analyst assesses the relevance of evidence and the believability credentials, at the desired level of detail

14 Hypothesis Testing 3. Final result H 2 : cesium-137 canister stolen with truck very 2. TIACRITIS computes the inferential force on hypotheses H 21 : cesium-137 canister missing from warehouse Truck entered company very almost certain Cesium-137 canister stolen from locker H 22 : missing canister stolen with truck very Scenario: Truck entered company, canister stolen from locker, canister loaded into truck, truck left with canister. almost certain Cesium- 137 canister loaded in truck Truck left with canister very very 1. Analyst selects composition functions (min, max, average, weighted sum) 14

15 Overview Computational Theory of Intelligence Analysis Hypotheses Analysis with TIACRITIS Cyber Insider Threat Discovery and Analysis Future Research and Development Discussion 15

16 Cyber Insider Threat Discovery and Analysis H 51 : Covert reconnaissance, collection, and exfiltration H 41 : Non-account owner on IP 1 performed covered reconnaissance around T 1 H 31 : Non-account owner on IP 1 scanned network for shared files around T 1 H 21 : Network scan for shared files from IP 1, around T 1 H 11 : Sequence of denied accesses to network services for several systems from IP 1, around T 1 E: Log record of denied access to network service from IP 1 to IP 2 at time T 1. H 5p : Covert reconnaissance for remote vulnerabilities H 4n : Use for covert exfiltration using stepping stone H 3k : Account owner scan for files H 2j : Recent policy changes that affected user s access to specific services H 1i : Single isolated attempt Cyber Insider Threats are persons who operate inside an organization and use legitimate access and normal tactics to accomplish abnormal and malicious missions, such as, data reconnaissance, collection and exfiltration, or creating vulnerabilities for attacks by outsiders. Major national security concern. Major concern for businesses that need to protect their intellectual property. Major privacy concern. Subject Matter Expert: Angelos Stavrou What insider mission might explain this observation? Abductive reasoning (P possibly Q)

17 Cyber Insider Threat Discovery and Analysis H 41 : P 1, non-account owner on IP 1 performed reconnaissance between T 1 and T 2. H 21 : Network scan for shared files from IP 1, between T 1 and T 2 (very ). H 11 : Sequence of denied accesses to network services for several systems from IP 1, between T 1 and T 2 (certain). H 31 : Non-account owner on IP 1 scanned network for shared files between T 1 and T 2. (very ). H 4n : P n, non-account owner on IP 1 performed reconnaissance between T 1 and T 2. H 32 : Account owner on IP 1 scanned network for shared files between T 1 and T 2. (very un). E: Log record of denied access to network service from IP 1 to IP 2 at time T 1. H 22 : Recent policy changes (very un). H 12 : Single isolated attempt H: Non-account owner accessed an account on computer C 1 between T 1 and T 2. Scenario: Physical access of C 1 in conference room CR 1 A person from CR 1, accessed someone s account on computer C 1, between T 1 and T 2. Search for persons present in conference room CR 1 between T 1 and T 2. Search for persons who entered conference room CR 1 before T 2. (no possibility) Search for persons who entered CR 1 before T 2, based on door logs. Computer C 1 has IP 1 address CR 1 is a conference room Search for persons who entered CR 1 before T 2, based on scheduled meetings participants. H 31 : Non-account owner on IP 1 scanned the network for shared files, between T 1 and T 2. Search for user activity between T 1 and T 2 for the user(s) assigned to IP 1. H: almost certain Search for persons who entered CR 1 before T 2, based on outside surveillance video camera VC 1. H 21 : Network scan for shared files from IP 1, between T 1 and T 2. Search for logs of network scan for shared files, from IP 1, between T 1 and T 2. Search for Search for host machine logs with file sharing request from IP 1. H 21 : very H: very

18 Overview Computational Theory of Intelligence Analysis Hypotheses Analysis with TIACRITIS Cyber Insider Threat Discovery and Analysis Future Research and Development Discussion 18

19 TIACRITIS Development of the agent shell 1 Developer and knowledge engineer Future Work Agent teaching by expert analyst TIACRITIS 2 Knowledge base integration and optimization TIACRITIS 6 TIACRITIS TIACRITIS Expert analyst and knowledge engineer Training of analyst Analyst TIACRITIS Evidence marshaling and hypotheses generation 3 New capabilities for training Collaborating analysts P 1 1 S 1 1 Capturing analytic expertise P 1 4 S 1 Collaborative analysis P 1n 5 TIACRITIS Knowledge engineer and expert analyst After action review and learning Continuous evidence monitoring S 1n P 1 1 Analyst S 1 Question Answer Question Answer S 1 1 P 1 Question Answer Question Answer P 1 n S 1 n Search criteria Automatic report generation P 2 m S 2 m P 2 S Search criteria Agent use and non-disruptive learning Pm 2 Sm 2 Search criteria 19

20 Discussion 20

21 Acknowledgements This research was partially supported by the National Geospatial- Intelligence Agency (PM Phillip Hwang), by the Department of Defense (PMs Erin Gibbens and Benjamin Hamilton), and by George Mason University. It was also guided by the following Advisory Board: Donald Kerr (chair), Kelcy Allwein, Keith Anthony, Cindy Ayers, Sharon Hamilton, Jim Homer, Joan McIntyre, William Nolte, George Stemler, and Benjamin Wible. The views and conclusions contained in this document are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation thereon. 21