Test & Evaluation/Science & Technology Program

Transcription

1 Test & Evaluation/Science & Technology Program C4I & Software Intensive Systems Test (C4T) Test Technology Area (TTA) Big Data Analytic Technology Needs for T&E May 11, 2017 Approved for public release: distribution unlimited. Mr. Gil Torres Phone:

2 Big Data Data Varieties of Big Data data Structured Data (20%) (Sensor data, Comm, transaction data, man-machine interface I/O, Schema / RDBMS Unstructured Data (Imagery, voice, text, chat, , Docs) Semi-structured (Self describing -Label/ Value pairs, XML) Velocities of Big Data data Near real Time Situation Awareness Real time tactical and tactical edge communications Time Critical interactions Low latency Distributed Networks Volumes of T&E Big Data data Theater data stream 270 TB of NTM data/year Future Sensor data Petabytes to Yottabytes

3 C4I & Software Intensive Systems Test (C4T) Test Technology Area (TTA) Innovative approaches to how we fight, posture our force, and leverage our asymmetric strengths and technological advantages* Complex Warfare Environments C4T Overview Battle increasingly sophisticated adversaries in increasingly complex environments* BIG DATA & CLOUD COMPUTING * 2014 Quadrennial Defense Review Test Automation (TA) Advance Big Data Collection, Analysis & Visualization Improve Test Execution thru Automation/Cloud Computing Advance Testing for Next Generation of Handhelds Improve Automated Control of Targets Distributed Testing (DT) Remove Test Infrastructure Biases Cross Domain Solutions (CDS) and Multi-Level Security (MLS) Assess Big Data Warfighter Systems Testing Warfighter Systems Employing Agile Communications Emulate Contest/Dense Communications Environments Innovate T&E: Joint, Early, Often, and Agile Modeling & Simulation (M&S) Required Fidelity in Live and Simulated Environments V&V Techniques Aggregation Techniques Run-time Performance for RT Applications Systems, Communications, Environmental Representations

4 C4T TTA Projects Supporting Technology Needs for Live Virtual & Constructive T&E

5 Summary of C4T TTA Projects Development Timeline Project Initiation Technology Demonstration Technology Assessment Technology Transition Concept Formulation (BAA Process) Concept Exploration Engineering, Integration, & Experimentation Technology Characterization & Transition Transitioned (Tracked) C4T Test Automation Distributed Testing Modeling & Simulation IPT3N LVCT RAID F-35 WAF BB RAID PRISM FAST GRaB (AE) TNTI FAST GRaB Domain Project Title Project Acronym Performer Test Automation Real-time Automated Insight Engine for Data to Decision RAID F-35 Jet Propulsion Laboratory Test Automation Real-time Automated Insight Engine for Data to Decision Patuxent River Infrared Signature Measurements RAID PRISM Jet Propulsion Laboratory Test Automation Integrated Planning of Tactical, Test Support, & Tactical Engagement Networks IPT3N Scalable Network Technologies Modeling & Simulation FAST Gaussian Ray Bundle FAST GRaB NUWC Newport Modeling & Simulation Weapons Analysis Facility: Broadband Capability WAF BB NUWC Distributed Testing Technologies for Net-centric Test Interoperability TNTI Leidos Corporation Distributed Testing Technologies for Interoperability of Live Platforms with Virtual & Constructive Simulation for Distributed Test LVCT Lockheed Martin Rotary and Mission Systems

6 Modeling & Simulation Technology Use Case M&S for T&E of Systems in Complex Environments (Representations and V&V) Sensor Platform/ Aircraft Movement Simulation Update Rate (200 msec) Use of Simulated M&S Test in T&E Environment will of increase Battlefield Robustness Big Datasets Rocket Fly Out Simulation Update Rate (100 msec) Non-Continuous Terrain Generator 100 Meters Per Grid Tile Rocket Sensor Guidance Software System Under Test (SUT) Ground Target/Vehicle Movement Simulation Update Rate (1 sec) Non-Continuous Sensor Update Rate (50 msec); Guidance Update Rate (100 msec); Accuracy 6m (CEP) Mature technologies that increase the scope of modeling & simulation across the Department of Defense (DoD) distributed T&E infrastructure.

7 Test Automation Technology Use Case Software Intensive Systems Testing in a Cloud Environment Automated Testing in a Cloud Environment Executing 1000 s of Test Case Scenarios in Parallel Virtualized HW Aspects of System(s) Under Test Software Intensive Systems (SIS) Typically Operate Within a System-of-System (SoS) to Assess Effectiveness Test Harness Technologies Support automated/agile testing (individual system) Minimize labor-intensive test procedures Support of regression, developmental, operational, and interoperability testing Automated Testing Creates Big Datasets with Big Data Analytic Needs Virtual Test Harness Virtual Test Harness System Under Test Virtual Test Harness Virtual Test Harness Automated Tools For Testing System Effectiveness & Performance New virtualization and cloud-based test technologies are needed to automatically create, execute, and control a large number of test cases operating in parallel creating Big datasets.

8 Test Automation Technology Use Case NextGen Handhelds and Widgets Big Data Analytics to Improve Speed to Fleet for Mobile Applications & Technologies On-Demand Test Services for Speed to Fleet for Mobile Applications and Technologies

9 Test Automation Domain Technology Use Case T&E Big Data Rapid Analysis VIDEO CHAT UNSTRUCTURED DATA Use Smart Algorithms & Database Design Technologies to Structure Data TEXT STRUCTURED DATA VOICE SCREENS TACTICAL TSPI C2 Operations/Analysts Advanced Semantic Algorithms SEARCH: TARGET LOCATION Parallel Search Engine Technology Test Conductor Improve the test communities ability to analyze and make decisions from large volumes of structured and unstructured test data.

10 T&E Big Data Rapid Analysis Technology Needs (1) Software algorithms that rapidly process large video datasets captured in multiple file format with multiple focal lengths (e.g. metric zoom lens) and scale factors to correlate into precise angular measurements. Algorithms that rapidly process video captured with multiple focal lengths and scale factors into precise angular (e.g. TSPI) measurements; Pattern Recognition for T&E Data; Advanced Multi-Variate Time Series Analytic Techniques; Integrating Simulation results with collected data to better understand systemunder-test performance; Advanced Big Data Visualization Techniques Tailored to the T&E Domain; Low Latency Federated Query on Disparate Big Data Sets; Artificial Intelligence Techniques applied to T&E data analysis; Rapid Data Ingest into the big data analysis system;

11 T&E Big Data Rapid Analysis Technology Needs (2) Sample big data representative repositories to evaluate C4T warfighter platforms with reusable and customizable data capture, T&E algorithms, data warehouse and analytics for Big Data warfighting platform analytics ; DoD T&E data types fusion, federation & integration technologies; Near real time analytics for streaming DoD data types; Hybrid cloud interfaces for efficient Community of Interest (COI) shared Big Data infrastructure and tools

12 Distributed Testing Technology Use Case Assess Big Data Warfighter Systems Test Control Test Control & Analysis Distributed Common Ground System (DCGS) - C4I cross Service System employing Data-to- Decisions Data generators and simulators for Velocity, Variety, and Volume Platform analytics & fusion assessments Big Data environment replication Effectiveness and Performance of Individual system within SoS Automate regression testing Joint Mission Environment Distributed Test Infrastructure T&E Instrumentation Live Warfighter Systems Virtual and Constructive Systems/Environments Improve ability to create and assess big data warfighter systems under emulated wartime modes of operation

13 Assess Big Data Warfighter Systems Technology Needs (1) Data generators and simulations to stimulate platform inputs with representative velocity, volume, and data type variety requirements including error inputs, and verification and validation of generated datasets. Big Data Analytics to evaluate Warfighter Systems; various data types and sources storage, federation, transformations and integration; Platform analytics and responses to inputs; Evaluating platforms Big Data use and share, Effectiveness/performance evaluations Analysis of Big Data System fusion algorithms Analytics to evaluate the correctness of the Big Data platform analytics Big Data T&E infrastructure monitoring, visualization and control for Big Data Platform distributed event analytics for real time analysis of streaming input data types including ones that can be located near the source or before storage for batch processing

14 Assess Big Data Warfighter Systems Technology Needs (2) Analytics for Big data birth to deployment artifacts evaluation Analytics for Big Data platform analysis T&E data history usage Analytics for Big Data platform deployment monitoring Automate regression testing for Big Data Platform changes to its big data capture or analytics Big Data IT infrastructures: Cloud and nosql data bases Analytics to drill down to individual system performance and contribution, and then determine the impact to the effectiveness of the SoS performance

15 C4T Test Automation Project Real-time Automated Insight Engine for Data to Decision (RAID) F-35 Jet Propulsion Laboratory / Pasadena, CA FY17 Accomplishments: Improved RAID human-like Artificial Intelligence (AI) software to reuse tester knowledge for data analytics in realistic environment with high degrees of uncertainty. Integrate reasoner with teleo-reactive prog. Enhanced Associative Symbolic Memory (ASSUME) software to support probabilistic inferencing. Demonstrated using RAID to accomplish target identification with Edwards AFB DIADS data. Deliverables (Mo/Yr): Prototype RAID-DIADS software to Edwards AFB. RAID Phase 2 Final Report. Description: Big data analytical technology developed to automatically analyze, extract, & manage actionable knowledge from terabytes of data per test day. Enables: Automated reuse of knowledge to enable DT quality testing throughout the lifecycle of weapon systems & provide the potential for discovery of unknown-unknowns problems. Current Status: Developing RAID knowledge management framework & proof-of-concept insight extraction demo using F-16 data. Continue to try to get F-35 engine data. Transition Partner / Date: Edwards 771st Flight Test Squadron, TRMC Knowledge Management, Edwards 412th Range Squadron / 3QFY18 Key Future Events: Enable correlation of test data with knowledge base models and ontologies. Initial demonstration of RAID capability addressing a representative F-35 T&E scenario. 15

16 C4T Test Automation Domain Project Real-time Automated Insight Engine for Data to Decision (RAID) Patuxent River Infrared Signature Measurements (PRISM) Jet Propulsion Laboratory / Pasadena, CA Description: Develops big data technologies to analyze infrared (IR) videos that utilizes Intelligent Neural Network (NN) based segmentation and 2D/3D correlation used for system identification, tracking and performance assessment. Enables: Real-time automatic segmentation of multiple band IR videos allow rapid data analysis from man-months to hours. Realtime feedback of IR signature quality during test runs. Current Status: Developing neural network based segmentation algorithm & automatic multi-band signature segmentation software. Transition Partner / Date: PAX ATR / 3QFY18 FY17 Accomplishments: Completed segmentation algorithm refinement. Completed applying neural net directly to MW. Completed Initial C++ GUI test program. Mid-Year progress review. Deliverables (Mo/Yr): Delivered incremental software builds and user s manual to PAX ATR PRISM team. Key Future Events: Complete target feature segmentation and implementation in complex neural network. Complete testing and optimization Software. Implement real-time automatic segmentation system. 16

17 C4T Test Automation Domain Decision Engine for Structured & Unstructured Data (DESU) Morgan State University / Baltimore, MD Improve Near Real-Time & Post-Test Analysis for T&E Unstructured Big Data FY15 Accomplishments: Completed 100% of Phase 3 exit requirements. Acquired and tested PAX 3-D Shearlet Algorithm. Description: Developing Big Data technologies for rapid analysis of voluminous structured & unstructured heterogeneous data (audio/video/text) in support of data to decisions for automated event pattern discovery. Enables: Assessment of large volumes of test data, real-time data fusion & analysis capability. Current Status: Developing event classifiers & algorithms to correlate within/across data sets. Transition Partner / Date: PAX ATR 4QFY15 Key Future Events: None, project successfully completed. Deliverables (Mo/Yr): Phase 2 Report & Software. 17

18 Big Data Analytic Technology Needs for T&E Closing Big Data and associated Analytics presents technological challenges for the T&E community game changing innovations are needed NOW C4T TTA Broad Area Announcement defines investment topics for Industry and Academia to propose new technology projects Primary Big Data Topics T&E Big Data Rapid Analysis Assess Big Data Warfighter Systems Big Data Topics Resulting from New Technology Innovations M&S for T&E of Systems in Complex Environments Software Intensive Systems Testing in a Cloud Environment T&E for NextGen Handhelds and Widgets

19 C4T Points of Contact Executing Agent Mr. Gil Torres Deputy Executing Agent Ms. Gail Holmes (401)