Real-Time Scene Understanding

Transcription

1 Real-Time Scene Understanding Dynamic Data Driven Applications Systems Dr. Frederica Darema Dr. Alex Aved Research Computer Scientist Analytical Systems Branch 1

2 Problem Statement Focus full motion video exploitation to support decision-making; event specification, detection, situational awareness. Problem we collect more data than we can process data is complex, fragmented; increasing in quantity & complexity at-a-glance processing, exploitation and dissemination infeasible without software. Goal break the linear relationship (staffing) between collection and analysis, Leverage analysts for synthesis and insight; computers for calculations and memory. Simply stated, the need for accurate intelligence and prescient analysis [ ] has never been greater in 2013 or than it will be in the coming years John Brennan, Director, Central Intelligence Agency 14 February 2016 Many sensors Need to quickly leverage data for decisions Integrated information architecture 2

3 Approaches Business users Identify questions SP Platform SP Builds database structure Business users Explore correlations Structured and repeatable Questions (queries) drive insight Data at rest Relational data model, warehouses Note Talent matters as much as technology SP Solution Providers; researchers etc. Iterative and exploratory Insight drives answers Data in motion Hadoop (MapReduce), LVDBMS 3

4 Background Network-connected video cameras leveraged for monitoring (bridges, malls, parks ). Currently monitored by humans who can become fatigued, get interrupted, attention lapses, work in shifts The Live Video Database Management System [1,2] (LVDBMS) prototype will be leveraged to develop and apply machine learning and fusion algorithms, and associated data structures and communication protocols. [3] Academia Conventional Solution Example application Traffic monitoring HP: Generates events. Events can be analyzed. Face detection Recognize models of automobiles Integration with Cyber (via event model) IBM: 03.ibm.com/software/products/en/intelligentvideo analytics CBIR Alerting (tripwires; boxes) Size correlation (3d scene reconstruction) Industry [1] A. J. Aved, K. A. Hua, and V. Gurappa, An Informatics Based Approach to Object Tracking for Distributed Live Video Computing, in Multimedia Communications, Services and Security, 2011, pp [2] A. J. Aved and K. A. Hua, A general framework for managing and processing live video data with privacy protection, Multimedia Systems., vol. 18, no. 2, pp , Feb [3] Anwar, Fahad, et al. "An Efficient Event Definition Framework for Retail Sector Surveillance Systems." MMEDIA 2014, The Sixth International Conferences on Advances in Multimedia February

5 Domain-Specific Systems Application specific programming A 1 A 2 A 3 FP 1 Sharing similar underlying software FP 2 FP 3 A specific set of known cameras A: Application FP: Fusion Processor 5

6 The LVC Approach 6

7 Query Language LVQL ACTION <action> ON EVENT <composite event> <window> Query Optimizer Mathematical example 7

8 Query Language LVQL ActionEvent := [action UserSpecifiedAction] on EventSpecification appear north northwest inside meet... before meets and or not A West B A Contains B A A B B A Before B t A B 8

9 Event Model Spatial Event Spatial event stream Stream of spatial events on two objects E s (t 1 ) = T E s (t 2 ) = T E s (t 3 ) = T E s (t 1 ) = Disjoint(O i (t 1 ), O j (t 1 )) E s (t 4 ) = T E s (t 5 ) = T E s (t 6 ) = T E s = {T, T, T, T, T, T, F, F, F} o t E s (t 7 ) = F E s (t 8 ) = F E s (t 9 ) = F 9

10 Query Flow Results A query is decomposed into Three sub queries Query Query Translation Stream Processing Server Subquery results Subqueries Camera Server Camera Server... Camera Server 10

11 Video Analytics The LVDBMS is a scalable, distributed video database management system designed for processing continuous queries over live video streams in near real time. Video analytics Object matching Video characterization Multi camera Creating an effective analytical environment can substantially improve operational visibility for public sector organizations and help them execute on their mission of cost [improvement] February Aberdeen 2016 Group, BI in the Public Sector: Enhanced Efficiency with Data Discovery Data flow 11

12 Challenges and Opportunities Increasing value and reducing costs Growing demand for improved end user experience, quality and functionality Vision (of solution) and analytics/metrics to measure Expands research and business/operations communication, collaboration and representation Improved service and value to stakeholders Measurement, results and analytics Creating an environment of analytical engagement and assessment Thank You.Questions.? Creating an effective analytical environment can substantially improve operational visibility for public sector organizations and help them execute on their mission of cost [improvement] Aberdeen Group, BI in the Public Sector: Enhanced Efficiency with Data Discovery 12

13 Planned Future Research Pattern Search, Fusion and Prediction POL for Anomaly Discovery and Prediction Historical Pattern Search Federated LVC PaaS Cloud Architecture Policy-based Information Dissemination Extend Fusion to Remote Platforms 14 February

14 14 February