Live Video Analytics at Scale with Approximation and Delay-Tolerance

Size: px

Start display at page:

Download "Live Video Analytics at Scale with Approximation and Delay-Tolerance"

Tracy Turner
5 years ago
Views:

1 Live Video Analytics at Scale with Approximation and Delay-Tolerance Haoyu Zhang, Ganesh Ananthanarayanan, Peter Bodik, Matthai Philipose, Paramvir Bahl, Michael J. Freedman

2 Video cameras are pervasive 2

3 Video analytics queries Intelligent Traffic System AMBER Alert Electronic Toll Collection Video Doorbell 3

4 Video query: a pipeline of transforms Vision algorithms chained together Example: traffic counter pipeline transform decode transform detect object transform track object transform count object 4

5 Video queries are expensive in resource usage Best car tracker [1] 1 fps on an 8-core CPU DNN for object classification [2] 3GFlops transform decode transform b/g subtract transform track object transform count object When processing thousands of video streams in multi-tenant clusters How to reduce processing cost of a query? How to manage resources efficiently across queries? [1] VOT Challenge 215 Results. [2] Simonyan et al. CVPR abs/ , 214 5

Vision algorithms are intrinsically approximate Knobs: parameters / implementation choices for transforms Frame Rate Resolution Window Size

6 Vision algorithms are intrinsically approximate Knobs: parameters / implementation choices for transforms Frame Rate Resolution Window Size Mapping Metric License plate reader window size Car tracker mapping metric Object classifier DNN model Query configuration: a combination of knob values 6

7 Knobs impact quality and resource usage 3 72p Frame Rate Resolution Quality=.93, CPU= p Quality=.57, CPU=.9 7

8 Knobs impact quality and resource usage Quality CPU Frame Rate Frame Rate Quality Quality CPU 48p 576p 72p 9p18p Frame Resolution Resolution Quality CPU CPU DIST HIST SURF SIFT Window Size Step Object Mapping Metric Window Size Mapping Metric 8

9 Knobs impact quality and resource usage Quality of Result.4.2 License Plate Reader Resource Demand [CPU cores, log scale] Orders of magnitude cheaper resource demand for little quality drop No analytical models to predict resource-quality tradeoff Different from approximate SQL queries 9

10 Diverse quality and lag requirements Lag: time difference between frame arrival and frame processing Toll Collection Intelligent Traffic AMBER Alert Quality? High Moderate High Lag? Hours Few Seconds Few Seconds 1

11 Goal Decide configuration and resource allocation to maximize quality and minimize lag within the resource capacity Configuration. Quality Resource Allocation. Lag 11

Video analytics framework: Challenges 1. Many knobs large configuration space No known analytical models to predict quality and resource impact 2.

12 Video analytics framework: Challenges 1. Many knobs large configuration space No known analytical models to predict quality and resource impact 2. Diverse requirements on quality and lag Hard to configure and allocate resources jointly across queries Configuration. Quality Resource Allocation. Lag 12

13 VideoStorm: Solution Overview Profiler query resource-quality profile Scheduler Workers utility function offline online 13

14 VideoStorm: Solution Overview query Profiler Builds model Reduces config space resource-quality profile utility function Trades off Scheduler quality and Workers lag across queries offline online 14

15 VideoStorm: Solution Overview Profiler query resource-quality profile Scheduler Workers offline online 15

16 Offline: query profiling Profile: configuration resource, quality Ground-truth: labeled dataset or results from golden configuration Explore configuration space, compute average resource and quality higher quality Quality of Result Resource Demand [CPU cores, log scale] is strictly better than in quality and resource efficiency more efficient 16

17 Offline: Pareto boundary of configuration space Pareto boundary: optimal configurations in resource efficiency and quality Cannot further increase one without reducing the other Orders of magnitude reduction in config. search space for scheduling higher quality Quality of Result Resource Demand Pareto optimal more efficient 17

18 VideoStorm: Solution Overview Profiler resource-quality profile utility function Scheduler Workers offline online 18

Online: utility function and scheduling Utility function: encode

and tolerable lag Reward additional quality, penalize higher lag

utility (max-min) fairness Maximize the total utility overall

19 Online: utility function and scheduling Utility function: encode goals and sensitivities of quality and lag Users set required quality and tolerable lag Reward additional quality, penalize higher lag higher quality Schedule for two natural goals: Maximize the minimum utility (max-min) fairness Maximize the total utility overall performance higher lag Allow lag accumulation during resource shortage, then catch up 19

20 Online: scheduling approximate video queries Time 1 C2 C1 1 C Time C2 C1 1 Time C Time 38 1 Time Lag =1.5 C = Lag Resource (cores) 1 Quality Total CPU: Fair scheduler: best configurations w/o lag Quality-aware scheduler: allow lag catch up 2 Quality-aware Quality CPU (core). Resource (cores) 3 3 Lag 2 4 Lag 1 Quality Fair Quality Quality Queries: blue and orange (tolerate 8s lag) C3 C3 1 C2 22 C Time 1

21 Additional Enhancements Handle incorrect resource profiles Profiled resource demand might not correspond to actual queries Robust to errors in query profiles Query placement and migration Better utilization, load balancing and lag spreading Hierarchical scheduling Cluster and machine level scheduling Better efficiency and scalability 21

22 VideoStorm Evaluation Setup Platform: Microsoft Azure cluster Each worker contains 4 cores of the 2.4GHz Intel Xeon processor and 14GB RAM Four types of vision queries: license plate reader car counter DNN classifier object tracker VideoStorm Manager Profiler + Scheduler 1 Worker Machines 22

23 Experiment Video Datasets Operational traffic cameras in Bellevue and Seattle 14 3 frames per second, 24P 18P resolution 23

24 Resource allocation during burst of queries Start with 3 queries: Lag Goal=3s, low-quality ~6% Lag Goal=2s, low-quality ~4% Burst of 15 seconds (5 2): 2 LPR queries (AMBER Alert) High-Quality, Lag Goal=2s VideoStorm scheduler: dominate resource allocation significantly delay run with lower quality All meet quality and lag goals Share of Cluster CPUs Quality Lag (sec) Lag Goal=3s Lag Goal=2s High-Quality, Lag Goal=2s Time (seconds) Lag Goal=3s Lag Goal=2s High-Quality, Lag Goal=2s Time (seconds) 24

25 Resource allocation during burst of queries Start with 3 queries: Lag Goal=3s, low-quality ~6% Lag Goal=2s, low-quality ~4% Compare Burst of 15 to a seconds fair scheduler (5 with 2): varying burst duration: Quality 2 LPR improvement: queries (AMBER up to 8% Alert) High-Quality, Lag reduction: Lag up Goal=2s to 7x VideoStorm scheduler: significantly delay run with lower quality dominate resource allocation All meet quality and lag goals Share of Cluster CPUs Quality Lag (sec) Lag Goal=3s Lag Goal=2s High-Quality, Lag Goal=2s Time (seconds) Lag Goal=3s Lag Goal=2s High-Quality, Lag Goal=2s Time (seconds) 25

26 VideoStorm Scalability Frequently reschedule and reconfigure in reaction to changes of queries Even with thousands of queries, VideoStorm makes rescheduling decisions in just a few seconds Scheduling Time (s) Number of Machines Number of Queries 26

27 VideoStorm: account for errors in query profiles Errors in profile on resource demands Over/under allocate resources miss quality and lag goals! Example: 3 copies of same query, should get same allocation Profiled resource synthetically doubled, halved and unchanged VideoStorm keeps track of mis-estimation factor # multiplicative error between the profiled demand and actual usage Time (seconds) 4 5 Twice Half ¹ CPU CPU (w/ adaptation) (w/ adaptation) CPU (w/o adaptation) Accurate Time (seconds) Time (seconds) Time (seconds) 27

28 Conclusion VideoStorm is a video analytics system that scales to processing thousands of video streams in large clusters quality, F1 score resource demand [CPU cores, log scale] Share of Cluster CPUs Lag Goal=3s Lag Goal=2s High-Quality, Lag Goal=2s Time (seconds) Offline profiler: efficiently estimates resource-quality profiles Online scheduler: optimizes jointly for the quality and lag of queries VideoStorm is currently deployed in Bellevue Traffic Department, and soon will be deployed in more cities 28

edge computing a historical perspective & direction 10 years & counting

edge computing a historical perspective & direction 10 years & counting Victor Bahl Distinguished Scientist Director, Mobility & Networking Research Microsoft Resea Monday, August 20, 2018 Microsoft s