Big Data Europe and Data Platforms for Tests. Thessaloniki Dr. Josep Maria Salanova Grau Center for Research and Technology Hellas - Hellenic Institute of Transport Head of Data collection and processing, algorithm design, and use of specialized transport software packages laboratory Email: jose@certh.gr Web: www.hit.certh.gr
Big Data Europe project
Big Data Europe Empowering Communities with Data Technologies Enable European companies to build innovative multilingual products and services based on semantically interoperable, largescale, multi-lingual data assets and knowledge, available under a variety of licenses and business models. Big Data Europe aims to: Collect requirements for the ICT infrastructure needed by dataintensive science practitioners tackling a wide range of societal challenges; covering all aspects of publishing and consuming semantically interoperable, large-scale, multi-lingual data assets and knowledge. Design and implement an architecture for an infrastructure that meets requirements, minimizes the disruption to current workflows, and maximizes the opportunities to take advantage of the latest European RTD developments, including multilingual data harvesting, data analytics, and data visualization.
Big Data Europe Empowering Communities with Data Technologies Health Heterogeneous data linking and integration, biomedical semantic indexing Food & Agriculture Large-scale distributed data integration Security Real-time monitoring, stream processing and data analytics, image data analysis Energy Real-time monitoring, stream processing, data analytics, decision support Transport Streaming sensor network and geospatial data integration Climate Real-time monitoring, stream processing and data analytics Social Sciences Statistical and research data linking and integration
Big Data Europe Empowering Communities with Data Technologies Probe data that is used Floating Car Data (500-2.500 locations per minute) Bluetooth detections (millions of daily detections in 43 locations) Services that are being implemented Improved topology-based map matching Mobility patterns recognition and forecasting Bluetooth Data GPS Data Map Matching Traffic Classification and Prediction Results Classification and Prediction Data
Big Data Europe Empowering Communities with Data Technologies Map Matching Algorithm
Big Data Europe Empowering Communities with Data Technologies Map Matching Algorithm rownum recorded_times tamp transfer osmids 263005 08:43:18 1 213910068 263171 08:43:27 1 16457977 263350 08:43:37 1 213910068 263657 08:43:55 1 213910068 263919 08:44:10 1 22564731 264174 08:44:25 1 222137984 264701 08:44:56 1 222137984 264813 08:45:04 1 222137984 265071 08:45:19 0 22564731 265363 08:45:35 0 213910068 265565 08:45:48 0 213910068 265737 08:45:57 0 16457977 265906 08:46:07 0 213910068
Big Data Europe Empowering Communities with Data Technologies Start Historical Link Traffic State BT Historical Link Traffic State (FCD) Historical Link Traffic State (Loop Detectors) Historical Link Traffic State Classification Store in Historical States Define Current Link Traffic State Compare Traffic States (ML, NN) Predict (ARIMAX NN) Traffic Classification and Prediction Validate Prediction
Thessaloniki, Data Platform for Tests
Thessaloniki on the map ~1.400.000 inhabitants ~450.000 private cars & 20.000 motorcycles 1.300.000 daily trips 1 (+1) public transport operator for urban trips ~35 public transport operators for extra-urban trips 1.950 taxis 6.433 kms of streets 8,8 kms of dedicated bus lanes 12 kms of dedicated bicycle lanes 89,4 kms of ring road 197.696 parking places
Existing ITS layout Peripheral Urban Central
Thessaloniki Mobility lab
Mobile sensors in Thessaloniki Stationary sensors network: Point to point tracking of MAC ids along the network through 43 Bluetooth device detectors. Travel time estimation Route choice model calibration Origin Destination matrix estimation / Mobility patterns estimation Traffic flow extrapolation Dynamic sensors fleet: Floating Car Data provided in real time by a professional fleets composed of 1.200 taxis and 600 buses Traffic status estimation (average speed) Origin Destination matrix estimation / Mobility patterns estimation Taxi/bus performance indicators Social media (geolocated tweets & Facebook check-in events) Activity patterns estimation Events / incidents detection Attraction models estimation
Mobile sensors in Thessaloniki http://160.40.63.110/desktop/
Dynamic Sensors Data
Floating Car Data (taxis) Device ID GPS position (X, Y, Z) Orientation (degrees) Speed (km/h) Timestamp Zone Status
Floating Car Data (taxis) 1.200 vehicles (dispatching application) Circulating 16-24 hours per day Pulse generated each 100 meters (10-12 seconds) 500-2.500 pulses per minute
Mobile sensors in Thessaloniki www.trafficthess.imet.gr/
Stationary Sensors Data
Bluetooth devices detectors 43 detectors (EEA, SEE-ITS & EASYTRIP) 4 million detections per week (peak period) 25.000 unique devices detected per day (one intersection) 1 million tracked trips per week 20.000 tracked trips per day (one path) More detectors installed in other cities and in Bulgaria (SEE-ITS & EASYTRIP)
Point to point BT network Real time travel time provision to drivers (VMS, internet, smart device)
Mobile sensors in Thessaloniki www.trafficpaths.imet.gr
Mobile sensors in Thessaloniki http://opendata.imet.gr/ Datatank (Back office + APIs) CKAN (front end)
Social media
Social media 44.000 check-in events per week (750 locations) Up to 50 check-in events per minute (in the 136 locations tagged as bar) 17 check-in events per minute (in the 150 locations tagged as restaurant) 12 check-in events per minute (in the 32 locations tagged as outdoor) 10 check-in events per minute (in the 125 locations tagged as cafe) 10 check-in events per minute (in the 55 locations tagged as nightlife) Up to 1265 check-in events during the peak hour 920 check-in events in bars (Sunday 01.00) 300 check-in events in restaurants (Saturday 22.00)
Social media BAR 60 50 40 30 20 10 0 22/02/2016-10 00:00 23/02/2016 00:00 24/02/2016 00:00 25/02/2016 00:00 26/02/2016 00:00 27/02/2016 00:00 28/02/2016 00:00 29/02/2016 00:00 01/03/2016 00:00 02/03/2016 00:00 15 CAFE 10 5 0 22/02/2016 00:00 23/02/2016 00:00 24/02/2016 00:00 25/02/2016 00:00 26/02/2016 00:00 27/02/2016 00:00 28/02/2016 00:00 29/02/2016 00:00 01/03/2016 00:00 02/03/2016 00:00 12 NIGHTLIFE 10 8 6 4 2 0 22/02/2016 00:00 23/02/2016 00:00 24/02/2016 00:00 25/02/2016 00:00 26/02/2016 00:00 27/02/2016 00:00 28/02/2016 00:00 29/02/2016 00:00 01/03/2016 00:00 02/03/2016 00:00
Social media
Social media
Thank you! Dr. Josep Maria Salanova Grau jose@certh.gr +30 2310 498 433