SAFESPOT INTEGRATED PROJECT - IST IP DELIVERABLE ANNEX. SP5 CoSSIB Cooperative Safety Systems Infrastructure Based

Transcription

1 SAFESPOT INTEGRATED PROJECT - IST IP DELIVERABLE ANNEX SP5 CoSSIB Cooperative Safety Systems Infrastructure Based Test and validation results - ANNEX Deliverable No. (use the number indicated on technical annex) D ANNEX SubProject No. SP5 SubProject Title CoSSIB Workpackage No. WP5.5 Workpackage Title Test and validation Task No. T5.5.2 Task Title Test and validation results Authors (per company, if more than one company provide it together) Status (F: final; D: draft; RD: revised draft): P. J. Feenstra TNO (coordinator); D. Vilain, Cofiroute; N. Etienne, Sodit; S. Glaser, LCPC; A. Possani and D. Anguita, DIBE; T. Schendzielorz, TUM; F. Visintainer, CRF; C. Torres, MIZAR F Version No: 1.4 File Name: SF_D5.5.2_Test and validation results ANNEX v1.4.doc Planned Date of submission according to TA: May 2009 Issue Date: June 2010 Project start date and duration 01 February 2006, 48 Months SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 1 of 113 COSSIB

2 Revision Log Version Date Reason Name and Company V0.5 17/05/10 Splitting of the deliverable in a main report and this annex P.J. Feenstra, TNO V0.5b 21/05/10 Textual updates P.J. Feenstra, TNO V0.6 23/06/10 Including peer review P.J. Feenstra, TNO V1.0 25/06/10 Final version P.J. Feenstra, TNO V1.1 26/06/10 Peer reading and approval for submission to EC G. Vivo, CRF P.J. Feenstra, TNO V1.2 30/06/10 Further final improvements P.J. Feenstra, TNO A. Spence, MIZAR V1.3 06/07/10 Minor corrections F. Visintainer, CRF V1.4 15/09/10 Reply to reviewers comments A.R.A. van der Horst, TNO SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 2 of 113 COSSIB

3 Abbreviation List AEV AMR API AV BSCW CoSSIB CRF DFL DFS ECAID ESPOSYTOR EV GD GPS H&IW HMI I2V ICT IRIS LDM LDP LED LIVIC MARS PC PND PRM PTV-AG Pyro RDep RFID RQ RSU SCOVA SDM SiVIC SMA SMAEV SOAP SP SpA TBC TBD TLC TNO UC UDP VANET VMS VTT WP WSN Assistance and Emergency Vehicles Anisotropic Magneto-Resistive Application Programming Interface Assistance Vehicle Basic Support for Cooperative Work Cooperative Safety Systems Infrastructure Based Centro Ricerche FIAT Data Fusion Logic Data Fusion Sensor Simulation Enhanced Cooperative Automatic Incident Detection SAFESPOT SYSTEM MONITOR Emergency Vehicle Ghost Driver Global Positioning System Hazard and Incident Warning Human Machine Interface Infrastructure-to-Vehicle Information Communication Technology Intelligent Cooperative Intersection Safety - System Local Dynamic Map LDM Data Player Light Emitting Diode Laboratoire sur les Interactions Véhicules-Infrastructure-Conducteurs Multi-Agent Real-Time Simulator Personal Computer Portable Navigation Device Priority Request Manager Planung Transport Verkehr AG Pyroelectric Road Departure Prevention Radio Frequency Identification Requirement Road Side Unit Cooperative systems applications vehicle based Safe Drive Map Simulateur Véhicule Infrastructure Capteurs Safety Margin Assistant Safety Margin for Assistance and Emergency Vehicles Simple Object Access Protocol Sub-project Speed Alert To Be Confirmed To Be Determined Traffic Light Controller Nederlandse Organisatie Voor Toegepast Natuurwetenschappelijk Onderzoek Use Case User Datagram Protocol (communication protocol) Vehicle Ad hoc Network Variable Message Sign Valtion Teknillinen Tutkimuskeskus Work-package Wireless Sensor Network SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 3 of 113 COSSIB

4 Table of contents Revision Log... 2 Abbreviation List... 3 Table of contents... 4 List of Figures... 5 List of Tables... 5 Appendix A - Test forms: Detailed test results per application... 6 Appendix B - Information about the H&IW test results SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 4 of 113 COSSIB

5 List of Figures Figure 1 Diagram of the Amsterdam Test Track Figure 2 Logical architecture of SMAEV application Figure 3 Pictures of the Obstacle (traffic jam & slow vehicle) Figure 4 Technical Test Result Amsterdam wrong way driver (Infrastructure analysis) List of Tables Table 1 IRIS LDM Static Content Test Simulation... 6 Table 2 IRIS Functionality Test Simulation... 8 Table 3 IRIS Functionality Test Laboratory Table 4 IRIS Functionality Test Test site Table 5 SpA_01 Simulation Table 6 SpA_01 Test site...21 Table 7 SpA_03 Simulation Table 8 SpA_03 Test site...30 Table 9 H&IW_01 and H&IW_02 Simulation Table 10 H&IW_01 Test site Table 11 H&IW_01 Test site Table 12 H&IW_02 Test Site Table 13 H&IW_02 H&IW_03 Test Site Table 14 Rdep SDM creator Simulation Table 15 Rdep Runtime Simulation Table 16 Rdep SDM creator Test site Table 17 Rdep Runtime Test site Table 18 SMAEV_01 Functional test of components Bench (CRF) Table 19 SMAEV_01 Test site (CRF) Table 20 SMAEV_01 Test site (SODIT) Table 21 SMAEV_02 Test site Table 22 Verification Table RFID Ghost driver two cars in opposite direction Table 23 Verification Table RFID Shadowing 2 Ghost Drivers Table 24 Obstacle-Pedestrian on the CRF Test Track Thermal imaging cameras Table 25 Results Wrong Way Driver CRF Correctness Table 26 Wrong Way Driver CRF Communication Test SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 5 of 113 COSSIB

6 Appendix A - Test forms: Detailed test results per application. Table 1 IRIS LDM Static Content Test Simulation TEST FORM SP 5 WP 5 Prototype Version - System Configuration - HW Release - SW Release - DFL_v1.0.8 (incl. IRIS) - PG LDM using the C++ interface R_1_7_10 and SQLite LDM (LDM data model version ) Compiled by / Company TUM Date July 2009 to March 2010 Form Progressive Numb. 1 Functional Component IRIS Reference Document D5.3.3 Test Type Test Objective Test Purpose Test Environment Unitary Verification Correctness Simulation Test Case 1: IRIS LDM Static Content Test Simulation Test Description Prerequisite : - Installed LDM software and the static supply for the intersection - Installed DFL incl. IRIS Description: - Starting DFL incl. IRIS basic LDM checks are performed automatically and a report on the terminal is issued Requirements: - SP5_RQ41_19_v1.0: The LDM shall describe the static geometry of critical points (e.g. intersections) in a detailed, accurate and systematic way. The geometry shall comprise at least approaches, exits, lanes (also bicycle lanes), stop lines and pedestrian crossings as well as the topology of the lanes (left-turn, right-turn, straight ahead). - SP5_RQ42_19_v1.0: The LDM shall provide a unique scheme for dynamic traffic information to refer to. Use cases: - SP5_UC22: Safe signalized intersection (crossing, turning) - SP5_UC31: Safe signalized intersection (red light violation) Expected Values / Results SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 6 of 113 COSSIB

7 - The listed requirements are met, specifically for the LDM. Obtained Values / Results SP5_RQ41_19_v1.0 The LDM shall describe the static geometry of critical points (e.g. intersections) in a detailed, accurate and systematic way. The geometry shall comprise at least approaches, exits, lanes (also bicycle lanes), stop lines and pedestrian crossings as well as the topology of the lanes (left-turn, right-turn, straight ahead). SP5_RQ42_19_v1.0 The LDM shall provide a unique scheme for dynamic traffic information to refer to Status Test Case 1 Link to external annexed documentation (if any) - More iterations were needed to prepare the content for all the three test sites (Helmond, Dortmund, Amsterdam) SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 7 of 113 COSSIB

8 Table 2 IRIS Functionality Test Simulation TEST FORM SP 5 WP 5 Prototype Version - System Configuration - HW Release - SW Release - DFL_v1.0.8 (incl. IRIS) - PG LDM using the C++ interface R_1_7_10 and SQLite LDM (LDM data model version ) Compiled by / Company TUM Date July 2009 to March 2010 Form Progressive Numb. 1b Functional Component IRIS Reference Document D5.3.3 Test Type Test Objective Test Purpose Test Environment Unitary Verification Correctness Simulation Test Case 2: IRIS Functionality Test Simulation Test Description Prerequisite : - The LDM contains the detailed static description of the intersection such as allocations of stop lines, the reference tracks, signal groups and the assignment of the signal groups of the appropriate lanes. In addition, the data concerning moving objects need to be provided by the SP2 data platform. - Furthermore, the intersection needs to be modelled in the simulation environment including all vehicles, cyclists and movements. The traffic is generated randomly. - The simulated data need to be fed into the LDM. Description: - Based on the current situation, the IRIS estimates the future trajectories of the vehicles at the intersection and analysis the emerging situation. If the situation is going to be safety critical, a warning is generated. This basic function is needed for all use cases of the IRIS Application (see list below) Tools: VISSIM (micro simulation tool) Requirements: - SP5_RQ01_36_v1.0: The system shall be able to calculate the trajectories of all vehicles approaching and passing critical points e.g. urban intersections. - SP5_RQ03_36_v1.0: The system shall be able to calculate the trajectories of all cyclists approaching and passing the intersection. - SP5_RQ04_36_v1.0: The system shall be able to predict the vehicle s trajectories. - SP5_RQ08_19_v1.0: The system shall take into account the actual and short-term forecast of the traffic light control status. - SP5_RQ18_19_v1.0: The system shall be able to manage all vehicles in the vicinity of a critical point (e.g. the intersection) simultaneously. - SP5_RQ19_36_v1.0: The system shall have nearly the same performance in case there are just two or 80 vehicles to manage. - SP5_RQ79_6_v1.1: The intended route on the intersection shall be known either in some direct manner, or indirectly through the status of vehicles indicator. SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 8 of 113 COSSIB

9 Use cases: - SP5_UC22: Safe signalized intersection (crossing, turning) - SP5_UC31: Safe signalized intersection (red light violation) Expected Values / Results - The listed requirements are met. - The situaiton is interpreted correctly and the right warning is triggerd. Obtained Values / Results SP5_RQ01_36_v1.0 The system shall be able to calculate the trajectories of all vehicles approaching and passing critical points e.g. urban intersections. SP5_RQ03_36_v1.0 The system shall be able to calculate the trajectories of all cyclists approaching and passing the intersection. SP5_RQ04_36_v1.0 The system shall be able to predict the vehicle s trajectories. SP5_RQ08_19_v1.0 The system shall take into account the actual and short-term forecast of the traffic light control status. not tested SP5_RQ18_19_v1.0 The system shall be able to manage all vehicles in the vicinity of a critical point (e.g. the intersection) simultaneously. not tested SP5_RQ19_36_v1.0 The system shall have nearly the same performance in case there are just two or 80 vehicles to manage. not tested SP5_RQ79_6_v1.1 The intended route on the intersection shall be known either in some direct manner, or indirectly through the status of vehicles indicator. Status Test Case 1 Link to external annexed documentation (if any) - More iterations were needed to prepare the content for all the three test sites (Helmond, Dortmund, Amsterdam) This requirement was not tested hence at the real intersection only the current state of the traffic light was available. The test scenarios only comprised the minimum number of vehicle which needed to be involved to run the test. This number could be managed without any loss in the performance. The vehicles send the status of the indicator. IRIS can interpret this information. SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 9 of 113 COSSIB

10 Table 3 IRIS Functionality Test Laboratory TEST FORM SP 5 WP 5 HW Release - ebox - RSU antenna - vehicle antenna - HMI display Form Progressive Numb. SW Release 1c Test Type - DFL_v1.0.8 (incl. IRIS) - SP5 Message Manger_v PG LDM using the C++ interface R_1_7_10 and SQLite LDM (LDM data model version ) - SNT router Translator (CVIS release 7) - CALM manager (CVIS release 7) Prototype Version Compiled by / Company - System Configuration - TUM Date July 2009 to August 2009 Functional Component IRIS Reference Document D5.3.3 Test Objective Test Purpose Test Environment Integration Verification Correctness Bench Test Case 3: IRIS Functionality Test Laboratory Test Description Prerequisite : - The LDM contains the detailed static description of the intersection such as allocations of stop lines, the reference tracks, signal groups and the assignment of the signal groups of the appropriate lanes. In addition, the data concerning moving objects need to be provided by the SP2 data platform. - Furthermore, the intersection needs to be modelled in the simulation environment including all vehicles, cyclists and movements. - Complete RSU and vehicle software components need to be set up at the test bench Description: - First run to test the proper configuration of the HMIMessage by the use of the VANET player - Second run to test the whole data chain based on simulated data by the micro simulator. Tools: - VANETplayer - VISSIM (micro simulation tool) SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 10 of 113 COSSIB

11 Requirements: - SP5_RQ06_19_v1.0: The system shall be able to provide a priority level for each transmitted message. This is valid for V2I as well as for I2V communication - SP5_RQ07_19_v1.0: The system shall be able to determine the period of validity of the transmitted and received messages. - SP5_RQ40_19_v1.0: Messages that are exchanged between vehicles and RSU have to be unified to insure system interoperability. - SP5_RQ50_36_v1.0: The system shall be able to transmit the warning / recommendation to equipped vehicles. - SP5_RQ86_6_v1.1: It shall be possible to give warnings to drivers. Preferably with a graphical identification of the type and location of potential conflicts. Use cases: - SP5_UC22: Safe signalized intersection (crossing, turning) - SP5_UC31: Safe signalized intersection (red light violation) Expected Values / Results - The listed requirements are met. - IRIS application is able to transmitt the correct warning to the vehicle system. Obtained Values / Results SP5_RQ06_19_v1.0 The system shall be able to provide a priority level for each transmitted message. This is valid for V2I as well as for I2V communication SP5_RQ07_19_v1.0 The system shall be able to determine the period of validity of the transmitted and received messages SP5_RQ40_19_v1.0 Messages that are exchanged between vehicles and RSU have to be unified to insure system interoperability SP5_RQ50_36_v1.0 The system shall be able to transmit the warning / recommendation to equipped vehicles SP5_RQ86_6_v1.1 It shall be possible to give warnings to drivers. Preferably with a graphical identification of the type and location of potential conflicts. Status Test Case 1 Link to external annexed documentation (if any) - SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 11 of 113 COSSIB

12 Table 4 IRIS Functionality Test Test site TEST FORM SP 5 WP 5 Prototype Version - System Configuration - HW Release See Desrciption in D5.6.2 SW Release - DFL_v1.0.8 (incl. IRIS) - SP5 Message Manger_v PG LDM using the C++ interface R_1_7_10 and NavteqLDM (LDM data model version ) - SNT router Translator (CVIS release 7) - CALM manager (CVIS release 7) Compiled by / Company TUM Date August 2009 and 25 February 2010 Form Progressive Numb. 2 Functional Component IRIS Reference Document D5.3.3 Test Type Test Objective Test Purpose Test Environment Integration Verification Correctness Test site Test Case 4: IRIS Functionality Test Test site Test Description Prerequisite : - The LDM contains the detailed static description of the intersection such as allocations of stop lines, the reference tracks, signal groups and the assignment of the signal groups of the appropriate lanes. In addition to that, the data concerning moving objects need to be provided by the SP2 data platform. The connection to the SINTECH router is set up and the routing software works properly. For the comprehensive description of the test site and the test procedure, check D Description: - It is checked, whether in each test scenario the vehicles and vulnerable road user are present in the LDM. It will be checked, whether the traffic light status is in the LDM. In the case IRIS generates a warning, it will be checked, whether this warning is transmitted to the router via the SP5 message manager or not. SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 12 of 113 COSSIB

13 Red Light Violation: The violator is warned Red Light Violation: The other road users are warned Gerichtsstraße virtual stop line Need to be marked at the road need to changed in LDM SAFESPOT Vehicle Critical Warning (broadcast) Critical Warning Zone virtual stop line Need to be marked at the road need to changed in LDM RSU TLC Critical Distance Critical Warning Zone RSU TLC Critical Distance Critical Warning (unicast) 5 20m SAFESPOT Vehicle Critical Warning (broadcast) Critical Warning (unicast) 5 20m SAFESPOT Vehicle (but does not react on warning) SAFESPOT Vehicle Hamburgerstraße West East Right Turning: cyclist / pedestrian warning Left Turning: oncoming vehicle warning RSU TLC VRU (cyclist) (no warning possible) RSU TLC Critical Warning (unicast) SAFESPOT Vehicle Critical Warning (unicast) SAFESPOT Vehicle Critical Distance Critical Warning Zone Critical Warning Zone SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 13 of 113 COSSIB

14 Requirements: - SP5_RQ06_19_v1.0: The system shall be able to provide a priority level for each transmitted message. This is valid for V2I as well as for I2V communication - SP5_RQ07_19_v1.0: The system shall be able to determine the period of validity of the transmitted and received messages. - SP5_RQ21_36_v1.0: The system must be an open system that will be able to host modules from various vendors. - SP5_RQ26_36_v1.0: The system shall receive the exact position of all pedestrians at the intersection from the LDM. - SP5_RQ28_19_v1.0: The short range communication shall be available at the intersection itself and its vicinity (minimum 150 m in each direction). - SP5_RQ33_19_v1.0: All data transmitted from the vehicle to the infrastructure shall have a timestamp referring to the creation time of the data (and not to the transmission time point). - SP5_RQ34_19_v1.0: The roadside sub-system must be able to address data directly to one vehicle distinctively (or a group of vehicles) or to broadcast to all vehicles in the vicinity of the RSU. - SP5_RQ40_19_v1.0: Messages that are exchanged between vehicles and RSU have to be unified to insure system interoperability. - SP5_RQ41_19_v1.0: The LDM shall describe the static geometry of critical points (e.g. intersections) in a detailed, accurate and systematic way. The geometry shall comprise at least approaches, exits, lanes (also bicycle lanes), stop lines and pedestrian crossings as well as the topology of the lanes (left-turn, right-turn, straight ahead). - SP5_RQ42_19_v1.0: The LDM shall provide a unique scheme for dynamic traffic information to refer to. - SP5_RQ44_19_v1.0: The system shall receive the position of the vehicles with an accuracy enabling to distinguish between two vehicles. - SP5_RQ45_27_v1.0: In critical points the position of the vehicles shall be determined with a minimal accuracy of +/- 1m. - SP5_RQ46_19_v1.0: The positioning of vehicles have to fulfil the accuracy up to a lane detection extend. - SP5_RQ47_19_v1.0: In case of pedestrians and cyclists the system shall take into account demand signals (push buttons) or data from according road-side sensors. - SP5_RQ49_19_v1.0: The RSU subsystem shall be able to receive at minimum the current vehicle position and speed. - SP5_RQ50_36_v1.0: The system shall be able to transmit the warning / recommendation to equipped vehicles. - SP5_RQ53_19_v1.0: The system shall receive in the vicinity of the urban intersection the position, speed and acceleration and driving direction with a frequency of 2/sec or shorter. - SP5_RQ54_36_v1.0: The system shall receive in the vicinity of the urban intersection extended dynamic vehicle data like position of brake and acceleration pedal, angel of steering wheel. - SP5_RQ56_19_v1.0: The system shall receive in the vicinity of the urban intersection the indicator state of the vehicles or alternatively / additionally in case of an activated navigation system - their turning relations with respect to this intersection. - SP5_RQ61_36_v1.0: The system must be able to receive data from the vehicles as well as the infrastructure. - SP5_RQ79_6_v1.1: The intended route on the intersection shall be known. Either in some direct manner, or indirectly through the status of vehicles indicator. - SP5_RQ80_6_v1.1: Information about emergency vehicles that need to ignore a red light shall be available. - SP5_RQ86_6_v1.1: It shall be possible to give warnings to drivers. Preferably with a graphical identification of the type and location of potential conflicts. Use cases: - SP5_UC22: Safe signalized intersection (crossing, turning) - SP5_UC31: Safe signalized intersection (red light violation) Expected Values / Results The requirements are met. Specifically: - successful interaction with the LDM - successful interaction with the SP2 data platform - successful reception of the IRIS message at the message manager and the SINTECH router, respectively SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 14 of 113 COSSIB

15 SP5_RQ06_19_v1.0 SP5_RQ07_19_v1.0 SP5_RQ21_36_v1.0 SP5_RQ26_36_v1.0 SP5_RQ28_19_v1.0 SP5_RQ33_19_v1.0 SP5_RQ34_19_v1.0 SP5_RQ40_19_v1.0 SP5_RQ41_19_v1.0 SP5_RQ42_19_v1.0 SP5_RQ44_19_v1.0 SP5_RQ45_27_v1.0 Obtained Values / Results The system shall be able to provide a priority level for each transmitted message. This is valid for V2I as well as for I2V communication The system shall be able to determine the period of validity of the transmitted and received messages. The system must be an open system that will be able to host modules from various vendors. The system shall receive the exact position of all pedestrians at the intersection from the LDM. The short range communication shall be available at the intersection itself and its vicinity (minimum 150 m in each direction). All data transmitted from the vehicle to the infrastructure shall have a timestamp referring to the creation time of the data (and not to the transmission time point). The roadside sub-system must be able to address data directly to one vehicle distinctively (or a group of vehicles) or to broadcast to all vehicles in the vicinity of the RSU. Messages that are exchanged between vehicles and RSU have to be unified to insure system interoperability. The LDM shall describe the static geometry of critical points (e.g. intersections) in a detailed, accurate and systematic way. The geometry shall comprise at least approaches, exits, lanes (also bicycle lanes), stop lines and pedestrian crossings as well as the topology of the lanes (left-turn, right-turn, straight ahead). The LDM shall provide a unique scheme for dynamic traffic information to refer to. The system shall receive the position of the vehicles with an accuracy enabling to distinguish between two vehicles. In critical points the position of the vehicles shall be determined with a minimal accuracy of +/- 1m. System was tested with PEEK and Siemens Traffic Light Controllers. The laser scanners have provided positions of the pedestrians being at the crossing, which was part of the scenario at the intersection. On the main road in Dortmund up to 500m could be achieved. In the different scenarios, either unicast or broadcast modes of the communication were used. This strongly depends on the vehicle s positioning system. For the test purpose, the positioning was sufficient. SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 15 of 113 COSSIB

16 SP5_RQ46_19_v1.0 The positioning of vehicles have to fulfil the accuracy up to a lane detection extend. SP5_RQ47_19_v1.0 In case of pedestrians and cyclists the system shall This feature was not implemented. take into account demand signals (push buttons) or not tested data from according road-side sensors. SP5_RQ49_19_v1.0 The RSU subsystem shall be able to receive at minimum the current vehicle position and speed. SP5_RQ50_36_v1.0 The system shall be able to transmit the warning / recommendation to equipped vehicles. SP5_RQ53_19_v1.0 The system shall receive in the vicinity of the urban intersection the position, speed and acceleration and driving direction with a frequency of 2/sec or shorter. SP5_RQ54_36_v1.0 The system shall receive in the vicinity of the urban intersection extended dynamic vehicle data like position of brake and acceleration pedal, angel of steering Partly wheel. SP5_RQ56_19_v1.0 The system shall receive in the vicinity of the urban intersection the indicator state of the vehicles or Only the indicator status was received, but no information on the intended rout based in the alternatively / additionally in case of an activated Partly navigation system s advice. navigation system - their turning relations with respect to this intersection. SP5_RQ61_36_v1.0 The system must be able to receive data from the vehicles as well as the infrastructure. SP5_RQ79_6_v1.1 The intended route on the intersection shall be known. The indicator status was received. Either in some direct manner, or indirectly through the status of vehicles indicator. SP5_RQ80_6_v1.1 Information about emergency vehicles that need to ignore a red light shall be available. not tested SP5_RQ86_6_v1.1 It shall be possible to give warnings to drivers. Preferably with a graphical identification of the type and location of potential conflicts. Status Test Case 1 Link to external annexed documentation (if any) - SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 16 of 113 COSSIB

17 Table 5 SpA_01 Simulation TEST FORM SP 5 WP 5 Prototype Version - System Configuration - LDM Schema 25, LDM HW Release - SW Release server 11.9, Compiled by / Company S. Glaser / LCPC Date October 2009 SP1 Framework v10 Form Progressive Numb. 3 Functional Component SpA_01 Reference Document D5.3.1 Prerequisite : - Functional LDM and VANET modules Test Type Test Objective Test Purpose Test Environment Unitary Verification Correctness Simulation Test Case 1: Speed transition at a straight road Test Description Description: The aim of this test is to prove the basic functionality of SpA_01. It warns the driver on excessive speed, regarding the legal speed limit. The road is a simple straight line with a modification of the speed limit at a given position. - The driver enters an area that is covered by a road-side unit (RSU), the vehicle transmits its beaconing message. - The RSU detects the vehicle, and starts managing it. - If the vehicle speed is higher than the speed limit, a warning is sent from the RSU - The vehicle receives the message and displays the warning to the driver - Vehicles entering the area are chosen randomly, as is their speed Specific test conditions: - The vehicle type is identified using simple classes: passenger car or truck (which may have a specific speed limit) - The nominal road speed ranges from 90km/h; 110km/h and 130 km/h - The modification of the speed in the middle of the road section decrease to 50km/h, 90km/h and 110km/h respectively SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 17 of 113 COSSIB

18 V Tools: LDM Data player Internally developed tools for the vehicle S Requirements: - RQ02_36_v1.0 (Message Management) The system shall be able to decide if and to whom a message has to be send. - RQ06_19_v1.0 (Priority Level of Message) The system shall be able to provide a priority level for each transmitted message. This is valid for V2I as well as for I2V communication - RQ12_33_v1.0 (Data exchange) The system shall be able to transmit information towards several supporters (e.g. VMS, radio, PND). - RQ16_36_v1.0 (Query from the LDM) The system shall be able to query needed data from the LDM. - RQ17_36_v1.0 (Receive data from the LDM) The system shall be able to receive and handle the LDM data. - RQ18_19_v1.0 (Simultaneity) The system shall be able to manage all vehicles in the vicinity of a critical point (e.g. the intersection) simultaneously. - RQ19_36_v1. (Scalability) The system shall have nearly the same performance in case there are just two or 80 vehicles to manage. - RQ20_19_v1.0 (Time of Warning Generation) The system shall generate and transmit warning information to the drivers timely enough, so that the reaction time left to the drivers is appropriate. - RQ24_19_v1.0 (Robustness of System) The system has to be robust enough in order to operable under different extreme weather conditions (heat, cold, snow, etc.). - RQ29_27_v1.0 (Range of Communication) The short range communication shall be available at the critical points and their vicinity (minimum 600 m). - RQ31_27_v1.0 (Simultaneous Communication) The system shall be able to communicate with all vehicles in the vicinity of the intersection (minimum radius <= 600 m) simultaneously. - RQ48_36_v1.0 (Environmental Data) The system shall receive data from environmental sensors about weather (rain, snow, fog ). - RQ49_19_v1.0 (Position and speed of vehicles) The RSU subsystem shall be able to receive at minimum the current vehicle position and speed. - RQ50_36_v1.0 (Transmission of warnings) The system shall be able to transmit the warning / recommendation to equipped vehicles. - RQ68_10_v1.0 (Road status) The system shall acquire data on the weather conditions in the 'critical area' (extent TBD) of the obstacle or accident, i.e. wet/dry road surface, rain, fog, ice, which may affect stopping distances and visibility. - RQ69_10_v1.0 (V Warning) The system shall send appropriate warnings to equipped vehicles within the critical (e and a) zones to permit them to reduce speed and/or change lane and avoid a (secondary) collision. SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 18 of 113 COSSIB

19 User Needs: SP5_UN38: Give to the driver a continuous access to the information of speed limit. SP5_UN39: Infrastructure can warn the driver in case of excessive speed Use cases: SP5_UC32 : Prevention of Driver Excessive Speed Expected Values / Results The requirements are met, specifically: - The vehicle transmits its beaconing message. - The RSU detects the vehicle - In case of speeding, the vehicle is specifically warned, otherwise, the vehicle on-board system knows the legal speed limit - The legal speed limit in the first area is consistent with the speed limit in the RSU LDM - The time elapsed between the crossing of the new area and the reception of the new speed limit, is not higher than 1s - The time between the start of the over speeding and the reception of the message from the RSU is not higher than 1s - The vehicle type is identified using simple classes Obtained Values / Results RQ02_36_v1.0 Message Management) The system shall be able to decide if and to whom a message has to be send. RQ06_19_v1.0 (Priority Level of Message) The system shall be able to provide a priority level for each transmitted message. This is valid for V2I as well as for I2V communication RQ12_33_v1.0 (Data exchange) The system shall be able to transmit Only VMS link was verified information towards several supporters (e.g. VMS, radio, PND). RQ16_36_v1.0 (Query from the LDM) The system shall be able to query needed data from the LDM. RQ17_36_v1.0 (Receive data from the LDM) The system shall be able to receive and handle the LDM data. RQ18_19_v1.0 (Simultaneity) The system shall be able to manage all vehicles in the vicinity of a critical point (e.g. the intersection) simultaneously. RQ19_36_v1. (Scalability) The system shall have nearly the same Up to 10 vehicles performance in case there are just two or 80 vehicles to Partially manage. RQ20_19_v1.0 to the drivers is appropriate. (Time of Warning Generation) The system shall generate and transmit warning information to the Not tested Verification only aims at testing the application itself not the interface with the driver drivers timely enough, so that the reaction time left RQ24_19_v1.0 (Robustness of System) The system has to be robust Not tested The hardware was not tested under these SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 19 of 113 COSSIB

20 RQ29_27_v1.0 RQ31_27_v1.0 RQ48_36_v1.0 RQ49_19_v1.0 RQ50_36_v1.0 RQ68_10_v1.0 RQ69_10_v1.0 enough in order to operable under different extreme weather conditions (heat, cold, snow, etc.). (Range of Communication) The short range communication shall be available at the critical points and their vicinity (minimum 600 m). (Simultaneous Communication) The system shall be able to communicate with all vehicles in the vicinity of the intersection (minimum radius <= 600 m) simultaneously. (Environmental Data) The system shall receive data from environmental sensors about weather (rain, snow, fog ). (Position and speed of vehicles) The RSU subsystem shall be able to receive at minimum the current vehicle position and speed. (Transmission of warnings) The system shall be able to transmit the warning / recommendation to equipped vehicles. (Road status) The system shall acquire data on the weather conditions in the 'critical area' (extent TBD) of the obstacle or accident, i.e. wet/dry road surface, rain, fog, ice, which may affect stopping distances and visibility. (V Warning) The system shall send appropriate warnings to equipped vehicles within the critical (e and a) zones to permit them to reduce speed and/or change lane and avoid a (secondary) collision. Not tested Not tested Partially ok conditions The handled situations were mainly weather related. Status Test Case 1 Link to external annexed documentation (if any) SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 20 of 113 COSSIB

21 Table 6 SpA_01 Test site TEST FORM SP 5 WP 5 Prototype Version - System Configuration - LDM Schema 25, LDM HW Release VANET SW Release server 11.9, Compiled by / Company S. Glaser / LCPC Date March 24-26, 2010 SP1 Frame work V10 Form Progressive Numb. 3b Functional Component SpA_01 Reference Document D5.3.1 Test Type Test Objective Test Purpose Test Environment Integration Validation Performance Test Site Prerequisite : - Functional LDM and VANET modules Test Case 1: Speed transition at a straight road Test Description Description: The aim of this test is to prove the basic functionality of SpA_01. It warns the driver on excessive speed, regarding the legal speed limit. The road is a simple straight line with a modification of the speed limit at a given position. - The driver enters an area that is covered by a road-side unit (RSU), the vehicle transmits its beaconing message. - The RSU detects the vehicle, and starts managing it. - If the vehicle speed is higher than the speed limit, a warning is sent from the RSU - The vehicle receives the message and displays the warning to the driver - Vehicles entering the area are chosen randomly, as is their speed Specific test conditions: - The vehicle type is identified using simple classes: passenger car or truck (which may have a specific speed limit) - The nominal road speed range depends on the test site possibility Tools: Recorded data from the VANET SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 21 of 113 COSSIB

22 Requirements: - RQ02_36_v1.0 (Message Management) The system shall be able to decide if and to whom a message has to be send. - RQ06_19_v1.0 (Priority Level of Message) The system shall be able to provide a priority level for each transmitted message. This is valid for V2I as well as for I2V communication - RQ12_33_v1.0 (Data exchange) The system shall be able to transmit information towards several supporters (e.g. VMS, radio, PND). - RQ16_36_v1.0 (Query from the LDM) The system shall be able to query needed data from the LDM. - RQ17_36_v1.0 (Receive data from the LDM) The system shall be able to receive and handle the LDM data. - RQ18_19_v1.0 (Simultaneity) The system shall be able to manage all vehicles in the vicinity of a critical point (e.g. the intersection) simultaneously. - RQ19_36_v1. (Scalability) The system shall have nearly the same performance in case there are just two or 80 vehicles to manage. - RQ20_19_v1.0 (Time of Warning Generation) The system shall generate and transmit warning information to the drivers timely enough, so that the reaction time left to the drivers is appropriate. - RQ24_19_v1.0 (Robustness of System) The system has to be robust enough in order to operable under different extreme weather conditions (heat, cold, snow, etc.). - RQ29_27_v1.0 (Range of Communication) The short range communication shall be available at the critical points and their vicinity (minimum 600 m). - RQ31_27_v1.0 (Simultaneous Communication) The system shall be able to communicate with all vehicles in the vicinity of the intersection (minimum radius <= 600 m) simultaneously. - RQ48_36_v1.0 (Environmental Data) The system shall receive data from environmental sensors about weather (rain, snow, fog ). - RQ49_19_v1.0 (Position and speed of vehicles) The RSU subsystem shall be able to receive at minimum the current vehicle position and speed. - RQ50_36_v1.0 (Transmission of warnings) The system shall be able to transmit the warning / recommendation to equipped vehicles. - RQ68_10_v1.0 (Road status) The system shall acquire data on the weather conditions in the 'critical area' (extent TBD) of the obstacle or accident, i.e. wet/dry road surface, rain, fog, ice, which may affect stopping distances and visibility. - RQ69_10_v1.0 (V Warning) The system shall send appropriate warnings to equipped vehicles within the critical (e and a) zones to permit them to reduce speed and/or change lane and avoid a (secondary) collision. User Needs: SP5_UN38: Give to the driver a continuous access to the information of speed limit. SP5_UN39: Infrastructure can warn the driver in case of excessive speed Use cases: SP5_UC32 : Prevention of Driver Excessive Speed Expected Values / Results The requirements are met, specifically: - The vehicle transmits its beaconing message. - The RSU detects the vehicle - In case of speeding, the vehicle is specifically warned, otherwise, the vehicle on-board system knows the legal speed limit - The legal speed limit in the first area is consistent with the speed limit in the RSU LDM - The time elapsed between the crossing of the new area and the reception of the new speed limit, is not higher than 1s - The time between the start of the over speeding and the reception of the message from the RSU is not higher than 1s - The vehicle type is identified using simple classes SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 22 of 113 COSSIB

23 RQ02_36_v1.0 RQ06_19_v1.0 RQ12_33_v1.0 RQ16_36_v1.0 RQ17_36_v1.0 RQ18_19_v1.0 RQ19_36_v1. RQ20_19_v1.0 to the drivers is appropriate. RQ24_19_v1.0 RQ29_27_v1.0 RQ31_27_v1.0 RQ48_36_v1.0 RQ49_19_v1.0 Obtained Values / Results Message Management) The system shall be able to decide if and to whom a message has to be send. (Priority Level of Message) The system shall be able to provide a priority level for each transmitted message. This is valid for V2I as well as for I2V communication (Data exchange) The system shall be able to transmit information towards several supporters (e.g. VMS, radio, PND). (Query from the LDM) The system shall be able to query needed data from the LDM. Partially (Receive data from the LDM) The system shall be able to receive and handle the LDM data. (Simultaneity) The system shall be able to manage all vehicles in the vicinity of a critical point (e.g. the intersection) simultaneously. (Scalability) The system shall have nearly the same performance in case there are just two or 80 vehicles to manage. (Time of Warning Generation) The system shall generate and transmit warning information to the drivers timely enough, so that the reaction time left (Robustness of System) The system has to be robust enough in order to operable under different extreme weather conditions (heat, cold, snow, etc.). (Range of Communication) The short range communication shall be available at the critical points and their vicinity (minimum 600 m). (Simultaneous Communication) The system shall be able to communicate with all vehicles in the vicinity of the intersection (minimum radius <= 600 m) simultaneously. (Environmental Data) The system shall receive data from environmental sensors about weather (rain, snow, fog ). (Position and speed of vehicles) The RSU subsystem shall be able to receive at minimum the current vehicle position and speed. Not Partially Not tested Not Not Only VMS link was evaluated The system uses NavtTeq LDM and subscription mechanism. Closing the subscription makes the LDM crash Only three vehicles System has sometimes huge latency (more than 1s) Hardware was not evaluated under these conditions Ranges were around 300m, with a clear line of sight Ranges were around 300m, with a clear line of sight SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 23 of 113 COSSIB

24 RQ50_36_v1.0 (Transmission of warnings) The system shall be able to transmit the warning / recommendation to equipped vehicles. RQ68_10_v1.0 (Road status) The system shall acquire data on the weather conditions in the 'critical area' (extent TBD) of the obstacle or accident, i.e. wet/dry road surface, rain, fog, ice, which may affect stopping distances and visibility. RQ69_10_v1.0 (V Warning) The system shall send appropriate warnings to equipped vehicles within the critical (e and a) zones to permit them to reduce speed and/or change lane and avoid a (secondary) collision. Status Test Case 1 Link to external annexed documentation (if any) Partially ok Situation handled were mainly weather situations SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 24 of 113 COSSIB

25 Table 7 SpA_03 Simulation TEST FORM SP 5 WP 5 Prototype Version - System Configuration - LDM Schema 25, LDM HW Release - SW Release server 11.9, Compiled by / Company S. Glaser / LCPC Date 01/2010 SP1 Frame work v10 Form Progressive Numb. 5 Functional Component SpA_03 Reference Document D5.3.1 Test Type Test Objective Test Purpose Test Environment Unitary Verification Correctness Simulation Prerequisite : - Functional LDM and VANET modules Test Case 1: Basic Validation of SpA_03 Test Description Description: The aim of this test is to prove the basic functionality of SpA_03: the definition of a speed profile with respect to the road geometry and the road surface condition. The application warns the driver on excessive speed, regarding the computed speed profile. The road presents a simple geometry with a straight line a clothoid and a curve. The radius of the curve can be set at different values. - The driver enters an area covered by a road side unit, the vehicle transmits its beaconing message - The RSU detects the vehicle, and starts managing it. - If the vehicle speed is higher than the speed limit, a warning is sent from the RSU - The vehicle receives the message and displays the warning to the driver SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 25 of 113 COSSIB

26 ρ V s s Requirements: - RQ34_19_v1.0 (Directly address a Vehicle) The roadside sub-system must be able to address data directly to one vehicle distinctively (or a group of vehicles) or to broadcast to all vehicles in the vicinity of the RSU. - RQ35_19_v1.0 (Time of Data Delay 2) In the roadside sub-system the delay between generation of warnings and transmission of this data to the vehicles shall be smaller than 50 ms. - RQ36_27_v1.0 (Time of Connection) The time needed to set-up a connection between an incoming vehicle and the roadside communication system should be smaller than 0.8 seconds. - RQ41_19_v1.0 (Static Map contents) The LDM shall describe the static geometry of critical points (e.g. intersections) in a detailed, accurate and systematic way. The geometry shall comprise at least approaches, exits, lanes (also bicycle lanes), stop lines and pedestrian crossings as well as the topology of the lanes (left-turn, right-turn, straight ahead). - RQ46_19_v1.0 (Position of Vehicles 3) The positioning of vehicles have to fulfill the accuracy up to a lane detection extend. - RQ49_19_v1.0 (Position and speed of vehicles) The RSU subsystem shall be able to receive at minimum the current vehicle position and speed. - RQ50_36_v1.0 (Transmission of warnings) The system shall be able to transmit the warning / recommendation to equipped vehicles. User needs: SP5_UN39: Infrastructure can warn the driver in case of excessive speed. Use cases: SP5_UC41 : Prevention of lack of adherence SP5_UC45 : Sudden reduce Visibility Expected Values / Results The requirements are met. Specifically: - In the case of over speeding, the vehicle is warned. Otherwise, the vehicle on-board system knows the legal speed limit SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 26 of 113 COSSIB

27 RQ34_19_v1.0 RQ35_19_v1.0 RQ36_27_v1.0 Obtained Values / Results (Directly address a Vehicle) The roadside sub-system must be able to address data directly to one vehicle distinctively (or a group of vehicles) or to broadcast to all vehicles in the vicinity of the RSU. (Time of Data Delay 2) In the roadside subsystem the delay between generation of Partially warnings and transmission of this data to the vehicles shall be smaller than 50 ms. (Time of Connection) The time needed to Not tested set-up a connection between an incoming vehicle and the roadside communication system should be smaller than 0.8 seconds. RQ41_19_v1.0 (Static Map contents) The LDM shall describe the static geometry of critical points (e.g. intersections) in a detailed, accurate and systematic way. The geometry shall comprise at least approaches, exits, lanes (also bicycle lanes), stop lines and pedestrian crossings as well as the topology of the lanes (left-turn, right-turn, straight ahead). RQ46_19_v1.0. (Position of Vehicles 3) The positioning of vehicles have to fulfill the accuracy up to a lane detection extend RQ49_19_v1.0. (Position and speed of vehicles) The RSU subsystem shall be able to receive at minimum the current vehicle position and speed RQ50_36_v1.0 (Transmission of warnings) The system shall be able to transmit the warning / recommendation to equipped vehicles. Status Test Case1 Link to external annexed documentation (if any) Mainly depends on the load of the LDM This RQ was not evaluated because we have no exact description of the intended performance of the VANET: when is a vehicle qualified as incoming, what is exactly the range? However, we have evaluated the average range of communication, taking into account the various antenna provided. SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 27 of 113 COSSIB

28 Prerequisite : - Functional LDM and VANET modules Test Case 2: Road surface condition interaction with SpA Test Description Description: The aim of this test is to verify that the SpA_03 application can handle the environmental variations and that it defines the associated speed. In this test case, the road is the same as defined in test case 1. Various weather conditions are injected during the simulation, as rain (various densities) or fog. Tools: LDM Data player SiVIC simulator and SP2 weather monitoring Requirements: - RQ48_36_v1.0 (Environmental Data) The system shall receive data from environmental sensors about weather (rain, snow, fog ). - RQ60_36_v1.0 (Lack of Friction) The system shall receive information about the lack of friction of a road segment. User Needs: - SP5_UN39: Infrastructure can warn the driver in case of excessive speed. - SP5_UN49: Be warned in case of low adherence road - SP5_UN51: The system transmits warning information towards the driver approaching the risky site that signals a slippery bend or a puddle road. Use cases: SP5_UC41 : Prevention of lack of adherence SP5_UC45 : Sudden reduce Visibility Expected Values / Results The requirements are met, specifically two results are expected: the speed advice is varied in accordance with the weather status a warning of over speeding is provided, which is a function of the weather condition Obtained Values / Results SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 28 of 113 COSSIB

29 RQ48_36_v1.0 (Environmental Data) The system shall receive data from environmental sensors about weather (rain, snow, fog ). RQ60_36_v1.0. (Lack of Friction) The system shall receive information about the lack of friction of a road segment Status Test Case 2 Link to external annexed documentation (if any) Not tested Data were directly generated with a dummy interface Not tested Road friction was estimated by rain density SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 29 of 113 COSSIB

30 Table 8 SpA_03 Test site TEST FORM SP 5 WP 5 Prototype Version - System Configuration - LDM Schema 25, LDM HW Release VANET SW Release serveur 11.9, SP1 Framework V10 Compiled by / Company S. Glaser / LCPC Date 04/2010 Form Progressive Numb. 5b Functional Component SpA_03 Reference Document D5.3.1 Test Type Test Objective Test Purpose Test Environment Integration Verification Correctness Test Site Prerequisite : - Functional LDM and VANET modules Test Case 1: Basic Validation of SpA_03 Test Description Description: The aim of this test is to prove the basic functionality of SpA_03: the definition of a speed profile with respect to the road geometry and the road surface condition. The application warns the driver on excessive speed, regarding the computed speed profile. The road presents a simple geometry with a straight line a clothoid and a curve. The radius of the curve is set accordingly to data stored in the LDM - The driver enters an area covered by a road side unit, the vehicle transmits its beaconing message - The RSU detects the vehicle, and starts managing it. - If the vehicle speed is higher than the speed limit, a warning is sent from the RSU - The vehicle receives the message and displays the warning to the driver SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 30 of 113 COSSIB

31 ρ V s s Requirements: - RQ34_19_v1.0 (Directly address a Vehicle) The roadside sub-system must be able to address data directly to one vehicle distinctively (or a group of vehicles) or to broadcast to all vehicles in the vicinity of the RSU. - RQ35_19_v1.0 (Time of Data Delay 2) In the roadside sub-system the delay between generation of warnings and transmission of this data to the vehicles shall be smaller than 50 ms. - RQ36_27_v1.0 (Time of Connection) The time needed to set-up a connection between an incoming vehicle and the roadside communication system should be smaller than 0.8 seconds. - RQ41_19_v1.0 (Static Map contents) The LDM shall describe the static geometry of critical points (e.g. intersections) in a detailed, accurate and systematic way. The geometry shall comprise at least approaches, exits, lanes (also bicycle lanes), stop lines and pedestrian crossings as well as the topology of the lanes (left-turn, right-turn, straight ahead). - RQ46_19_v1.0 (Position of Vehicles 3) The positioning of vehicles have to fulfill the accuracy up to a lane detection extend. - RQ49_19_v1.0 (Position and speed of vehicles) The RSU subsystem shall be able to receive at minimum the current vehicle position and speed. - RQ50_36_v1.0 (Transmission of warnings) The system shall be able to transmit the warning / recommendation to equipped vehicles. User needs: SP5_UN39: Infrastructure can warn the driver in case of excessive speed. Use cases: SP5_UC41 : Prevention of lack of adherence SP5_UC45 : Sudden reduce Visibility Expected Values / Results The requirements are met. Specifically: - In the case of over speeding, the vehicle is warned. Otherwise, the vehicle on-board system knows the legal speed limit SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 31 of 113 COSSIB

32 Obtained Values / Results RQ34_19_v1.0 (Directly address a Vehicle) The roadside sub-system must be able to address data directly to one vehicle distinctively (or a group of vehicles) or to broadcast to all vehicles in the vicinity of the RSU. RQ35_19_v1.0 (Time of Data Delay 2) In the roadside subsystem the delay between generation of warnings and transmission of this data to the vehicles shall be smaller than 50 ms. Not RQ36_27_v1.0 (Time of Connection) The time needed to Not tested set-up a connection between an incoming vehicle and the roadside communication system should be smaller than 0.8 seconds. RQ41_19_v1.0 (Static Map contents) The LDM shall ok describe the static geometry of critical points (e.g. intersections) in a detailed, accurate and systematic way. The geometry shall comprise at least approaches, exits, lanes (also bicycle lanes), stop lines and pedestrian crossings as well as the topology of the lanes (left-turn, right-turn, straight ahead). RQ46_19_v1.0. (Position of Vehicles 3) The positioning of Not vehicles have to fulfill the accuracy up to a lane detection extend RQ49_19_v1.0. (Position and speed of vehicles) The RSU subsystem shall be able to receive at minimum the current vehicle position and speed RQ50_36_v1.0 (Transmission of warnings) The system shall be able to transmit the warning / recommendation to equipped vehicles. Status Test Case1 Link to external annexed documentation (if any) Lot of delay with map interaction Only beacons were available to log the communication. We have no means to estimate the time of connection We used a standard GPS mode, without any vision sensor to detect lane. SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 32 of 113 COSSIB

33 Table 9 H&IW_01 and H&IW_02 Simulation TEST FORM SP 5 WP 5 Prototype Version - System Configuration - HW Release - SW Release - Compiled by / Company MIZAR Date 22/05/2009 (test case 1) 27/05/2009 (test case 2) Form Progressive Numb. 6 Functional Component H&IW_01 H&IW_02 Reference Document D5.2.3 Test Type Test Objective Test Purpose Test Environment Integration Verification Correctness Simulation Test Case 1: Basic functionality test for H&IW_01 and H&IW_02 #1: Event Notification to BlackSpot Monitor Test Description Prerequisite : - Information about an event inside the traffic event table of LDM. - [trafficevent.eventcause = 35 (ghost driver)] - [trafficevent.eventcause = 28 (animal)] - [trafficevent.eventcause = 30 (pedestrian)] - [trafficevent.eventcause = 35 (obstruction on road)] - Simulation of [motorvehicle speed > 20 km/h] - Sensor source simulator: WSN (Wireless Sensor Network), RFID sensors, Thermal imaging cameras (provided by VTT). - Synchronization of the clock and Positioning: - connect GPS receiver to Positioning PC (Application PC) - SP3 Positioning SW (responsible to acquire PPS signal from GPS and synchronize PC time) - Start NTP Server on Positioning PC - Start NTP Client on RSU PC and on ROUTER VANET PC) - In order to check synchronization it is possible to compare Beacon sending time from RSU to beacon receiving time from Vehicle using the file log vanet_msg_incoming. - DataReceiver from SP2 - RSU with OS linux and Windows Description: The goal of this test is verify the correctness in the communication between the BlackSpotMonitor (application responsible for detecting H&IW events) and the LDM after the detection of each H&IW event. This basic function is needed for all use cases of the H&IW 01 and The H&IW sub-application is by means of a trigger launched by the LDM communicates with the BlackSpotMonitor. SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 33 of 113 COSSIB

34 - The BlackSpotMonitor creates a log that shows the state of the message (Correct message received from LDM, Error connection with LDM, Error message received from LDM) Extended Description - The test start with the simulation of: o Vehicle that goes in the wrong way (Ghost Driver) o Pedestrian on Motorway o Accident as Obstacle, Slow moving vehicles, or traffic jams as an obstacle - The simulator of the relative application generates a corresponding message regarding the presence of an event o For case Ghost Driver detected by WSN and RFID o For case Pedestrian on Motorway detected by VTT o For case Obstacle by WSN - The LDM sends the corresponding message to the BlackSpotMonitor depending on the case - The Black Spot Monitor receives this message. Two important test were developed: 1. One using one simulator per time, (one simulator for each type of sensor) 2. Second using all use cases simultaneouly. Test lab location: CRF Centro di Ricerche FIAT lab Requirements: - SP5_RQ02_36_v1.0: The system shall be able to decide if and to whom a message has to be send. - SP5_RQ07_19_v1.0: The system shall be able to determine the period of validity of the transmitted and received messages. - SP5_RQ16_36_v1.0: The system shall be able to query needed data from the LDM. - SP5_RQ17_36_v1.0: The system shall be able to receive and handle the LDM data. Use Cases: - SP5_UC34_v1.0 (Ghost Driver) - SP5_UC17 (Pedestrian on Motorway) - SP5_UC13 (Accident as Obstacle) - SP5_UC14 (Traffic jams as obstacles (slow moving vehicles)) - SP5_UC15 (Traffic jams as an obstacle results of an accident, with poor visibility) Expected Values / Results The listed requirements are met, specifically, - The reception of the correct message in the BlackSpotMonitor for all the UC simulated. - Detection of the obstacle with the associated position A log that shows the message reception of the corresponding UC. Note: For SP5_UC13, SP5_UC14, SP5_UC15, the message event will be the same. SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 34 of 113 COSSIB

35 Obtained Values / Results Results The BlackSpotMonitor detects correctly the H&IW events. The high frequency used to writte data in the LDM makes that the trigger is not able to detect all the messages. Some traffic events are lost. The Trigger detects one event per each second. Conclusions (difference actual and expected requirements, system limitations) The two test were successful.. Even tough the use of high speed of sending UDP messages cause that the BlackSpotMonitor is not able to detect all the messages. Some traffic events are lost. The BlackSpotMonitor detects one event per each second. The events detected by the BlackSpotMonitor were correctly interpreted as H&IW events. The event log shows that the event cause corresponds with the parameters entered in the simulator regarding the type of Use Cases. It was important to implement a policy of cleanning the LDM because after some hours of test the LDM start to increase. The script implemented the information inside the database (each hour) was implemented. Status Test Case1 Link to external annexed documentation (if any) Test Case 2: Basic functionality test for H&IW_01 and H&IW_02 #2: Send and Reception of the Message HMI Test Description Prerequisite : In the table traffic event of LDM there is the information regarding the presence of an event corresponding to H&IW_02 or H&IW_01 - [motorvechicle speed > 20 km/h] - [trafficevent.eventcause = 35] - Sensor source: WSN and RFID sensors - Simulator of WSN, RFID and VTT messages. - DataReceiver from SP2 - RSU with OS linux and Windows - The BlackSpotMonitor have received the message from LDM. - All the test before have to be successful - Vanet unit inside RSU and Vanet in the vehicle - 1 vehicle Description: The goal of this test is to verify the correct operation of the generation of the SendHmiEventMessage and the Reception of the RSUHmiMsg in the vehicle. Extended Description: This is a IV communication test and has to be tested for all the use cases, to do this test it is necessary to do a simulation of all the cases in order to generate an event. After the reception of the corresponding notification message from the Black Spot Monitor the corresponding message has to be sent to the vehicle by means of Vanet. The Vanet has to be able to display a log of the correct receipt of this message and the correctness of it. Test lab location: - CRF Centro di Ricerche FIAT lab SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 35 of 113 COSSIB

36 Requirements: - SP5_RQ02_36_v1.0: The system shall be able to decide if and to whom a message has to be send. - SP5_RQ07_19_v1.0: The system shall be able to determine the period of validity of the transmitted and received messages. - SP5_RQ16_36_v1.0: The system shall be able to query needed data from the LDM. - SP5_RQ17_36_v1.0: The system shall be able to receive and handle the LDM data. Use Cases - SP5_UC34_v1.0 (Ghost Driver) - SP5_UC17 (Pedestrian on Motorway) - SP5_UC13 (Accident as Obstacle) - SP5_UC14 (Traffic jams as obstacles (slow moving vehicles)) - SP5_UC15 (Traffic jams as an obstacle results of an accident, with poor visibility) Expected Values / Results The requirements are met, specifically: - To Hmi Message that correspond to the use cases is received in the vehicle. - The messages arrive in the vehicle with the correct information of the obstacle and its position - The log will show a message of correct message reception.. Note: For SP5_UC14, SP5_UC15, SP5_UC13 the message event will be the same. Obtained Values / Results The expected results were achieved. Some issues were corrected in order to have the better response of the systems. The most relevant observations are: In the system the HMI message should be sent in broadcast. The first results failed because the HMI have been defined using a unicast instead that a broadcast destination address The configuration of the correct port was important. The use of a definition of a HMI message with a relative timevalidity gives better results than the use of an absolute time validity, because the possible difference between vehicle time and rsu time can affect the results Status Test Case 2 Link to external annexed documentation (if any) SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 36 of 113 COSSIB

37 Table 10 H&IW_01 Test site 1 TEST FORM SP 5 WP 5 Prototype Version Cofi 1.8 System Configuration RSU : PC / Linux PC / XP VANET / p OBU: Positioning PC PC / XP (Display) VANET / p HW Release 1 SW Release 1 Compiled by / Company COFIROUTE Date 10/03/2009 Form Progressive Numb. 10 Functional Component H&IW_01 Reference Document D5.3.5 Test Type Test Objective Test Purpose Test Environment Integration Verification Performance Test Site Test Case 1: Basic performance test for H&IW_01 (Obstacle) #1: Accident/Vehicle as obstacle Test Description Prerequisite: Functional - the functional unitary correctness validation simulation tests of H&IW_01_v0.1 should have been performed and assessed on the LDM of the test site (Navteq or TeleAtlas) - Integration of other SAFESPOT component of the architecture, both in vehicle and infrastructure, and corresponding test should have been performed and validated. Deployment - One RSU and two equipped vehicles are required for this test. - One equipped vehicle (B) is stopped within the range of the RSU preferably downstream the RSU (this is the Accident/Vehicle as obstacle). The other equipped vehicle (A) passes within the range of the RSU. It moves from upstream to downstream the RSU, in the direction of the road where the stopped vehicle VB is located (i.e. at the considered road section). - No specific sensors are required for this test. Accurate positioning and lane level details at the LDM are required for the extended version test. In addition, weather sensors at the RSU and/or at the vehicles are required in the extended version (note that RSU gateway to a legacy system can act as a weather sensor). Finally, VMS can be connected to the RSU to inform non-equipped vehicles. SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 37 of 113 COSSIB

38 VehB RSU VehA Description: The test consist on comparing a prepared table containing estimated time and position of when/where a warning from H&IW_01 should appear at a SAFESPOT equipped vehicle. This table is prepared for each test site (otherwise, it will contain too many entries). - First, the stopped vehicle (B) sends its beacons. - The H&IW_01 sub-application detects a stopped vehicle (an obstacle) through its BlackSpotMonitor function. - Then the ScenarioManager function is launched to compute the Threat level. - Then vehicle A enters the range of the RSU and sends its beacons. - The H&IW application sends the generic H&IW warning corresponding to the threat level (Comfort, Safety, Emergency) depending on its type, speed, road attributes and obstacle position. - If the vehicle A does not adapt its speed to stay in the comfort situation, the different Safety and Emergency warnings will follow. Test 1: vehicle A passes trough the coverage area of the event, with the maximum legal speed authorized at the road. The vehicle A does not adapt its speed and sees the three levels of warnings. First, the Comfort warning, then the Safety warning, this is shown 3 seconds before crossing the obstacle with constant speed, plus the estimated time to stop. The emergency warning is shown one second plus time to stop before crossing the event. Obviously, the vehicle A does not drive on the lane where the obstacle is located! Test 2: vehicle A slows down (to around 20 km/h) so it stays in the comfort situation. The driver should intuitively choose the exact speed reduction. The exact speed reduction expected by the H&IW application depends on many factors and will be calculated according to the specification of H&IW prior to the test. Extended description: - the application retrieves additional information from the LDM about weather and traffic status to adapt the warning message, i.e. the warning destination area should be reduced or extended depending on the weather and traffic status (this imply function scenarioanalysis() of object ScenarioManager). - then it generates the appropriate RsuHMIMessage and triggers the SendHMIPeriodicMessage from SP5_MessageManager (i.e. the SP5 interface with Vanet) and it triggers the display on possible VMS through the SP5ApplicationCoordinator. - The warning messages sent to vehicle A contains lane merging indications at the icon displayed onboard, if accurate positioning of the vehicles and a detailed LDM (lane level positioning) are available. Related Test sites: SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 38 of 113 COSSIB

39 from vehicle beacons: - Netherland/A16 (TeleAtlas) - West/A86 (Navteq) - Livic TBC (Navteq) - CG22 TBC (Navteq) from rsu : - Italy/Brescia-Prodova (Navteq) Tools: The timing values can be determined by analysing H&IW log files and observing SAFESPOT UDP messages. In addition, Esposytor, can be used. Requirements: H&IW functions - SP5_RQ02_36_v1.0 / Message Management: The system shall be able to decide if and to whom a message has to be send. - SP5_RQ05_36_v1.0 / Identify Safety Critical Situations: The system shall be able to identify safety critical situations at the surrounding of a critical point e.g. an urban intersection. - SP5_RQ06_19_v1.0 / Priority Level of Message: The system shall be able to provide a priority level for each transmitted message. This is valid for V2I as well as for I2V communication - SP5_RQ07_19_v1.0 / Validity of Messages: The system shall be able to determine the period of validity of the transmitted and received messages. - SP5_RQ09_19_v1.0 / Message Management 2 : The system must not send warnings concerning severe dangerous situations (that might cause drivers to extreme driving actions), which does in fact not happen. - SP5_RQ12_33_v1.0 / Data exchange: The system shall be able to transmit information towards several supporters (e.g. VMS, radio, PND). - SP5_RQ16_36_v1.0 / Query from the LDM: The system shall be able to query needed data from the LDM. - SP5_RQ17_36_v1.0 / Receive data from the LDM: The system shall be able to receive and handle the LDM data. LDM functionalities - SP5_RQ49_19_v1.0 / Position and speed of vehicles: The RSU subsystem shall be able to receive at minimum the current vehicle position and speed. - SP5_RQ68_10_v1.0 / Road status: The system shall acquire data on the weather conditions in the 'critical area' (extent TBD) of the obstacle or accident, SP5_RQ65_10_v1.0 / Obstacle description - The system shall provide other details of the obstacle where possible : type of object (accident, queue, rocks, dropped load, etc), lanes affected, speed (for moving object), precise location etc. i.e. wet/dry road surface, rain, fog, ice, which may affect stopping distances and visibility. TIMING and Performance - SP5_RQ18_19_v1.0 / Simultaneity: The system shall be able to manage all vehicles in the vicinity of a critical point (e.g. the intersection) simultaneously. - SP5_RQ20_19_v1.0 / Time of Warning Generation: The system shall generate and transmit warning information to the drivers timely enough, so that the reaction time left to the drivers is appropriate. - SP5_RQ22_10_v1.0 / R/S Warning: The system shall provide a roadside warning in fewer than 2 sec (TBC) in the critical zones to permit vehicles to reduce speed and/or change lane in order to avoid a (secondary) collision. - SP5_RQ35_19_v1.0 /Time of Data Delay 2: In the roadside sub-system the delay between generation of warnings and transmission of this data to the vehicles shall be smaller than 50 ms. MESSAGES EXCHANGE and Filtering - SP5_RQ34_19_v1.0 / Directly address a Vehicle: The roadside sub-system must be able to address data directly to one vehicle distinctively (or a group of vehicles) or to SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 39 of 113 COSSIB

40 broadcast to all vehicles in the vicinity of the RSU. - SP5_RQ50_36_v1.0 / Transmission of warnings: The system shall be able to transmit the warning / recommendation to equipped vehicles. - SP5_RQ69_10_v1.0: V Warning: The system shall send appropriate warnings to equipped vehicles within the critical (e and a) zones to permit them to reduce speed and/or change lane and avoid a (secondary) collision. - SP5_RQ37_19_v1.0 / Filter for relevant data: The vehicle subsystem should be able to filter the relevant information to the driver. - SP5_RQ51_19_v1.0 / HMI:In vehicle assistance HMI shall present warnings to the driver in an intelligent way without distracting him. Example: Use of different actuators to smooth signalizes hazards. - SP5_RQ48_36_v1.0 / Environmental Data: The system shall receive data from environmental sensors about weather (rain, snow, fog ). - SP5_RQ52_36_v1.0 / Static Vehicle Data: The system shall receive static vehicle data like width, length, type of vehicle, mass. - SP5_RQ60_36_v1.0 / Lack of Friction: The system shall receive information about the lack of friction of a road segment. - SP5_RQ61_36_v1.0 / Data Reception: The system must be able to receive data from the vehicles as well as the infrastructure. Use cases: SP5_UC13_v1.0 : Accident as an obstacle Expected Values / Results The requirements are met, specifically, - The VANET component sends the RsuHMImessage and the vehicle display the text / icon - Test 1 will validate the warning sequence. The expected sequence is: Comfort, Safety, Emergency - Test 2 should validate the ability to stay in what H&IW considers as a Comfort situation. - The time shift between the specification and the implementation/deployment will be measured. The expected time measures are based on the Safety Margin definition of H&IW, which describe where and when a warning (Safety or Critical) message should be received. It mainly describes the time left to the driver to react for each warning types. NB: The overall SAFESPOT system, including H&IW, will introduce delays due to computation and transmission that will reduce the time left to the driver to react. Therefore, different runs should be achieved to calibrate H&IW (mainly by adding average system delay into the Safety Margin Computation of H&IW). Test1 and Test2 should be performed for different vehicle speeds corresponding to different road environments and situations (30km/h, 50km/h, 90km/h etc...) and with different speed recommendations corresponding to different obstacles (road blocked, speed limited to 30km/h or 50km/h). This lead to different expected measures on the generated warnings, mainly in terms of destination areas, transmission dates and durations. The following figure shows an example of expected warning destination areas for a vehicle driving at 90km/h, and with an obstacle speed limitation to 50km/h: SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 40 of 113 COSSIB

41 An example of expected measures for test 1 are shown in the table below. Obtained Values / Results For validation test purpose, case 1 (vehicle/accident) and case 2 (roadwork) has been tested together in one time by writing a simulated event into the LDM; therefore, obtained values for test case 1 are displayed in hereafter test case 2. Status Test Case1 Link to external annexed documentation (if any) Test Case 2: Basic performance test for H&IW_01(Obstacle) #2: Road Works as an obstacle Test Description Prerequisite: Functional - the functional unitary correctness validation simulation tests of H&IW_01_v0.1 have been performed and assessed at the LDM of the test site (Navteq or TéléAtlas) - Integration of other SAFESPOT component of the architecture, both in vehicle and infrastructure, and corresponding test have been performed and validated. Deployment - One RSU and one equipped vehicle are required for this test. - An area at the road contains road works or something similar, downstream the RSU. - The RSU is equipped with a Gateway able to provide information about the road works location to the RSU. - The equipped vehicle is within the range of the RSU from upstream to downstream of the RSU, in the direction of the road where the test road works is located (i.e. in the considered road section). - No specific sensors are required for this test (except the RSU Gateway). Precise positioning and lane level details of the LDM are required for the extended version tests. In addition, weather sensors at the RSU and/or at the vehicles are required in extended version (note that RSU gateway to a legacy system can act as a weather sensor). Finally, VMS can be connected to the RSU to manage non-equipped vehicles. SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 41 of 113 COSSIB

42 Description: The test consist on comparing a prepared table containing estimated time and position on when/where a warning from H&IW_01 should appears at an SAFESPOT equipped vehicle. This table is be prepared for each test site (otherwise it will contains too many entries). - First the RSU Gateway transmits information about roadwork presence to the RSU. - the H&IW_01 sub-application detects the roadwork (an obstacle) through its BlackSpotMonitor function. - then the ScenarioManager function is launched to compute the Threat level. - Then the vehicle enters in the range of the RSU, sending its beacons. - The H&IW application sends the generic H&IW warning that corresponds to the threat level (Comfort, Safety, Emergency) depending on it s type, speed, road attributes and obstacle position. - If the vehicle does not adapt its speed to stay into the comfort situation, the different Safety and Emergency warnings will follow. Test 1: vehicle passes trough the coverage area of the event, with the maximum legal speed authorized at the road. The vehicle does not adapt its speed and sees the three levels of warnings. The Safety warning is shown 3 seconds plus the estimated time to stop before crossing the obstacle with constant speed. The emergency warning is shown one second plus time to stop before crossing the event. Obviously, the vehicle does not drive on the lane where the obstacle is located! Test 2: The vehicle slows down (to around 20 km/h) so it stays in the comfort situation. The exact speed reduction should be intuitively chosen by the driver. The exact speed reduction expected by the H&IW application depends on many factors and will be calculated according to the specification of H&IW prior to the test. Extended description: - The application retrieves additional information from the LDM about the weather and traffic status to adapt the warning message, i.e. the warning destination area should be reduced or extended depending on the weather and traffic status (using the function scenarioanalysis() of the object ScenarioManager). - Then it generates the appropriate RsuHMIMessage and triggers the SendHMIPeriodicMessage from SP5_MessageManager (i.e. the SP5 interface with Vanet) and it triggers the display on possible VMS through the SP5 ApplicationCoordinator. - The warning messages sent to vehicle A contains lane merging indications at the icon displayed onboard, if accurate positioning of the vehicles and a detailed LDM (lane level positioning) are available, Test sites: - from rsu : West/ A86 (Navteq) SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 42 of 113 COSSIB

43 Requirements: - same as H&IW_01 Test Site : Test Case 1: Basic performance test for H&IW_01 (Obstacle) #1: Accident/Vehicle as obstacle Use cases: - UC : SP5_UC16_v1.0: Deviations for road-works Expected Values / Results The requirements are met, specifically: - The VANET component sends the RsuHMImessage and the vehicle displays the text / icon - In test 1 the warning sequence is: Comfort, Safety, Emergency - In Test 2 the H&IW stays in the Comfort situation. Those test should enable to measure the shift, in term of time, between the specification and the implementation/deployment. See the expected values of H&IW_01 Test Case 1 for more details. Obtained Values / Results The LIVIC track segment used for validation test is shown in the following pictures : SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 43 of 113 COSSIB

44 Hereafter are shown the matching between the application performances and the original specifications. H&IW functions requirements SP5_RQ02_36_v1.0 Message Management Identify Safety Critical Situations SP5_RQ05_36_v1.0 SP5_RQ06_19_v1.0 / Priority Level of Message SP5_RQ07_19_v1.0 / Validity of Messages SP5_RQ09_19_v1.0 Message Management 2 The system shall be able to decide if and to whom a message has to be send. The system shall be able to identify safety critical situations at the surrounding of a critical point. The system shall be able to provide a priority level for each transmitted message. The system shall be able to determine the period of validity of the transmitted and received messages The system must not send warnings concerning severe dangerous situations (that might cause Not Tested SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 44 of 113 COSSIB

45 Data exchange: SP5_RQ12_33_v1.0 SP5_RQ16_36_v1.0 Query from the LDM SP5_RQ17_36_v1.0 Receive data from the LDM drivers to extreme driving actions), which does in fact not happen The system shall be able to transmit information towards several supporters (e.g. VMS, radio, PND) The system shall be able to query needed data from the LDM. The system shall be able to receive and handle the LDM data. LDM requirements SP5_RQ49_19_v1.0 / Position and speed of vehicles SP5_RQ68_10_v1.0 / Road status: SP5_RQ65_10_v1.0 / Obstacle description The RSU subsystem shall be able to receive at minimum the current vehicle position and speed. The system shall acquire data on the weather conditions in the 'critical area' (extent TBD) of the obstacle or accidentnt. The system shall be able to provide a priority level for each transmitted message. The system shall provide other details of the obstacle where possible : type of object (accident, queue, rocks, dropped load, etc), lanes affected, speed (for moving object), precise location etc. i.e. wet/dry road surface, rain, fog, ice, which may affect stopping distances and visibility. Not Tested Timing and Performance SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 45 of 113 COSSIB

46 SP5_RQ18_19_v1.0 Simultaneity The system shall be able to manage all vehicles in the vicinity of a critical point simultaneously. Not Tested SP5_RQ20_19_v1.0 Time of Warning Generation The system shall generate and transmit warning information to the drivers timely enough, so that the reaction time left to the drivers is appropriate. / Partially tested SP5_RQ22_10_v1.0 R/S Warning SP5_RQ35_19_v1.0 Time of Data Delay 2: The system shall provide a roadside warning in fewer than 2 sec (TBC) in the critical zones to permit vehicles to reduce speed and/or change lane in order to avoid a (secondary) collision. In the roadside sub-system the delay between generation of warnings and transmission of this data to the vehicles shall be smaller than 50 ms. Partially It mainly depends on the load of the LDM. When the number of vehicles that must be written and updated in the LDM is small, the load of the LDM is low and the access delay has the same behaviour. For a high number of vehicles, the update mechanism of the beacon message in the LDM may block the Q-API, decreasing the performance of the application. Message Exchange and Filtering SP5_RQ34_19_v1.0 / The roadside sub-system must be able to address Directly address a Vehicle data directly to one vehicle distinctively (or a group of vehicles) or to broadcast to all vehicles in the vicinity of the RSU. SP5_RQ50_36_v1.0 / The system shall be able to transmit the warning / Transmission of warnings: recommendation to equipped vehicles SP5_RQ69_10_v1.0: The system shall send appropriate warnings to SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 46 of 113 COSSIB

47 Warning: equipped vehicles within the critical (e and a) zones to permit them to reduce speed and/or change lane and avoid a (secondary) collision. SP5_RQ37_19_v1.0 Filter for relevant data: The vehicle subsystem should be able to filter the relevant information to the driver. SP5_RQ51_19_v1.0 In vehicle assistance HMI shall present warnings HMI to the driver in an intelligent way without distracting him. Example: Use of different actuators to smooth signalize hazards. SP5_RQ48_36_v1.0 The system shall receive data from environmental Environmental Data: sensors about weather (rain, snow, fog...). SP5_RQ52_36_v1.0 / Static The system shall receive static vehicle data like Vehicle Data width, length, type of vehicle, mass SP5_RQ60_36_v1.0 The system shall receive information about the Lack of Friction lack of friction of a road segment SP5_RQ61_36_v1.0 / Data The system must be able to receive data from the Reception: vehicles as well as the infrastructure. Partially Tested Not Tested Not Tested Partially Tested Not Tested SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 47 of 113 COSSIB

48 Different contributions to the total loop time Vehicle Speed Loop time (From VANET log) Loop error in distance Expected distance to obstacle when warning is displayed Measured distance to obstacle when warning is displayed 50 km/h av.: 333ms [312, 353] ms 4.07m [3.81, 4.31] m 55 m 20 m 90 km/h av.: 522ms [458, 707] ms 12.83m [10.75, 17.86] m 120 m 80 m 120 km/h av.: 441ms [240, 751] ms 14.89m [8.09, 25.52] m 150 m 100 m Major limitation faced during the validation tests was the VANET range which never exceeded 300 meters Status Test Case2 Link to external annexed documentation (if any) SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 48 of 113 COSSIB

49 Table 11 H&IW_01 Test site 2 TEST FORM SP 5 WP 5 Prototype Version - System Configuration - 12/2009 (test case 1) 08/ /2010 (test HW Release - SW Release - Compiled by / Company MIZAR Date case 2) 12/2009, 03/2010 (test case 3) Form Progressive Numb. 10b Functional Component H&IW_01 Reference Document D5.3.5 Test Type Test Objective Test Purpose Test Environment Integration Verification Correctness Test Site Test Case 1: Test #1: Obstacle (stopped vehicle) CRF- WSN Test Description Prerequisite: - The successfully results of all the previous tests - Synchronization of the clock and Positioning have to be done - Base Board with a Local Detection Algorithm installed (FW_LDA) - Main PC (ECW-281B-ATOM) - SP2 applications(data Fusion Module) - SP5 applications installed inside Main PC - Vanet Router dedicated gateway PC connected to main PC - 2 SF FIAT (CRF) called Vehicle A and Vehicle B Functional Description This test aims to verify the correctness of the detection of an obstacle and the communication of a warning message to approaching SAFESPOT vehicles. The components will communicate according to the following sequence: 1. Two SF-equipped vehicles drive around the test track 2. One SF-vehicle stops by the roadside 3. Both vehicles are beaconing and the messages are received by the RSU. 4. The RSU detects the stopped vehicle by reading its position inside beacon messages. 5. The communication to the Data Fusion is done and the event is written in the LDM SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 49 of 113 COSSIB

50 6. The trigger generates the notification that an obstacle is detected 7. The Black Spot Monitor receive this message 8. The SP5 application analyse the risk of the event and takes the decision of to send the message to the router Vanet (SP5_MessageManager) 9. The second vehicle (B) passes the area of risk. 10. The vehicle is able to see the message with the External Message Applicator as soon as it entered the area of area (Risk area 170 m X 8m) and switch off automatically when it left the area Stopped vehicle Geovalidity Area Test site: CRF Related Test: Netherlands A16 TeleAtlas France - West/A86 - Navteq Requirements: - SP5_RQ02_36_v1.0: The system shall be able to decide if and to whom a message has to be send. - SP5_RQ07_19_v1.0: The system shall be able to determine the period of validity of the transmitted and received messages. - SP5_RQ16_36_v1.0: The system shall be able to query needed data from the LDM. - SP5_RQ17_36_v1.0: The system shall be able to receive and handle the LDM data. SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 50 of 113 COSSIB

51 Use cases: - SP5_UC13 (Accident as Obstacle) - SP5_UC14 (Traffic jams as obstacles (slow moving vehicles)) - SP5_UC15 (Traffic jams as an obstacle results of an accident, with poor visibility) Expected Values / Results The requirements are met, specifically: - The WSN detects the obstacle and its position - Communication between the components Obtained Values / Results This test was not carried out as the original planning, because the Wireless Sensor Network alone are not able to detect the presence of a stopped vehicle with its specific position. It was decided to use this opportunity to test the possibility of adopting vehicle beaconing for the same purpose. The test of the detection of an obstacle was carried out in the Test Case 2: See Test#2 considering an obstacle as a traffic jam using the ECAID system. IIn this test an stopped vehicle was used, the detection was made using the beaconing data from the SF-vehicle. The area of the validity of the messages was enough to alert the incoming vehicles about the risk (stopped vehicle). The test considered different speeds from 50km/h until 80 Km/h. The duration of the messages were between 10 sec until 6.8 sec (depending of speed) Results: The communication between infrastructure and Vehicle was successful. The vehicle received the messages inside the area defined by the geovalidtity No messages out of the geovalidity area were received by the vehicle. The area of the validity of the messages was enough to alert the incomming vehicles about the risk (stopped vehicle). This test was repeated ten times. On each occasion, as shown by the RSU data log, the SP2 Platform was able to detected a stationary vehicle (vehicle speed = zero and position fixed) and the event was written in the LDM. A trigger produced notification that an obstacle had been detected, and the H&IW_01 then generated a message using the SP5_MessageManager. It was observed that the message appeared on the HMI of the second vehicle each time it entered the area of risk and then switched off when it left. The duration of the messages were between 6.8 and 10 sec, depending on the speed at which it was travelling. It worked for all speeds up to 80 km/h Status Test Case1 Link to external annexed documentation (if any) Test Case 2: Test #2: Obstacle (traffic jam & slow vehicle ) Brescia Padova-WSN-ECAID Test Description Prerequisite - Successful results of all the test before - The system ECAID has the communication with the LDM - Installation of the WSN system in Brescia-Padova( test area) SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 51 of 113 COSSIB

52 o 10 Sensors (AMR and Pyro) o Base Board with a Local Detection Algorithm installed (FW_LDA) - Main PC (ECW-281B-ATOM) - SP2 applications(data Fusion) - SP5 applications installed inside Main PC - Vanet Router dedicated gateway PC connected to main PC - 2 FIAT BRAVO (CRF) called Vehicle A and Vehicle B (with Vanet) Description: The components will communicate in the following sequence: 1. The vehicles are driving in a normal way 2. A traffic event is detected by ECAID (analysing the information of mini-tdc) using the data from COMPANION sensors, WSN and radars 3. The communication to the Data Fusion is done and the event is written in the LDM 4. The trigger generates the notification that an obstacle has been detected 5. The SP5 application analyses the risk of the event and takes the decision to send the appropriate message to the router Vanet (SP5_MessageManager (SP5 interface with Vanet) 6. The second vehicle (B) passes the area of risk (using the simulation of a traffic jam) 7. The vehicle is able to see the message with the visualization tool Objectives of the TEST: The system will be tested also considering traffic jams as an obstacle. In this case the test will be performed in the following sub-test: 1. Performance of WSN prototype on a 3 lane motorway (data output and writing on LDM). 2. For Brescia-Padova: integration with (TDC and ECAID) i.e. other sensor data 3. Identification of traffic patterns 4. Detection of event (traffic jam & slow vehicle, etc) 5. Send of the HMI messages Test site - Brescia- Padova - Pila (Aosta) - Chivasso (Lab) BS-PD Installation of the sensors began at 07/2009. The sensors couldn t work properly for the high traffic conditions of the place. After different analysis of the problems with the sensors, two important upgrades to network were developed. A first update to the network was run on 12/04/2010 developed by ISMB and MIZAR. The update consisted in modification of the communication protocol and the way of manage memory inside the sensor, these changes allow to improve the number of vehicles detected by the WSN. For other hand the network was unstable and new changes were made on it A second updated run was on 21/04/2010 developed by ISMB and MIZAR. SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 52 of 113 COSSIB

53 Pila (Aosta) This test was carried out in the highway of Pila, the sensors were installed for one month (November 2009 December 2009). This test allows to test the sensors in a real environment Chivasso A mini-network of the WSN was installed in the laboratory using 10 sensors + gateway. The aim of this test was re-create the conditions of hardware and software as in BS-Pd in order to solve and analyze the possible issues found at BS_PD. Using this network it was possible to improve the limitations of the system installed in BS-PD Related test sites - Nether-lands A16 TeleAtlas - France - West/A86 - Navteq Requirements: - SP5_RQ02_36_v1.0: The system shall be able to decide if and to whom a message has to be send. - SP5_RQ07_19_v1.0: The system shall be able to determine the period of validity of the transmitted and received messages. - SP5_RQ16_36_v1.0: The system shall be able to query needed data from the LDM. - SP5_RQ17_36_v1.0: The system shall be able to receive and handle the LDM data. Use cases: - SP5_UC13 (Accident as Obstacle) - SP5_UC14 (Traffic jams as obstacles (slow moving vehicles)) - SP5_UC15 (Traffic jams as an obstacle results of an accident, with poor visibility) Expected Values / Results The requirements are met, specifically: - Interaction between components - Detection of the obstacle and its position - I-V communication - By comparing the detection of the WSN and ECAID systems, it is expected to obtain more precision in the results with the WSN system. Obtained Values / Results Results: The sensors support extreme conditions, were installed At the beginning the sensor WSN were not able to work under real time traffic conditions. After modification the WSN were able to manage intense conditions of traffic. The first limitation was the high number or vehicles, this issue was solved increasing the time of request from gateway to the sensors from 5 sec to 2 sec. Also an improvement in the nformation transmited was applied, so the new configuration gives high priority to the information regarding speed and number of vehicles and low priority to the extra- information like (type SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 53 of 113 COSSIB

54 of vehicle, direction, etc). This approach allows to have a system that worked under real traffic conditions. The max number of vehicles detected by each sensor was 15 vehicles each 2 seconds The communication between all the components was successful and the V-I and I-V communication was successful.. Experiments carried out with direct RSU-vehicle communications demonstrated correct receipt of messages at approx 570m and with multi-hop communication via a stationary vehicle (acting as a safety vehicle) the distance increased to 640m. This latter can help to increase the safety coverage provided by the system when there are obstacles to direct communication of warning messages due the physical configuration of the ground or other obstacles. This test was repeated ten times. On each occasion, as shown by the RSU data log, the SP2 Platform was able to detected a stationary vehicle (vehicle speed = zero and position fixed) and the event was written in the LDM. A trigger produced notification that an obstacle had been detected, and the H&IW_01 then generated a message using the SP5_MessageManager. It was observed that the message appeared on the HMI of the second vehicle each time it entered the area of risk and then switched off when it left. The duration of the messages were between 6.8 and 10 sec, depending on the speed at which it was travelling. It worked for all speeds up to 80 km/h Status Test Case2 Link to external annexed documentation (if any) Test Case 3: Test #1: Obstacle-Pedestrian on the road-crf-test Track- Thermal imaging cameras (provided by VTT) Test Description Prerequisite: - Successful results of the previous tests - Synchronization between the clock and Positioning - Installation of the Thermal imaging cameras (provided by VTT) ( test area) - Main PC (ECW-281B-ATOM) - SP2 applications (DataReceiver, Map Matching, Object Refinement) - SP5 applications installed inside Main PC - Vanet Router dedicated gateway PC connected to main PC - 2 FIAT BRAVO (CRF) called Vehicle A (with Vanet) The components will communicate according the next two stages: Method First stage: Firstly detection system was firstly calibrated to the local test site in accordance with the detailed specifications prepared by VTT. The camera was installed on a metal support in a position where it could record the scene. Within the camera range, a person crossed the road several times from one side to another. When the camera detected an object and recognized the form as a pedestrian with a confidence parameter of over >40%, the detection was considered reliable and a broadcast message was generated. It then sent to the RSU a UDP message with the detection data (broadcast message). Second stage: In the second stage, this message was simulated to test the ability of the H&IW application to recognise the message, trigger an alert and send a message to a SAFESPOT equipped SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 54 of 113 COSSIB

55 vehicle. An observer in a moving vehicle recorded whether and when the message was received. It was also possible to verify receipt of the message with the visualization tool (Esposytor). The test was repeated 50 times over a period of four hours. Test site: CRF Requirements: - SP5_RQ02_36_v1.0: The system shall be able to decide if and to whom a message has to be send. - SP5_RQ07_19_v1.0: The system shall be able to determine the period of validity of the transmitted and received messages. - SP5_RQ16_36_v1.0: The system shall be able to query needed data from the LDM. - SP5_RQ17_36_v1.0: The system shall be able to receive and handle the LDM data. Use cases: SP5_UC17 (Pedestrian on Motorway) Expected Values / Results The requirements are met, specifically: - Components provide the correct information - Messages are correctly display in vehicles - Communication between all the components Obtained Values / Results Results - When the UDP message was written in the LDM traffic event table: event = 30 (pedestrian), this triggered a message to the BlackSpotMonitor that an event had been found. The H&IW_01 generated and sent a message which was activated the display of the pedestrian icon on the vehicle HMI. - In the first stage, some defects were revealed in the detection system. When a person crossed the road, the system usually registered the presence of an object, but did not always make a correct classification. - Some problems were encountered initially with the writing of the message in the LDM, necessitating a reset of the parameters. When these were adjusted, the trigger was successfully generated and the warning message received by the vehicle. Conclusions - With respect to the H&IW_01 functions, after some resets of the system had been carried out, the communication of the message from the infrastructure (RSU) to an equipped vehicle was successful with the display of the warning (pedestrian icon). SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 55 of 113 COSSIB

56 Status Test Case3 Link to external annexed documentation (if any) Table 24 Obstacle-Pedestrian on the CRF Test Track Thermal imaging cameras SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 56 of 113 COSSIB

57 Table 12 H&IW_02 Test Site TEST FORM SP 5 WP 5 Prototype Version - System Configuration - HW Release - SW Release - Compiled by / Company MIZAR Date 06/2009 Form Progressive Numb. 11 Functional Component H&IW_01 Reference Document D5.3.5 Test Type Test Objective Test Purpose Test Environment Integration Verification Correctness Test Site Test Case 1: Test #1: Ghost Driver in CRF using RFID sensors. Communication Test Test Description Prerequisite: - The successful results of all the test before - Synchronization of the clock and Positioning - Installation of the RFID system in CRF test area o RFID readers (PCMCIA) o RFID antennas (both Omni directional and directed) o RIFD tags (active tags) - Main PC (ECW-281B-ATOM) - SP5 applications installed inside Main PC - SP2 applications(datareceiver, RFID application, Map Matching, Object Refinement) - Vanet Router dedicated gateway PC connected to main PC - 2 FIAT BRAVO (CRF) called Vehicle A (with Vanet) and Vehicle B (ghost driver) Description: The test will check the correct communication between all the components using the detection system RFID. The components will communicate in the following sequence: 1. Vehicle A drives in the correct way at the area of test 2. Vehicle B drives in the wrong way at the area of test 3. The RFID system (block of antenna, reader and elaboration system) detects the presence of a ghost driver. By means of software used for ghost driver detection correct messages are produced and send to the main PC 4. The message arrive at the DataFusion Block and it is saved inside the traffic event [trafficevent.eventcause = 35 (ghost driver)] 5. The trigger detects the event inside LDM and communicates with the Black Spot Monitor 6. SP5 applications evaluates the Risk and generates a corresponding message 7. The message is received by the Vanet router SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 57 of 113 COSSIB

58 8. The Vanet router sends this message to the Vanet of Vehicle A 9. The Driver will see the message by means of the visualization tool (e.g. Esposytor) that shows the presence of a ghost driver in a x, y position. 10. Driver B avoids and accident by slowing down. Antenna RFID and RFID tag Antenna RFID omnidirectional Ghost driver: two cars in opposite direction Wrong Direction Correct Direction Shadowing 2 Ghost Drivers (2 ghost drivers traveling in Components communicated in this sequence: 1. Each Vehicle have one RFID tag 2. The vehicles moves at 50km/h. Vehicle A follows the correct direction and vehicle B the wrong direction 3. When the vehicle A passes out of the range of the antennas, no message is generated. 4. The vehicle B enters in the range of antenna B, the PC connected to antenna B, receives the information and sends the message by Wireless Network WiFi to computer A. The time difference between sensing by antenna A and B is set by a time window. Otherwise false alarms would be generated if a car later travels in the opposite direction. The time window has to be adjusted considering the speed of traffic flow at the particular road segment. During the tests, the time window was 20 seconds. 5. Then vehicle B passes antenna A, the Pc connected to antenna A receives the information and the algorithm of ghost driver detects the (by analysing the passing orders) ghost driver 6. The message is sent to the DataReceiver, and it is saved in table motorvehicle and traffic event of LDM. 7. The RSU sent the HMI message to the vehicle vehicles. 8. The message is displayed inside the vehicle. Components communicated in this sequence: SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 58 of 113 COSSIB

59 parallel lanes next to each other) Wrong Direction The test pretends to check if there is a shadowing effect in the case that two ghost drivers are present in the track at the same time, but in different lanes. 1. Each Vehicle have one RFID tag 2. The vehicles A moves at 30km/h in the lane 1 and at the same time the vehicle B moves at 30km/h in the lane 2 (parallel lines). 3. The two vehicles pass (at the same time) in the range of antenna B, the PC connected to antenna B receives the information and sends the 2 messages by Wireless Network to computer A. 4. Then the 2 vehicles (at the same time) pass antenna A, the Pc connected to antenna A receives the information and the algorithm of ghost driver detects the (by analyzing the passing orders) ghost driver 5. The messages of ghost driver detection are sent to the DataReceiver, and it is saved in table motorvehicle and traffic event of LDM. 6. The RSU sends the HMI message to the vehicle vehicles 7. The message is displayed inside the vehicle. This test was developed in different sessions: BME arrives to Italy in May 2009, for one day, the test were developed in different sessions one with all the equipment of BME for develop the test of detection, and the rest of the test were developed using the replay of the UDP messages recorded at that time Test site: CRF: Centro di Ricerca FIAT test area Related Test sites: Italian Test Requirements: - SP5_RQ02_36_v1.0: The system shall be able to decide if and to whom a message has to be send. - SP5_RQ07_19_v1.0: The system shall be able to determine the period of validity of the transmitted and received messages. - SP5_RQ16_36_v1.0: The system shall be able to query needed data from the LDM. - SP5_RQ17_36_v1.0: The system shall be able to receive and handle the LDM data. Use cases: SP5_UC34 (Ghost Driver) Expected Values / Results The requirements are met, specifically: The expected results will be that two tables are verified: - Verification Table 1 will have the Messages sent for each component described above Expected Send message, Real Send message, Component - Verification Table 2 will have the Message receive for each component described above Expected Receive Message Real Receive Message. Component The test will be considered as successful if the group of Send and Receive Messages corresponds exactly to the Expected Send and Receive Messages Obtained Values / Results Comments: 1. The Message sent GhostDriverFrom RFID Sensor was correctly received by DataReceiver in the RSU with the correct format described in SF_SP7_Data_formatmessages_v SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 59 of 113 COSSIB

60 (3).doc 2. In this case the message value for ghostdriver = 1 implies that the Data Receiver writes inside the table traffic event the cause of this event (35 for Ghost Driver) and also the message GhostDriverFrom RFID Sensor 3. The HMI messages send to the vehicle were correctly received by the vehicle, the messages were displayed correctly inside the vehicle Conclusions The result of the test satisfied the requirements of: a. Correct communication between elements b. Correct detection of the event Ghost Driver c. Correct communication with datareceiver and LDM d. Correct detection of the traffic event e. Correct construction of the HMImessage f. Correct reception of the messages inside the vehicle by the External Message Applicator. The time difference between sensing by antenna A and B is set by a time window. This time window should be adjusted considering the speed of traffic flow at the particular road segment. During the tests, the time window was 20 seconds. The verification table one and two showed that the messages expected by each components were correctly interpreted Shadowing effect might be critical at high speed Using low speeds, aproximately km/h the test in CRF was successful, for high speeds new test should be conducted and a calibration of the time window is needed. The whole system is still now a prototype. it was decided that the system need to be improved in order to have a new small version of the system to be carried to the highway Status Test Case1 Link to external annexed documentation (if any) Test Case 2: Test#2 Ghost Driver on CRF test track-using WSN Test Description Prerequisite: - The successful results of all previous tests - Synchronization of the clock and Positioning - Installation of the WSN system in Torino Caselle (test area) o 3 Sensors (AMR and Pyro) o Base Board with a Local Detection Algorithm installed (FW_LDA) - Main PC (ECW-281B-ATOM) - SP2 applications (DataReceiver, RFID application, Map Matching, Object Refinement) - SP5 applications installed inside Main PC - Vanet Router dedicated gateway PC connected to main PC - 2 FIAT BRAVO (CRF) called Vehicle A and Vehicle B (with Vanet) Table 22 Verification Table RFID Ghost driver two cars in opposite direction Table 23 Verification Table RFID Shadowing 2 Ghost Drivers SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 60 of 113 COSSIB

61 Extend description SF vehicle CRF Hardware - Application PC: TF-AEC-6830-A1 - Main PC: ebox638 FL - Positioning PC: AAEON AEC GPS 1: Ublox (only for synchronization) - GPS 2 Novatel (for DGPS). Radio system to receive the correction from the base station Software - FW à OR à SR à DAQ à LDM à API à Positioning software à V Router à Application sw à 2.0 Description: This test will focus on the verification of the correct detection of a wrong way driver and the correct creation of warning for the incoming vehicles. communication between all components using the detection system of WSN. Giving the fact that is not possible to simulate a ghost driver in real traffic the test was developed under controlled conditions in CRF.In this test, a broadcast message will be send. The two vehicles provided with Vanet will receive a message from the Vanet router about the detected ghost drivers. The components will communicate in this following sequence: 1. The vehicles are driving in the legal direction 2. The sensors detects that there are ghost drivers and communicate this fact to the Main RSU PC. 3. The message arrives to the DataFusion Block and it is saved inside the traffic event [trafficevent.eventcause = 35 (ghost driver)] 4. The trigger detects the event inside the LDM and communicates with the Black Spot Monitor 5. The SP5 application evaluates the Risk and generates a corresponding message. SP5_MessageManager (SP5 interface with Vanet)) 6. Vehicle A enters the ghost driver area 7. The traffic lights turns on in order to alerts the Vehicle A (Wrong Way Driver) 8. The Driver of vehicle B will sees the message by means of the visualization tool (External Message App). 9. Additional, the driver will be alerted that there are other ghost drivers in the surrounding. Test site: CRF SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 61 of 113 COSSIB

62 Traffic lights Vanet + Positioning PC +GPS +-Antenna Test setting at the Orbassano test track The 3 Sensors were distributed in the next CRF Test Track, with a distance between them of 15 meters. Then the distance between the RSU and the Gateway was the 45 meters. Also the Vanet antenna was installed in a post (3,5 meters). The original set up is described in the Figure 2. After some test the Gateway of the RSU was located in a in a straight line, in order to improve the communication between all the sensors and use the barrier of the test Track. See Figure 2. For this scenario the warning used where HMI messages, and traffic lights signaling. When the Wrong way driver was detected the trafic lights turn on and the vehicle passing by the area of risk received the message during the time that pass through the area of the geovalidity. The test considered two lanes where the wrong way driver can pass in front of the sensors. For more details see Annex: Results Wrong Way Driver CRF. Requirements: - SP5_RQ02_36_v1.0: The system shall be able to decide if and to whom a message has to be send. - SP5_RQ07_19_v1.0: The system shall be able to determine the period of validity of the transmitted and received messages. - SP5_RQ16_36_v1.0: The system shall be able to query needed data from the LDM. - SP5_RQ17_36_v1.0: The system shall be able to receive and handle the LDM data. Use cases: - SP5_UC34 Expected Values / Results The requirements are met, specifically: - All Real Sent and Received Messages correspond to the Expected Messages. - All components should communicate in a correct way. SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 62 of 113 COSSIB

63 All the use cases works correctly Obtained Values / Results For the case of the period of validity of the messages the system was able to send the messages with a correct period of time, and inside the area of the geovalidity. After some tries the traffic event were not saved in the LDM, the reason was the static table in the LDM for the sensors. At the beginning the communication didn t works because this table was empty, and the SP2 and SP5 framework needed this information in order to work properly. The problem was solved after inserted the correct information about sensors (positioning and d). The quality of the detection depends of the position of the sensors. The angle of inclination of the sensors change the results of the detections. Changing the angle of inclination the detections change dramatically. Using the (See Table Result Wrong Way Driver) the first system of detection was possible to see that some detections were lost when the vehicle passes by the second lane. After change the algorithm of detection inside the sensors, increase the perception of the vehicles that passes by the second lane as well as the time of detection was improved.. Messages to Esposytor were correctly sent and received. All requirements were fulfilled Status Test Case2 Link to external annexed documentation (if any) Table 25 Results Wrong Way Driver CRF Correctness Table 26 Wrong Way Driver CRF Communication Test SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 63 of 113 COSSIB

64 Table 13 H&IW_02 H&IW_03 Test Site TEST FORM SP 5 WP 5 Prototype Version - System Configuration - HW Release - SW Release - Compiled by / Company MIZAR Date th March 2010 Form Progressive Numb. 12b Functional Component H&IW03 Reference Document D5.3.5 Test Type Test Objective Test Purpose Test Environment Multiple Integration Verification Correctness Test Site Test Case 1: H&IW_02 and H&IW03: Combined Wrong Way Driver and Slippery Road Amsterdam Test Description Prerequisite: - The successful results of all previous tests - Synchronization of the clock and Positioning - Installation of the WSN system in Torino Caselle (test area) o 3 Sensors (AMR and Pyro) o Base Board with a Local Detection Algorithm installed (FW_LDA) - Main PC (ECW-281B-ATOM) - SP2 applications - SP5 applications installed inside Main PC - Vanet Router dedicated gateway PC connected to main PC - 1 SF BMW and INCA vehicle 1 Mercedes S Class - 75 m of barrier Description: The Test combines two applications Wrong Way Driver and Slippery Road. The first objective is to identify two event traffics: ghost drivers as well as slippery road. The second objective is test the correctness of the H&IW applications, which should send the corresponding warnings to the incoming drivers. Notice that the Slippery Road is an application originally part of SP4 applications. For this test was decided to use the opportunity of the DEMO of Amsterdam Show case to test the sub H&IW_03 application for Abnormal road Condition using not only the V-V communication of this application but also the V-I communication. SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 64 of 113 COSSIB

65 Wrong Way Driver Method 1. The SF-vehicles (INCA or Daimler) are going around the track 2. The vehicle BMW enters the slip road from the wrong entry See Figure diagram of the Amsterdam Test Track 3. Using one of the SP2 sensing technologies (Wireless Sensor Network) and data fusion module, the SAFESPOT system detects the vehicle travelling in the wrong direction 4. The H&IW application then generates a warning message which is sent to SF-equipped vehicles in the vicinity 5. The system also activates signs on the roadside to warn all approaching vehicles (including non SAFESPOT vehicles) that a vehicle is travelling the wrong way 6. The system switches on red traffic lights to warn the wrong way driver to stop. Slippery Road Method 1. A slippery area was created with a special sheet that was sprayed constantly with water to make the surface slippery 2. The BMW vehicle detects the slippery area using the sensors installed on it 3. The BMW generates the warning messages and send them through VANET to the incomming vehicles (INCA and S-Class) 4. The INCA vehicle display the message to the driver using a monitor and the driver feels the haptic feedback (accelerator pedal and driver seat). 5. The BMW generates the warning messages and send them through VANET to the Infrastructure (RSU) 6. The RSU analyse the message received by the BMW and turn on the VMS. Road Side Warning messages used for the Test Test site: Amsterdam Slipperty road Application VMS warning for Slippery road Wrong Way Driver Application VMS warning for drivers going in the correct direction Traffic lights for warning wrong way driver, Wireless Sensor Network and Guard-Rail SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 65 of 113 COSSIB

66 Figure 1 Diagram of the Amsterdam Test Track Legend Wireless sensor nodes VMS panel BMW vehicle INCA vehicle / DAIMLER vehicle Wrong way direction Correct direction Traffic light Lane lights SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 66 of 113 COSSIB

67 Requirements: - SP5_RQ02_36_v1.0: The system shall be able to decide if and to whom a message has to be send. - SP5_RQ07_19_v1.0: The system shall be able to determine the period of validity of the transmitted and received messages. - SP5_RQ16_36_v1.0: The system shall be able to query needed data from the LDM. - SP5_RQ17_36_v1.0: The system shall be able to receive and handle the LDM data. Use cases: - SP5_UC34 Expected Values / Results The requirements are met, specifically: All Real Sent and Received Messages correspond to the Expected Messages as indicated in tables. Obtained Values / Results Recommendations Memory in Vanet. All the logs should be canceled each day Issues faced PC positioning (laptop) was changed Logs in RSU should be canceled Disable of Logs was done Major issue there was indeed the time synchronization of the different systems causing the framework to disregard information send by the other vehicle. A system restart in the vehicles including a new time synchronization was necessary after about 2.5 to 3 hours of testing. Results The 85 % of the cases the whole system works well during the 5 days of DEMO. During the test demo an exception coming from the SP5 application was found. This exception occurred after some hours, around after 6 hours of work. The error implied the generation of HMI messages without geovalidity information, this effect was solved after the Amsterdam Demo (going back to Italy). The problem was solved after an update of the system. The reason was a lack of memory in the application PC, and a modification in the software SP5 was implemented (a garbage collector system). A second exception was identified, in the software of the gateway (Road Side), this exception was very rarely the reason for this effect was related with the generation of logs and the amount of memory used. An update to the gateway was installed in the gateway on 25 May after this update the problem was solved. This test demonstrate the correct definition of the geovalidity area and the correct interpretation of the External Messages application of both vehicles (INCA and Daimler) Beyond the small issues the system demonstrated to work 5 continuously days, with running applications on Infrastructure and Vehicles Total: 500 repetitions of the test Status Test Case1 Link to external annexed documentation (if any) Figure 4 Technical Test Result Amsterdam wrong way driver (Infrastructure analysis) SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 67 of 113 COSSIB

68 Table 14 Rdep SDM creator Simulation TEST FORM SP 5 WP 5 Prototype Version RDep_Beta System Configuration RDep_Sim HW Release 1 SW Release 1 Compiled by / Company Andre Possani, DIBE Date September 2009 Form Progressive Numb. 13 Functional Component RDep learning SDM DB LDM + player Reference Document SF_D5.3.4_Specifications_ for_rdep_v3.4 Test Type Test Objective Test Purpose Test Environment Unitary Verification Correctness Simulation Test Case 1: Safe Drive Map creator (learning phase) for a Deviation of Road Work Test Description Description: The Road Departure application relies on the Safe Drive Map (SDM), which is a Data Base filled with information from vehicles passing through a black spot (Deviation for Road Work). Creating the SDM is the offline part of the Road Departure application. The LDM data player (MARS) is used as input (provides vehicle information) and the output is stored in the Safe Drive Map. Input LDM data player LDM Road Departure SDM Creation Output Safe Drive Map The following procedure are done at least 20 times: 1. In a two lane straight road, the vehicle drives in the left lane with a speed of 15m/s ± 5m/s. 2. As soon as the driver can see that a Road Work deviation is present ahead obstructing the left lane, it should lower its speed and do the correct maneuver while changing to the right lane. The RDep_SDM_Creator program should monitor and record the vehicle s trajectory. SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 68 of 113 COSSIB

69 After having all the sample trajectories, the RDep_SDM_Creator will analyze the obtained trajectories and will calculate the values for the Safe Drive Map. Tools: - LDM data player (MARS) Requirements: - SP5_RQ16_36_v1.0 The system shall be able to query needed data from the LDM Expected Values / Results The requirements are me, specifically: 1. The RDep_SDM_Creator is able to query the LDM for all the static and dynamic information of the vehicles present in the coverage area. 2. The RDep_SDM_Creator is able to store the trajectories of the vehicles and analyze them. 3. The values for the Safe Drive Map are calculated by the RDep_SDM_Creator. Obtained Values / Results SP5_RQ16_36_v1.0 The system shall be able to query needed data from the LDM The RDep application is able to store the trajectories of the vehicles and analyze them. The values for the Safe Drive Map are calculated by the RDep application Status Test Case1 Link to external annexed documentation (if any) - Test Case 2: Safe Drive Map creator (learning phase) for a Dangerous Curve Test Description Description: The Road Departure application relies on the Safe Drive Map (SDM), which is a Data Base filled with information from vehicles passing through a black spot (Dangerous Curve). Creating the SDM is the offline part of the Road Departure application. The LDM data player (MARS) is used as input (provides vehicle information) and the output is stored in the Safe Drive Map. Input LDM data player LDM Road Departure SDM Creation Output Safe Drive Map The following procedure should be done at least 20 times: 1. While approaching to the dangerous bend, the vehicle enters the dangerous curve keeping the average speed of 15m/s ± 5m/s; 2. The vehicle should drive through the curve in a comfortable situation and should never leave its lane. The RDep_SDM_Creator program should monitor and record the vehicle s trajectory. SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 69 of 113 COSSIB

70 After having all the sample trajectories, the RDep_SDM_Creator will analyze the obtained trajectories and will calculate the values for the Safe Drive Map. Tools: - LDM data player (MARS) Requirements: - SP5_RQ16_36_v1.0 The system shall be able to query needed data from the LDM Expected Values / Results The requirements are met, specifically: 1. The RDep_SDM_Creator is able to query the LDM for all the static and dynamic information of the vehicles present in the coverage area. 2. The RDep_SDM_Creator is able to store the trajectories of the vehicles and analyze them. 3. The values for the Safe Drive Map is calculated by the RDep_SDM_Creator. Obtained Values / Results SP5_RQ16_36_v1.0 The system shall be able to query needed data from the LDM The RDep application is able to store the trajectories of the vehicles and analyze them. The values for the Safe Drive Map are calculated by the RDep application Status Test Case2 Link to external annexed documentation (if any) - SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 70 of 113 COSSIB

71 Table 15 Rdep Runtime Simulation TEST FORM SP 5 WP 5 Prototype Version RDep_Alfa System Configuration RDep_Sim HW Release 1 SW Release 1 Compiled by / Company Andre Possani, DIBE Date September 2009 Form Progressive Numb. 14 Functional Component RDep_runtime SDM DB LDM HMI_simulator ShemServer Reference Document SF_D5.3.4_Specifications_ for_rdep_v3.4 Test Type Test Objective Test Purpose Test Environment Unitary Verification Correctness Simulation Test Case 1: Road Departure Runtime application for a Deviation of Road Work Test Description Description: Whenever a vehicle travel along the Road Work deviation, the RSU monitors the vehicle s trajectory and Road Departure Runtime application predicts the near future trajectory and compares it with the SDM information. Depending on the risk analysis of the application, a Comfort, Safety or Critical warning is issued and sent to the HMI through the Message Manager. Input LDM data player LDM Road Departure Safe Drive Map Output Simulated HMI The condition of "No Warning" is summarized by the following steps: 1. In a two lane straight road, the vehicle drives in the left lane with a speed of 15m/s ± 5m/s. 2a As soon as the driver can see that a Road Work deviation is present ahead obstructing the left lane, it lowers its speed and does the correct manoeuvre while changing to the right lane. No warning is issued. SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 71 of 113 COSSIB

72 In the condition of "Warning", the step 2 above is changed as follows: 2b - The vehicle travels through the monitored area without lowering its speed making it difficult to do the correct change lane manoeuvre; 2c - The vehicle travels too near the Road Work deviation area; Tools: - LDM data player (MARS) - Simulated HMI (simple green-yellow-red icon display, included in the RDep application) - ShemServer Requirements: - SP5_RQ02_36_v1.0 The system shall be able to decide if and to whom a warning message has to be send - SP5_RQ04_36_v1.0 The system shall be able to predict the vehicle s trajectory - SP5_RQ16_36_v1.0 The system shall be able to query needed data from the LDM Expected Values / Results The requirements are met, specifically: 1. The Road Departure Runtime application is able to query the LDM for all the static and dynamic information of the vehicles present in the coverage area. 2. The Road Departure Runtime application is able to predict the trajectory of the vehicle and compare it with the values of the SDM. 4. The Road Departure Runtime application is able to decide if and to whom a warning message has to be send. Obtained Values / Results Status Test Case1 Link to external annexed documentation (if any) SP5_RQ02_36_v1.0 The system shall be able to decide if and to whom a warning message has to be send SP5_RQ04_36_v1.0 The system shall be able to predict the vehicle s trajectory Partly Difficult and time consuming SP5_RQ16_36_v1.0 The system shall be able to query needed data from the LDM Test Case 2: Road Departure Runtime application for a Dangerous Curve Test Description Description: Whenever a vehicle travel along the Dangerous Curve, the RSU monitors the vehicle s trajectory and Road Departure Runtime application predicts the near future trajectory and compares it with the SDM information. Depending on the risk analysis of the application, a Comfort, Safety or Critical warning is issued and sent to the HMI through the Message Manager. Input LDM data player LDM Road Departure Safe Drive Map Output Simulated HMI SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 72 of 113 COSSIB

73 The condition of "No Warning" is summarized by the following steps: 1. While approaching to the dangerous bend, the vehicle enters the curve keeping the average speed of 15m/s ± 5m/s. 2a The vehicle should drive through the curve in a comfortable situation and should never leave its lane. No warning should be issued. In the condition of "Warning", the step 2 above is changed as follows: 2b The vehicle travels through the monitored area without respecting the speed limits, making it difficult to drive smoothly; 2c The vehicle travels too near the lane lines or even leaves its lane; Tools: - LDM data player (MARS) - Simulated HMI (simple green-yellow-red icon display, included in the RDep application) - ShemServer Requirements: - SP5_RQ02_36_v1.0 The system shall be able to decide if and to whom a warning message has to be send - SP5_RQ04_36_v1.0 The system shall be able to predict the vehicle s trajectory - SP5_RQ16_36_v1.0 The system shall be able to query needed data from the LDM Expected Values / Results The requirements are met, specifically: 1. The Road Departure Runtime application is able to query the LDM for all the static and dynamic information of the vehicles present in the coverage area. 2. The Road Departure Runtime application is able to predict the trajectory of the vehicle and compare it with the values of the SDM. 3. The Road Departure Runtime application is able to decide if and to whom a warning message has to be send. Obtained Values / Results SP5_RQ02_36_v1.0 The system shall be able to decide if and to whom a warning message has to be send SP5_RQ04_36_v1.0 The system shall be able to predict the vehicle s trajectory Partly Difficult and time consuming SP5_RQ16_36_v1.0 The system shall be able to query needed data from the LDM Status Test Case2 Link to external annexed documentation (if any) SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 73 of 113 COSSIB

74 Table 16 Rdep SDM creator Test site TEST FORM SP 5 WP 5 Prototype Version RDep_Final System Configuration HW Release 1 SW Release 1 Compiled by / Company Andre Possani, DIBE Date Form Progressive Numb. 15 Functional Component RDep_learning SDM DB LDM VANET Positioning_sys Reference Document RDep_Orbassano, RDep_Satory 16 / FEB / / ABR / 2010 SF_D5.3.4_Specifications_ for_rdep_v3.4 Test Type Test Objective Test Purpose Test Environment Integration Verification Correctness Test Site Test Case 1: Safe Drive Map creator (learning phase) for a Deviation of Road Work Test Description Description: The Road Departure application relies on the Safe Drive Map (SDM), which is a Data Base filled with information from vehicles passing through a black spot (Deviation for Road Work). Creating the SDM is the offline part of the Road Departure application. The following procedure is performed at least 20 times: 1. In a two lane straight road, the vehicle should drive in the left lane with a speed of 15m/s ± 5m/s. 2. As soon as the driver can see that a Road Work deviation is present ahead obstructing the left lane, it should lower its speed and do the correct manoeuvre while changing to the right lane. The RDep_SDM_Creator program should monitor and record the vehicle s trajectory. After having all the sample trajectories, the RDep_SDM_Creator will analyze the obtained trajectories and will calculate the values for the Safe Drive Map. SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 74 of 113 COSSIB

75 Test sites: Tthe CRF Test Track in Orbassano (Italy) will be used for the Test Case 1. Requirements: - SP5_RQ16_36_v1.0 The system shall be able to query needed data from the LDM - SP5_RQ49_19_v1.0 The RSU subsystem shall be able to receive at minimum the current vehicle position and speed - SP5_RQ52_36_v1.0 The system shall receive static vehicle data like width, length, type of vehicle, mass. - SP5_RQ61_36_v1.0 The system must be able to receive data from the vehicles as well as the infrastructure Expected Values / Results 1. The RDep_SDM_Creator should be able to query the LDM for all the static and dynamic information of the vehicles present in the coverage area. 2. The RSU subsystem should be able to provide at minimum the current vehicle position and speed 3. The system should receive static vehicle data like width, length, type of vehicle, mass 4. The system should be able to receive data from the vehicles as well as the infrastructure 5. The RDep_SDM_Creator should be able to store the trajectories of the vehicles and analyze them. 6. The values for the Safe Drive Map should be calculated by the RDep_SDM_Creator. Obtained Values / Results SP5_RQ16_36_v1.0 The system shall be able to query needed data from the LDM SP5_RQ49_19_v1.0 The RSU subsystem shall be able to receive at minimum the current vehicle position and speed SP5_RQ52_36_v1.0 The system shall receive static vehicle data like width, length, type of vehicle, mass SP5_RQ61_36_v1.0 The system must be able to receive data from the vehicles as well as the infrastructure The RDep application should be able to store the trajectories of the vehicles and analyze them. The values for the Safe Drive Map should be calculated by the RDep application Status Test Case1 Link to external annexed documentation (if any) Test Case 2: Safe Drive Map creator (learning phase) for a Dangerous Curve Test Description Description: The Road Departure application relies on the Safe Drive Map (SDM), which is a Data Base filled with information from vehicles passing through a black spot (Dangerous Curve). Creating the SDM is the offline part of the Road Departure application. The following procedure should be done at least 20 times: 1. While approaching to the dangerous bend, the vehicle enters the dangerous curve keeping the average speed of 15m/s ± 5m/s; 2. The vehicle should drive through the curve in a comfortable situation and should never leave its lane. The RDep_SDM_Creator program should monitor and record the vehicle s trajectory. After having all the sample trajectories, the RDep_SDM_Creator will analyze the obtained trajectories and will calculate the values for the Safe Drive Map. SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 75 of 113 COSSIB

76 Test sites: Most probably, the LIVIC Test Track in Satory (France) will be used for the Test Case 2. Requirements: - SP5_RQ16_36_v1.0 The system shall be able to query needed data from the LDM - SP5_RQ49_19_v1.0 The RSU subsystem shall be able to receive at minimum the current vehicle position and speed - SP5_RQ52_36_v1.0 The system shall receive static vehicle data like width, length, type of vehicle, mass. - SP5_RQ61_36_v1.0 The system must be able to receive data from the vehicles as well as the infrastructure Expected Values / Results The requirements are met, specifically: 1. The RDep_SDM_Creator is able to query the LDM for all the static and dynamic information of the vehicles present in the coverage area. 2. The RSU subsystem is able to provide at minimum the current vehicle position and speed 3. The system receives static vehicle data like width, length, type of vehicle, mass 4. The system is able to receive data from the vehicles as well as the infrastructure 5. The RDep_SDM_Creator is able to store the trajectories of the vehicles and analyze them. 6. The values for the Safe Drive Map is calculated by the RDep_SDM_Creator. Obtained Values / Results SP5_RQ16_36_v1.0 The system shall be able to query needed data from the LDM SP5_RQ49_19_v1.0 The RSU subsystem shall be able to receive at minimum the current vehicle position and speed SP5_RQ52_36_v1.0 The system shall receive static vehicle data like width, length, type of vehicle, mass SP5_RQ61_36_v1.0 The system must be able to receive data from the vehicles as well as the infrastructure The RDep application should be able to store the trajectories of the vehicles and analyze them. The values for the Safe Drive Map should be calculated by the RDep application Status Test Case2 Link to external annexed documentation (if any) SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 76 of 113 COSSIB

77 Table 17 Rdep Runtime Test site TEST FORM SP 5 WP 5 Prototype Version RDep_Final System Configuration HW Release 1 SW Release 1 Compiled by / Company Andre Possani, DIBE Date Form Progressive Numb. 16 Functional Component RDep_runtime ùsdm DB LDM VANET Positioning_sys Reference Document RDep_Orbassano, RDep_Satory 26 / FEB / / ABR / 2010 SF_D5.3.4_Specifications_ for_rdep_v3.4 Test Type Test Objective Test Purpose Test Environment Integration Verification Correctness Test Site Test Case 1: Road Departure Runtime application for a Deviation of Road Work Test Description Description: Whenever a vehicle travel along the Road Work deviation, the RSU monitors the vehicle s trajectory and Road Departure Runtime application predicts the near future trajectory and compares it with the SDM information. Depending on the risk analysis of the application, a Comfort, Safety or Critical warning is issued and sent to the HMI of the vehicle through the VANET. The condition of "No Warning" is summarized by the following steps: 1. In a two lane straight road, the vehicle should drive in the left lane with a speed of 15m/s ± 5m/s. 2a As soon as the driver can see that a Road Work deviation is present ahead obstructing the left lane, it should lower its speed and do the correct manoeuvre while changing to the right lane. No warning should be issued. In the condition of "Warning", the step 2 above is changed as follows: 2b The vehicle travels through the monitored area without lowering its speed making it difficult to do the correct change lane manoeuvre; 2c The vehicle travels too near the Road Work deviation area; SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 77 of 113 COSSIB

78 Test sites: The CRF Test Track in Orbassano (Italy) will be used for the Test Case 1. Requirements: - SP5_RQ02_36_v1.0 The system shall be able to decide if and to whom a warning message has to be send - SP5_RQ04_36_v1.0 The system shall be able to predict the vehicle s trajectory - SP5_RQ16_36_v1.0 The system shall be able to query needed data from the LDM - SP5_RQ49_19_v1.0 The RSU subsystem shall be able to receive at minimum the current vehicle position and speed - SP5_RQ50_36_v1.0 The system shall be able to transmit the warning to equipped vehicles - SP5_RQ52_36_v1.0 The system shall receive static vehicle data like width, length, type of vehicle, mass. - SP5_RQ61_36_v1.0 The system must be able to receive data from the vehicles as well as the infrastructure Expected Values / Results The requirements are met, specifically: 1. The Road Departure Runtime application is able to query the LDM for all the static and dynamic information of the vehicles present in the coverage area. 2. The Road Departure Runtime application is able to predict the trajectory of the vehicle and compare it with the values of the SDM. 3. The Road Departure Runtime application is able to decide if and to whom a warning message has to be send. 4. The RSU subsystem is able to provide at minimum the current vehicle position and speed 5. The system is able to transmit the warning messages to the equipped vehicles 6. The system receives static vehicle data like width, length, type of vehicle, mass 7. The system is able to receive data from the vehicles as well as the infrastructure Obtained Values / Results SP5_RQ02_36_v1.0 The system shall be able to decide if and to whom a warning message has to be send SP5_RQ04_36_v1.0 The system shall be able to predict the vehicle s trajectory Not tested Difficult and time consuming SP5_RQ16_36_v1.0 The system shall be able to query needed data from the LDM SP5_RQ49_19_v1.0 The RSU subsystem shall be able to receive at minimum the current vehicle position and speed SP5_RQ50_36_v1.0 The system shall be able to transmit the warning to equipped vehicles SP5_RQ52_36_v1.0 The system shall receive static vehicle data like width, length, type of vehicle, mass SP5_RQ61_36_v1.0 The system must be able to receive data from the vehicles as well as the infrastructure Status Test Case1 Link to external annexed documentation (if any) SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 78 of 113 COSSIB

79 Test Case 2: Road Departure Runtime application for a Dangerous Curve Test Description Description: Whenever a vehicle travel along the Dangerous Curve, the RSU monitors the vehicle s trajectory and Road Departure Runtime application predicts the near future trajectory and compares it with the SDM information. Depending on the risk analysis of the application, a Comfort, Safety or Critical warning is issued and sent to the HMI of the vehicle through the VANET. The condition of "No Warning" is summarized by the following steps: 1 While approaching to the dangerous bend, the vehicle enters the curve keeping the average speed of 15m/s ± 5m/s. 2a The vehicle should drive through the curve in a comfortable situation and should never leave its lane. No warning should be issued. In the condition of "Warning", the step 2 above is changed as follows: 2b - The vehicle travels through the monitored area without respecting the speed limits, making it difficult to drive smoothly; 2c - The vehicle travels too near the lane lines or even leaves its lane; Test sites: The LIVIC Test Track in Satory (France) will be used for the Test Case 2. Requirements: - SP5_RQ02_36_v1.0 The system shall be able to decide if and to whom a warning message has to be send - SP5_RQ04_36_v1.0 The system shall be able to predict the vehicle s trajectory - SP5_RQ16_36_v1.0 The system shall be able to query needed data from the LDM - SP5_RQ49_19_v1.0 The RSU subsystem shall be able to receive at minimum the current vehicle position and speed - SP5_RQ50_36_v1.0 The system shall be able to transmit the warning to equipped vehicles - SP5_RQ52_36_v1.0 The system shall receive static vehicle data like width, length, type of vehicle, mass. - SP5_RQ61_36_v1.0 The system must be able to receive data from the vehicles as well as the infrastructure Expected Values / Results The requirements are met, specifically: 1. The Road Departure Runtime application is able to query the LDM for all the static and dynamic information of the vehicles present in the coverage area. 2. The Road Departure Runtime application is able to predict the trajectory of the vehicle and compare it with the values of the SDM. 3. The Road Departure Runtime application is able to decide if and to whom a warning message has to be send. 4. The RSU subsystem is able to provide at minimum the current vehicle position and speed 5. The system is able to transmit the warning messages to the equipped vehicles SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 79 of 113 COSSIB

80 6. The system receives static vehicle data like width, length, type of vehicle, mass 4. The system is able to receive data from the vehicles as well as the infrastructure Obtained Values / Results SP5_RQ02_36_v1.0 The system shall be able to decide if and to whom a warning message has to be send SP5_RQ04_36_v1.0 The system shall be able to predict the vehicle s trajectory Not tested Difficult and time consuming SP5_RQ16_36_v1.0 The system shall be able to query needed data from the LDM SP5_RQ49_19_v1.0 The RSU subsystem shall be able to receive at minimum the current vehicle position and speed SP5_RQ50_36_v1.0 The system shall be able to transmit the warning to equipped vehicles SP5_RQ52_36_v1.0 The system shall receive static vehicle data like width, length, type of vehicle, mass SP5_RQ61_36_v1.0 The system must be able to receive data from the vehicles as well as the infrastructure Status Test Case2 Link to external annexed documentation (if any) SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 80 of 113 COSSIB

81 Table 18 SMAEV_01 Functional test of components Bench (CRF) TEST FORM SP 5 WP 5 Prototype Version 0.3 System Configuration - HW Release 0.1 SW Release 0.3 Compiled by / Company Filippo Visintainer/CRF Date 12/04/2010 Form Progressive Numb. 19b Functional Component SMAEV 01 Reference Document D5.3.5 Test Type Test Objective Test Purpose Test Environment Integration Verification Correctness Bench Test Case 1: Functional testing Test Description Objective: The functional test aimed at verifying that all components worked according to the specifications, without failure. Description: Figure 2 shows the logical architecture of SMAEV application, taken from D The method of the functional testing was to highlight the most critical functionalities related to each subsystem and test each one in the laboratory. The results reporting followed a simple checklist, stating for each functionality the correctness operation or the failure. SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 81 of 113 COSSIB

82 Figure 2 Logical architecture of SMAEV application Expected Values / Results Correct operation for each functionality Obtained Values / Results HMI testing: By navigating through the HMI client browser, all functionalities were correctly performed by the HMI server SMAEV HMI functionalities Functionality Correct Failure Use Case selection (tab panel switching) SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 82 of 113 COSSIB

83 Manual entering of data: all selectable options are actually valid, no fault, no conflict Manual entering of data: all selectable options are actually valid, no fault, no conflict Start/stop of application Switch from UC12 to UC51 Data correctly displayed to User (RSU message, current message) Start/stop sync (UC61) Local Dynamic Map: Data retrieval by SMAEV application from LDM was correctly performed Data Correct Failure Vehicle position Vehicle speed HMI messages from VANET Variable Message Sign: A subset of the data inserted via HMI was correctly transmitted via the serial port and displayed on the VMS panel. VANET Upon reception of an UDP message the router sent it to the VANET and another vehicle received it. VANET message transmission Message Correct Failure Assistance Vehicle (ego-) beacon HMI Message, periodic HMI Message, event Message reception Message Correct Failure SAFESPOT vehicle beacon RSU beacon HMI messages from VANET Conclusions: All SMAEV functions (HMI, LDM, VMS and VANET) operated as designed, i.e. responding to the specifications of D The system was then ready for outdoor testing. Status Test Case1 Link to external annexed documentation (if any) SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 83 of 113 COSSIB

84 Table 19 SMAEV_01 Test site (CRF) TEST FORM SP 5 WP 5 Prototype Version Final System Configuration HW Release 0.1 SW Release 0.3 Compiled by / Company Filippo Visintainer Date 20/05/2010 Form Progressive Numb. 19 Functional Component SMAEV 1 Reference Document D5.3.5 Test Type Test Objective Test Purpose Test Environment Integration Verification Correctness Test Site Overview of application testing The validation aimed at verifying the correct behaviour of the application. Though the final use case evaluation is left to WP5.6, the test scenario are based on a Use Case structure, mainly because of practical reasons, since each Use Cases implied a different scenarios and thus different preparation procedures. Some of these tests were actually supposed to characterise the overall application, for instance the results of event warning can be generalised also for other Use Cases; whereas some results are more specific, such as the ones related to slow vehicle UC. The test scenarios are briefly resumed: Test Scenario Title UC Test Case 1 Slow Assistance Vehicle moving on the road UC Assistance Vehicle signalling an event during its trip UC Event warning vehicle stopped UC Entering UC51 from UC12: continuity of service UC12, UC Assistance vehicle RSU cooperation UC61 5 Each test scenario included multiple trials, as reported in the test forms. The trials were carried out at Centro Sicurezza Fiat test track and on Brescia-Padova Motorway. They involved the following SAFESPOT elements: CRF Assistance Vehicle demonstrator, Fiat Croma, called AV CRF SAFEPROBE demonstrator, Fiat Bravo, called SF MIZAR Roadside Unit (limited to UC 61 testing), called RSU Data were logged in both the AV and the SF vehicle and then re-processed offline to analyse the experiments. SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 84 of 113 COSSIB

85 Objective: to test UC11 Slow Vehicle Test Case 1: Slow Assistance Vehicle moving on the road Test Description AV = Assistance Vehicle - i.e. CRF COSSIB demonstrator vehicle, integrating SP5-SMAEV01 sub-application SAFESPOT vehicle = CRF SAFEPROBE or SCOVA demonstrator vehicle, integrating SP5 client Description: 1. The AV moves at slow speed on lane 1. UC11 (Safety margin for maintenance vehicle on snow removal or salting operation) functionality is switched on 2. The operator inside the AV starts UC11 from the AV HMI module and gives corresponding warning of slow vehicle (due, for example, to road maintenance operation) 3. A SAFESPOT vehicle comes on lane 2 from behind, heading the same direction of the AV at higher speed Trials Trials with the following characteristics have been performed: A. 1 test track loop al varying the speed values of AV (e.g. 30, 20 km/h, stopped); SF vehicle queuing behind B. 3 trials with SAFESPOTvehicle overtaking at 3 different speed values (40, 60, 80 km/h); AV at 30 km/h C. 3 trials with SAFESPOT vehicle arriving at different speed values (40, 60, 80 km/h) and slowing down as soon as the VANET message is received and interpreted by the driver; AV at 30 km/h Test Sites: - Centro Sicurezza Fiat CRF test track SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 85 of 113 COSSIB

86 Requirements: - SP5_RQ02_36_v1.0 - SP5_RQ06_19_v1.0 - SP5_RQ07_19_v1.0 - SP5_RQ12_33_v1.0 - SP5_RQ14_33_v1.0 - SP5_RQ16_36_v1.0 - SP5_RQ17_36_v1.0 Use cases: SP5_UC51 -P5_UC11 Expected Values / Results The results should validate that the requirements are met, specifically: Results for every trial - A warning of the following kind: Operation type Lane Speed is displayed on the display of an incoming SAFESPOT vehicle. - A subset of this message is displayed on the VMS panel placed on top of the AV demonstrator, so that also non-safespot vehicles can see the warning. Results specific for trial A - By varying the AV speed, the attribute "Speed" in the warning should change accordingly. This is verified by comparing the message in step 3 and step 5. - When the vehicle stops the message should autonomously change into: Operation type Lane STOPPED VEHICLE AdditionalWarning ; Results specific for trial B - VANET message is received independently of the relative speed between SAFESPOT vehicle and AV. Results specific for trials C and D: VANET versus VMS-only - Assuming a straight road, upon reception of signal the SAFESPOT vehicle should be able to slow down and - queue behind the AV (i.e. the two vehicles should proceed at the same speed). - The distance between AV and SF vehicles when they are able to queue will be considered: the higher the distance, the better the results. - The comparison between trial C and D will give an indication of the added value of VANET with respect of VMS only. The trials will be made on two different lanes for safety reasons. Results for every trial Obtained Values / Results For single-run scenarios, out of 6 trials, for 5 times UC11 warning: SPECIAL VEHICLE-SPEED X KM/H was displayed on the SAFESPOT vehicle HMI, where X was the speed of the Assistance vehicle rounded to multiples of ten (20, 30, 40, 50, km/h, etc). Results specific for trial A Varying the speed the value changed accordingly. When the vehicle stopped, the message SPECIAL VEHICLE-STOPPED was displayed, in compliance with specifications. The wording SPECIAL VEHICLE was also displayed on the VMS panel. In one trial the message was received but not processed by the application. SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 86 of 113 COSSIB

87 For the queuing scenario, the SF vehicle followed the Assistance Vehicle. The latter changed its speed and was continuously sending UC11 message to the vehicle behind. The following graphs summarize the queue scenario (left image, showing that both vehicles were running at the same speed) and the slow vehicle signalling results (right image, AV speed versus signalised speed). 60 Queue scenario: SF vehicle speed (circles) compared to AV speed (dashes) 60 Actual AV speed (green line, smooth) compared to the speed suggested by the AV and displayed on the SF vehicle HMI (red line, step-like) Speed Car [kmh] 30 Speed [km/h] Time [s] Speed Bravo Speed Croma Time [s] The results are in line with the expectations. The signalised speed, as designed in the algorithms, rounds the actual speed down to the nearest ten, e.g. an actual speed of 58 km/h yields a signalised speed of 50 km/h. Moreover, in the transitions between images, the driver experienced hardly any blinking (the latter was the primary concern in this scenario). Only some fast changes happened at the end of the trial (time > 130s in the graph) but this was also expected, as the changes were intentionally exaggerated with respect to ordinary scenarios, and overall the signalling behaviour reflects the actual speed one. As a further improvement, for instance to filter out event the fast changes experienced in the time interval s, an hysteresis could be introduced in the algorithms. Results specific for trial B The results confirm that the message was correctly received independently from the relative speed, for relative speed values ranging between 0 and 50 km/h. SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 87 of 113 COSSIB

88 Results specific for trials C and D: VANET versus VMS-only The VANET messages were received at a distance from 55m to 100 m between the SF vehicle and the Assistance Vehicle, sufficient for the application needs, and giving an added value with respect to the Assistance vehicle VMS. The latter indeed has a readability distance of m (see Test Case 3). Status Test Case1 Link to external annexed documentation (if any) Objective: to test UC12 Trip Signalling Test Case 2: Assistance Vehicle signalling an event during its trip Test Description Description: 1. A SAFESPOT vehicle is running on lane 1 2. The AV, with UC12 (Assistance vehicle patrolling or signalling a traffic event on a road) active comes from far away, heading the same direction at higher speed, and overtakes the SAFESPOT vehicle (virtually the AV is moving to a site where an event has happened) Trials Trials will be performed varying the relative speed between AV and SF vehicle. Test Sites: Centro Sicurezza Fiat CRF test track Requirements: - SP5_RQ02_36_v1.0 - SP5_RQ06_19_v1.0 - SP5_RQ07_19_v1.0 - SP5_RQ12_33_v1.0 SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 88 of 113 COSSIB

89 - SP5_RQ14_33_v1.0 - SP5_RQ16_36_v1.0 - SP5_RQ17_36_v1.0 Use cases: - SP5_UC51 -P5_UC12 Expected Values / Results The requirements are met, specifically: Results for every trial A warning of the following kind: - ATTENTION Event type, Event relative position, Event direction lanes ; is displayed on the display the SAFESPOT vehicle. - A subset of this message should be correctly displayed on the VMS panel placed on top of the AV demonstrator, so that also non-safespot vehicle can see the warning. Results comparing the trials - The VANET warning should be received when the vehicle is still behind the SAFESPOT vehicle, independently from the speed. This should prove the improved effectiveness of the use of VANET with respect to VMS only. Indeed - with VMS only there is a difference in perception between the conditions AV coming from behind and AV running in front, as in the former the signal is read through the rear-view mirror and the latter is direct in the scene in front. This difference in perception is more critical the higher the relative speed. - with the addition of VANET, the signalling is the same on both situations, and it is displayed on the vehicle HMI Obtained Values / Results 6 trials have been performed, 2 for each value of the AV speed (40,60, 80 km/h), while the SF vehicle was proceeding at 30 km/h. Correctness of warning, including geovalidity validation The generic warning of accident was correctly received. The following graphs report some shots of the tested scenario of rear geovalidity as recorded by the data logger (just for representation sake, the shots are taken every 5 seconds ). The objective of this image is to to show the geo-validity shape and orientation: as can be seen the latter varies according to the AV heading, as designed in the algorithm. The squared dots represent the Assistance Vehicle approaching and overtaking the SAFESPOT Vehicle (circular dot), running clockwise around the track. When the AV overtook the SF vehicle, the latter entered the geovalidity area i.e. the plotted rectangle following the AV and HMI message reporting an accident was correctly displayed to the SF vehicle. SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 89 of 113 COSSIB

90 Geovalidity Scenario 3 Test 2 (speed AV = 60km/h, geovalidity = 150m) Map source: Google VANET range testing and comparison to VMS The VANET warning was received when the vehicle was still behind the SAFESPOT vehicle, independently from the speed. Probably due a shielding effect of the VMS panel itself, the range of VANET was not symmetrical, being around m in front of the AV and up to 150 m behind the AV. Therefore, the perceived range with the AV coming from the rear was limited. Nevertheless the range can potentially be considered symmetric (ca. 150 m) once the antenna placement is improved. With VMS only, the incoming AV is seen only through the rear-view mirror and the range of readability is only a few tenths of meters. Status Test Case2 Link to external annexed documentation (if any) SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 90 of 113 COSSIB

91 Test Case 3: Event warning vehicle stopped Test Description Objective: to test UC51 Event warning (especially focussing on spatial discriminiation) Description: 1. The AV is stopped and simulates accident signalling. 2. A SAFESPOT vehicle passes by. Trials A. With VMS and VANET, varying the absolute speed of SF vehicle (from 50 km/h to 90 km/h of relative speed). B. With VMS only, varying the absolute speed of SF vehicle (from 50 km/h to 90 km/h of relative speed). Test Sites:Centro Sicurezza Fiat CRF test track Requirements: - SP5_RQ02_36_v1.0 - SP5_RQ06_19_v1.0 - SP5_RQ07_19_v1.0 - SP5_RQ11_33_v1.0 - SP5_RQ12_33_v1.0 - SP5_RQ13_33_v1.0 - SP5_RQ14_33_v1.0 - SP5_RQ16_36_v1.0 - SP5_RQ17_36_v1.0 SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 91 of 113 COSSIB

92 Use cases: SP5_UC51 Expected Values / Results The requirements are met, specifically: Results for every trial - A warning is correctly displayed on the display of an incoming SAFESPOT vehicle. - A subset of this message should be correctly displayed on the VMS panel placed on top of the AV demonstrator, so that also non-safespot vehicle can see the warning. Capability to stop - Upon reception of a signal the SAFESPOT vehicle should be at such a distance to be capable to slow down and stop. - The distance between the accident (or a reference point) and the SF vehicle when it is stopped: the higher the distance, the better the result. - The comparison between trial sets A and B will give an indication of the added value of VANET with respect of VMS only, assuming a straight road. The trials will be made on two different lanes for safety reasons. Obtained Values / Results Correctness and accuracy For each trial 3 messages were sent by the AV: A Comfort message, a Safety message and a Critical one. The geovalidity width (GW) was set such as to cover only one lane (12 to 14 m width depending on vehicle position relatively to the lane), whereas the geo-validity length (GL) was chosen according to the different scenarios (Urban, Rural, Motorway) Geovalidity rectangle length (m) Priority Urban scenario Rural scenario Motorway scenario Comfort Message Safety Message Critical Message Overall, 45 trials were performed, 16 for Urban. 18 for rural and 11 for motorway scenarios. The SF vehicle speed spanned the range km/h with a limit of 50 km/h for the urban scenarios. In 28 cases the SF vehicle entered the geovalidity areas (positive cases) in 17 the SF vehicle, proceeding in the outer lane, did not enter the geovalidity area and thus was not supposed to display the message on the HMI (true-negative). Overall, the message ATTENTION CAR ACCIDENT was correctly received and displayed in the receiving car HMI. The table hereafter reports the details of the results per specific scenario. SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 92 of 113 COSSIB

93 Overall Urban scenario Rural scenario Motorway scenario Comfort Safety Critical Tot. Urban Comfort Safety Critical Tot. Rur. Comfort Safety Critical Tot. Motor. Trials 45 Total messages Positive cases Missing alarms Negative cases False alarms The table proves that, with the actual positioning system having an accuracy below 1m, message geovalidity can effectively discriminate the lanes. As can be seen, the only exceptions happened in one trial with three messages that were supposed to be displayed were not actually shown (missing alarms), and in another trial giving 2 false alarms. VANET range versus VMS: a comparison At Centro Sicurezza, for urban and extra-urban speed values the messages were received between 240 and 300 m, far beyond the distance at which the VMS panel was read (30-55 m) Distance at which the VANET message is received [m] Speed [km/h] Distance at which the VMS is read (m) Speed (km/h) SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 93 of 113 COSSIB

94 Data sampling : LDM-related time lag estimation Overall, there was between 7 ms and 180 ms time lag between message reception by the VANET router and correct LDM reading by the External message Application. The mean value was 56 ms and a standard deviation of 35 ms. This time interval is mostly due to the periodicity of LDM reading by the application (T=100 ms) which introduces an average of 50 ms time lag. Only two exceptions (3% of the total sampling) reported a time interval of 3s and 5s respectively. The results are thus in line with the expected system features and give a negligible contribution to the overall system performance. Display and driver Reaction to Comfort Area messages HMI messages were displayed just a few meters after the geovalidity area, whereas the driver reaction was between 0,5 and 1 s. The following graphs plot the driver reaction in terms of time lag and distance covered between the triggering of the message display and the driver s reaction. Time between display command and driver reaction [s] 1 0,9 0,8 0,7 0,6 0,5 1st msg 2nd msg 3rd msg 0, Speed [km/h] Distance covered before Reaction [m] Speed [km/h] 1st msg 2nd msg 3rd msg It can be seen that the reaction to the first message (comfort) was slower and more speed-dependent, but this effect deserves further investigation. Moreover, the distance covered before reaction does not show a strong dependence on speed. Overall, the collected data suggest, for future deployment, to use a geovalidity setting with an addition of m independently of the speed, for instance setting a geovalidity to 120 m if the message has to be read below 100 m). Status Test Case3 Link to external annexed documentation (if any) SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 94 of 113 COSSIB

95 Test Case 4: Entering UC51 from UC12: continuity of service Test Description Objective: to test the combined scenario UC12 - UC51 (continuity of service) Description: 1. The SAFESPOT vehicle moves on lane 1, at about 50 km/h. It is heading towards an accident, without knowing it. 2. The AV vehicle approaches and overtakes the SAFESPOT vehicle, sending an UC12 message. Both vehicles proceed together on two different lanes (for safety reasons, in this testing phase) keeping at a distance of about m from one another. 3. The AV stops 50m before the accident position and, in a single step operation on the HMI (like pressing a button) it switches to UC The SAFESPOT vehicle is still coming behind at 50 km/h. As soon as the SAFESPOT vehicle receives the warning of UC51 it stops. Trials Trials shall be performed with and without the shortcut UC12 => UC51. Test Sites: Centro Sicurezza Fiat CRF test track Requirements: - SP5_RQ12_33_v1.0 - SP5_RQ14_33_v1.0 - SP5_RQ16_36_v1.0 - SP5_RQ17_36_v1.0 - Use cases: SP5_UC51, SP5_UC12 SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 95 of 113 COSSIB

96 Expected Values / Results The requirements are met, specifically: Operator HMI - The operator was able to switch from UC12 to UC51 in sequence without interruption. Capability to stop - Having received the UC12 message, the vehicle should queue behind the safety car (at a distance D determined in the trial). - Upon reception of a new signal (switching from UC12 to UC51 signal), the SAFESPOT vehicle should slow down and stop. - The distance between the accident (or a reference point) and the SF vehicle when it is stopped. - The continuity of service is met if the vehicle is able to stop before the accident. - This will be tested with and without a quick link between Use Cases in the HMI, in order to quantitatively measure the effectiveness of this link. Obtained Values / Results The test was carried out at 45 km/h.the requirements were met, the transition between UC12 to UC51 signalling in the operator HMI was correctly working, however its effectiveness was proved more in terms of comfort of the operator than in terms of signalling improvement, as in both cases (with and without the quick link on the HMI) the SF vehicle driver coming after the AV was warned on time to eventually slow down and stop. This is generally a good result in terms of effectiveness of SMAEV application: the person on the AV can easily switch between Use Cases providing continuity of service. A possible further evaluation could be with higher speeds and with actual operators on the Assistance Vehicle, a scenario that could be feasible for instance in a Field Operational Tests. Status Test Case4 Link to external annexed documentation (if any) Objective: to test UC61 (application-based multihop) Test Case 5: Assistance Vehicle - RSU cooperation Test Description Description: 1.The AV approaches a fixed RSU ( m). The RSU is sending an 'RSU HMI-Message' from the application, for instance Hazard and Incident Warning. 2. As soon as the AV operator sees, through the HMI, what message of type RsuHMIMessage the RSU is sending, the operator decides to stop the AV. Distance is expected to be m before the RSU. 4. The operator decides to switch on amplification (UC61) and give the same warning as the RSU. 5. A SF vehicle probe passes by SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 96 of 113 COSSIB

97 Trials Trials have been performed varying the speed at which the AV approaches the RSU. Test Sites: - Centro Sicurezza Fiat CRF test track - Brescia-Padova Motorway Requirements: - SP5_RQ02_36_v1.0 - SP5_RQ12_33_v1.0 - SP5_RQ16_36_v1.0 - SP5_RQ17_36_v1.0 Use cases: SP5_UC61 Expected Values / Results Amplification effectiveness - Let the RSU range be D max, assuming that the physical communication devices have an equal range on RSU and AV, the AV should ideally be capable to extend the range up to a distance D amp = 2D max. Realistically this distance is expected to be less. One of the main causes is that the AV has to see the message, decide to amplify it and stop. Therefore it will not stop exactly at the border of the range, and most of the times it will not move backwards to reach the border. It makes sense to think that D amp will be higher the lower the initial speed of the AV. - These trials aim in fact at giving an estimation of Damp close not too far from the real conditions. SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 97 of 113 COSSIB

98 Obtained Values / Results Centro Sicurezza test track The following tests show how it is possible, via the UC61, to amplify the RSU signalling and cover the whole extension of the test Track. The roadside unit was sending HMI messages. The application inside the assistance vehicle, placed at 270 m from the RSU and receiving the HMI messages, created its own HMI messages with the same content as the one present in the RSU. In practice it acted as bridge for the warning message, performing a multi hop at application level. The SF vehicle ran multiple loops around the track the track, receiving the HMI messages from both sources. The image shows the points where the HMI messages were received. HMI messages received by vehicle and RSU based on 11 laps of the test track HMI RSU HMI AV AV postion RSU position The picture hereafter reports only the assistance vehicle. An interesting aspect is the coverage of an area which is partially shielded by trees. The blue point on top right shows the farthest point that could be reached by an HMI message from the AV, which is 530 m away from the RSU. SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 98 of 113 COSSIB

99 Coverage of the multihop HMI AV AV position RSU position Brescia Padova testing Brescia Padova testing was meant to extend UC61, as implemented in the Centro Sicurezza, through a longer and rectilinear path. In the motorway Test Site the Assistance vehicle could stop in a free emergency space 350 m away from the roadside unit, close to a subway. Another aspect noted was the elevation, which was not constant along the road segment, but varied between 73,6 and 86 m as reported in the following graph. SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 99 of 113 COSSIB

100 Elevation A4 from , to , AV position RSU position Position [m] SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 100 of 113 COSSIB Elevation [m]

101 Message plots Roadside unit messages were received up to 750 m away, as shown by the following graph. The picture represents the message received by the SF vehicle in different positions. test 1 HMI Messages RSU RSU position test 2 test 3 test 4 test 5 test 6 test Position [m] with respect to the direction of motion: negative values indicate before the RSU and positive values indicate after RSU (RSU position = 0m) SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 101 of 113 COSSIB

102 HMI Messages AV, replicating RSU Messages AV position RSU position test 1 test 2 test 3 test 4 test 5 test 6 test Position [m] with respect to the direction of motion: negative values indicate before the RSU and positive values indicate after RSU (RSU position = 0m, AV position = -350m) Status Test Case5 Link to external annexed documentation (if any) SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 102 of 113 COSSIB

103 Table 20 SMAEV_01 Test site (SODIT) TEST FORM SP 5 WP 5 Prototype Version FinalFinal System Configuration Nicolas Etienne/ HW Release 0.1 SW Release 1 Compiled by / Company Date 3-5 March 2010 SODIT Form Progressive Numb. 20b Functional Component SMAEV 1 Reference Document D5.3.5 Test Type Test Objective Test Purpose Test Environment Integration Verification Correctness Satory Test Track Test Case 1: Assistance vehicle provides information Test Description Description: The aim of this test was to validate the behaviour of the system when an assistance vehicle provides information for an event on the road. - the AV (Assistance Vehicle ie the RSU) enters an event concerning what is happening on the road (car accident simulation). - The OBU approaching the position on the event will receive a warning message describing the current event. The OBU will have a speed between 50 and 110 km/h. Test sites: Satory Test track Requirements: RQ02_36_v1.0, RQ06_19_v1.0, RQ07_19_v1.0, RQ14_33_v1.0 Use cases: SP5_UC01 case Expected Values / Results The requirements are met, specifically: - Assistance Vehicle can enter through HMI application the status of the event, lane, user message - The OBU shall receive and display the incoming warning information. - The distance between the position of the event and the position where the OBU received the message shall be more thant 400 meters. SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 103 of 113 COSSIB

104 Obtained Values / Results The application test was successful. The obtained value were: We always received the warning message for a car speed in the range between 50 and 110 km/h. The distance between the emitter (RSU) and the OBU (when the car received the event) was estimated to be about 600 meters. No messages were lost during the test. The message received on the OBU contained the user message that was dynamically entered by the Assistance vehicle user. Status Test Case1 Link to external annexed documentation (if any) SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 104 of 113 COSSIB

105 Table 21 SMAEV_02 Test site TEST FORM SP 5 WP 5 Prototype Version Final System Configuration Prototype HW Release 0.1 SW Release 2 Compiled by / Company Nicolas Ettiene/SODIT Date November 2009 Form Progressive Numb. 20b Functional Component SMAEV 2 Reference Document D5.3.5 Test Type Test Objective Test Purpose Test Environment Integration Verification Correctness Test Site Test Case 1: Emergency vehicle crosses intersection Test Description Description: The aim of this test is to validate the behaviour of the system when an emergency vehicle wants to cross an intersection in urban area. Nominal case. - the EV (Emergency Vehicle) Driver switches its status by using the SMAEV_02 HMI. - the RSU (Road side unit) is able to detect the EV and to handle its request. Test sites: - SATORY SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 105 of 113 COSSIB

106 Requirements: - SP5_RQ30 Use cases: SP5_UC52 case Expected Values / Results The requirements are met, specifically: - The EV receives information telling him its request is being processed ("YOUR REQUEST IS BEING TREATED") and after receive a confirmation message: "YOU ARE HAVING PRIORITY". - The EV Status changes to "Emergency status". - The SF vehicles are informed of the emergency situation and can take care of the EV (they receive warninghmimessage). - The PRM (Priority Request Manager) manages the behaviour or the TLC (Traffic Light Controller). - The TLC is set in such a way that vehicles and pedestrians lights are red. Green for the EV arrival lane. When the EV has left the crossing area, the TLC must return to normal behaviour. Obtained Values / Results The RSU received the priority request and was able to deliver the priority as wished. The OBU received a confirmation message to go through the crossing The TLC switched between normal sequence to Emergency priority sequence giving the priority to the vehicle. After crossing the intersection, the TLC went back to its normal behavior. Status Test Case1 Link to external annexed documentation (if any) Description: The aim of this test is to validate the behaviour in warning failure mode. The EV asks for the emergency status. One of the following failures occur: - Bad data into LDM - Cannot localize the EV - TLC in security mode - VANET does not work. Test sites: - SATORY Requirements: - SP5_RQ30 Test Case 2: Failure mode status Test Description SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 106 of 113 COSSIB

107 Use cases: - SP5_UC52 case Expected Values / Results The requirements are met, specifically: - No change in the EV Status - Other SF vehicle does not receive any information about the EV arriving. - PRM is not able to take any decision concerning the EV request. Obtained Values / Results The VANET wasn t working, so we could not really test this situation. Status Test Case2 To Be Done Link to external annexed documentation (if any) SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 107 of 113 COSSIB

108 Appendix B - Information about the H&IW test results Table 22 Verification Table RFID Ghost driver two cars in opposite direction Verification Table Test N: Ghost driver: two cars in opposite direction. No Ghost Driver LDM-Motorvehicle Table LDM-Traffic Event N o RFID Message Value Field Value Field Value 3 antennaposition X: Y: geom X: Y: event.cause 35 4 vehicleid nodeid message Ghost Driver From RFID sensor 5 typeofgoods 0 goodstype 0 status 1 VehicleDescription 6 0 vehicledescription 0 7 vehiclewidth 0 vehiclesize_width 0 8 VehicleLength 0 vehiclesize_length 0 9 VehicleHeight 0 vehiclesize_height vehiclemass 0 (unit 0) vehiclemass ghostdriver 1 heading 0 SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 108 of 113 COSSIB

109 Table 23 Verification Table RFID Shadowing 2 Ghost Drivers Verification Table Test N: 2 Test #3: Shadowing 2 Ghost Drivers Ghost Driver 1 No Ghost Driver LDM-Motorvehicle Table LDM-Traffic Event No RFID Message Value Field Value Field Value X: X: antennaposition Y: geom Y: event.cause 35 Ghost Driver From RFID 4 vehicleid nodeid message sensor 5 typeofgoods 0 goodstype 0 status 1 6 VehicleDescription 0 vehicledescription 0 7 vehiclewidth 0 vehiclesize_width 0 8 VehicleLength 0 vehiclesize_length 0 9 VehicleHeight 0 vehiclesize_height 0 10 vehiclemass 0 vehiclemass 0 11 ghostdriver 1 heading 0 Test N: 2 Test #3: Shadowing 2 Ghost Drivers Ghost Driver 2 No Ghost Driver LDM-Motorvehicle Table LDM-Traffic Event No Messagio RFID Value Field Value Field Value X: X: antennaposition Y: geom Y: event.cause 35 Ghost Driver From RFID 4 vehicleid nodeid message sensor 5 typeofgoods 0 goodstype 0 status 1 6 VehicleDescription 0 vehicledescription 0 7 vehiclewidth 0 vehiclesize_width 0 8 VehicleLength 0 vehiclesize_length 0 9 VehicleHeight 0 vehiclesize_height 0 10 vehiclemass 0 vehiclemass 0 11 ghostdriver 1 heading 0 SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 109 of 113 COSSIB

110 Table 24 Obstacle-Pedestrian on the CRF Test Track Thermal imaging cameras (provided by VTT) Test 1 Parameters % measured # Images. total of Images analysed Value Total Images.>40% ,96 animal 1 Total image with confidence >40% 0 0 pedestrian 3 Total image with confidence >40% 64 10,72 car 5 Total image with confidence >40% ,24 Real Parameters Images containing moving person ,31 Images containing moving vehicles 0 0 Images containing moving animals 0 0 Test 2 Parameters % measured # Images. total of Images analysed Value Total Images.>40% ,03 animal 1 Total image with confidence >40% ,54 pedestrian 3 Total image with confidence >40% ,49 car 5 Total image with confidence >40% Real Parameters Images containing moving person Images containing moving vehicles 0 0 Images containing moving animals 0 0 Test 3 Parameters % measured # Images. total of Images analysed Value Total Images.>40% ,93 animal 1 Total image with confidence >40% 98 6,06 pedestrian 3 Total image with confidence >40% 150 9,28 car 5 Total image with confidence >40% ,58 Real Parameters Images containing moving person 79, Images containing moving vehicles 0 0 Images containing moving animals 0 0 SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 110 of 113 COSSIB

111 Table 25 Results Wrong Way Driver CRF Correctness # of correct Detections for first line/no total detections # of Correct detection second line Detection Traffic event first line/no Total Detection Traffic event second line /No total of events # of Messages generated/# of events # of visualization of the message inside the vehicle inside area Detection System 1 9/10 6/10 8/10 8/10 52/13 13 Detection System 2 10/10 10/10 10/10 10/10 40/10 10 Table 26 Wrong Way Driver CRF Communication Test Distance Vanet Messages Reception of HMI Line of Sight RSU V (m) Received in RSU & Vehicle Message Successful Successful yes 104 Successful Successful yes Successful Successful yes Successful Successful yes Successful Successful yes Successful Successful yes Successful Successful yes Successful Successful yes Successful Successful yes Successful Successful yes No communication No communication Not at all (in the area there were some trees between RSU and Vehicle) SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 111 of 113 COSSIB

112 Installed Sensors in the Pila highway Hyper Lan Communication for remote control of the RSU Installation of the elements in the BS-PD highway Figure 3 Pictures of the Obstacle (traffic jam & slow vehicle) SF_D5.5.2_Test and validation results ANNEX v1.4.doc Page 112 of 113 COSSIB