Deliverable 5.4. A TRANSPORT SERVICES QUALITY AND ACCESSIBILITY EVALUATION MANUAL Publishable summary

Transcription

1 Deliverable 5.4 A TRANSPORT SERVICES QUALITY AND ACCESSIBILITY EVALUATION MANUAL Publishable summary Coordinator: Professor Andree Woodcock, Coventry University tel.:+44 (0) , a.woodcock@coventry.ac.uk Duration of Research: Project Duration Nov 2012 October 2015 Deliverable Duration August September 2015 Author: Responsible partner: POLITO Participants: COVUNI, ITENE, ZHAW, VTM Tel: marco.diana@polito.it WEBSITE Grant Agreement no: Project Full Title A Measurement Tool to determine the quality of the Passenger Experience Legal disclaimer: The sole responsibility for the content of this report lies with the authors. It does not represent the opinion of the European Communities. The European Commission is not responsible for any use that may be made of the information contained therein. METPEX is a Project funded by the EC Seventh Framework Programme (FP7)

2 A TRANSPORT SERVICES QUALITY AND ACCESSIBILITY EVALUATION MANUAL Summary details This deliverable detailed the methodological steps needed to carry out an assessment exercise of a traveller experience by appropriately selecting a subset of rating questions from the METPEX survey tools and computing the related indicators. Moreover, all the indicators defined in the project are presented with the description of the items composing them. A comparison of the results obtained when computing the indicators for each test sites, i.e. the 8 cities plus the FIA network, is proposed too. Purpose The purpose of this Deliverable is to give stakeholders a manual on the use of the METPEX tool through the computation of a set of indicators defined during the project. By following the approach described here, the reader will be able to do the following: 1) To choose the set of indicators that is needed to measure the perceived quality for the specific evaluation perspective of interest, 2) To identify the list of satisfaction rating questions related to such set of indicators, 3) To use the METPEX survey tool for eliciting answers for the above list of questions, 4) To compute the indicators on the basis of the results of the survey, 5) To benchmark the results against those obtained in the test sites during the project. Some examples are proposed in the deliverable in order to show clearly and practically the work to be done. 2

3 Method The first step in the procedure aims at helping stakeholders identify which indicators are relevant for the problem under consideration. Various possible evaluation frameworks are then presented and for each one the related list of indicators is reported. Four different cases are envisaged: 1) The analyst might wish to measure the perceived quality of a specific transport service. In this case, several different indicators are proposed according to the travel means under investigation. 2) If the focus of the evaluation is on specific groups of travellers, indicators specific to each of the following user groups are available: women, commuters, travellers over 65, travellers under 24, low income travellers, visitors, rural dwellers, travelling with children, mobility restricted, communication impaired. 3) One of the distinguishing ideas of the METPEX approach is that of journey experience, as opposed to the somewhat more limited notion of trip. A journey is in fact made up of different phases, from the pre-trip part related to the information acquisition to the arrival at the final destination. Indicators focussing on specific journey phases are provided too. 4) Several different indicators are grouped in 14 different sets according to the specific quality issue one is wishing to investigate, following the classification proposed in Deliverable 5.3. Such sets are called Super Quality Indicators (SQI) followed by a number ranging from 1 to 14. After having identified the set of indicators relevant for the specific case under consideration, it is possible to search for the indicators needed in the manual and to check the list of variables that are needed to compute each of them. Thus, the following step would be to involve a sample of specific users and to implement a survey made up of the variables composing the indicators selected. Then, it is possible to compute the indicators chosen on the basis of the answers obtained from those satisfaction ratings, following the example provided in the deliverable. The outcome of the computation is a mean score for each indicator, ranging from 1 (minimum satisfaction level) to 5 (maximum satisfaction level). If the analyst wishes so, these scores can be compared with those obtained from the METPEX test sites, which are reported for most indicators in the deliverable. If the number of observations was too low to report the cityspecific score, it is in any case possible to benchmark the results against those obtained across all METPEX test sites. As an example of possible use of the METPEX tool, the deliverable presents some profiling of the METPEX test sites through radar graphs that report the values of all indicators appertaining to different travel modes, users groups, journey phases or quality issues. This is an example of an effective way of communicating to stakeholders and policy makers the main areas where the investigated service performs relatively well or relatively badly. Finally, the reader is strongly encouraged to carefully review the indications contained in the Final recommendations in order to fully understand the 3

4 context and limitations due to analytical framework under which the indicators where developed, and therefore make an appropriate and more informed use of the indicators here proposed, eventually amending them to better fit the needs of the evaluation exercise. Example of selection of relevant indicators In this example, the goal is to assess quality issues of passengers travelling by bus. As first step, it is apparent that a mode-specific evaluation exercise is appropriate. Thus, we search for the section of the manual which groups indicators according to travel modes and we select Bus service. The relevant indicators are BUS1: Reliability, BUS2: Ticketing and other issues and BUS3: Comfort on board. The next step involves the check of the list of variables that are needed to compute each of the indicators. So, the 3 indicators selected previously must be chosen among the 92 indicators presented in alphabetic order: BUS1 Reliability C i v62 Reliability of services v57 Punctuality u18 Time the journey took was as promised v61 Reliability at off peak times u19 Transport availability was adequate for my needs v48 Notification on timetabling changes u11 The quality of pre-trip information before I started my journey was good u8 Provision of information on arrivals and departures was adequate for my needs u10 The quality of travel information available during journey was good v73 Value for money of services was good Table 1 Indicator BUS1: List of variables and components scores coefficients BUS2 Ticketing and other issues C i v1 Ability to buy one ticket which covers different forms of transport v29 Easiness of connections with other modes of transport v23 Comprehensibility of ticketing structure v60 Range of fares offered u61 Provision of public transport only lanes u17 Ticket purchasing process was easy to follow u3 Design of transport stops was adequate for my needs u15 Support for intermodal (e.g. different forms of transport during same journey) travel was provided Table 2 Indicator BUS2: List of variables and components scores coefficients

5 BUS3 Comfort on board C i v44 Level of noise v43 Level of crowding v8 Air temperature and ventilation inside vehicles v20 Cleanliness of vehicles v70 Speeding and driving behaviour v68 Shelter provided from weather v38 Helpfulness of customer facing staff Table 3 Indicator BUS3: List of variables and components scores coefficients Looking at the previous tables, we find a list of 10 variables needed for BUS1, 8 variables for BUS2 and 7 variables for BUS3. Thus, the satisfaction survey would comprise 25 questions in total, following a 5-point satisfaction rating format. The following step would be to implement the survey involving a sample of bus riders and to compute the three indicators on the basis of the answers obtained to those 25 satisfaction ratings. An example of this computation is provided in the deliverable, explaining how the coefficients of the last column of the tables are used. Benchmarking values of indicators The outcome of this computation is a mean score for each indicator, ranging from 1 (minimum satisfaction level) to 5 (maximum satisfaction level). In the deliverable, the results obtained from the METPEX test sites are reported for most indicators through bar charts. 5,0 4,0 3,0 2,0 2,7 3,4 2,8 2,9 2,2 3,5 3,7 3,7 3,0 3,6 3,1 1,0 0,0 City A City B City C City D City E City F City G City H All 8 FIA Overall Cities Obs *The indicator is not shown for samples inferior to 10 observations Figure 1 Indicator BUS1: Benchmarking values 5

6 5,0 4,0 3,0 2,0 2,8 3,7 3,2 2,0 2,6 3,6 3,7 3,9 3,2 3,6 3,3 1,0 0,0 City A City B City C City D City E City F City G City H All 8 FIA Overall Cities Obs *The indicator is not shown for samples inferior to 10 observations Figure 2 Indicator BUS2: Benchmarking values 5,0 4,0 3,0 2,0 2,6 3,4 3,1 3,1 1,9 3,4 3,6 3,2 3,0 3,5 3,1 1,0 0,0 City A City B City C City D City E City F City G City H All 8 FIA Overall Cities Obs *The indicator is not shown for samples inferior to 10 observations Figure 3 Indicator BUS3: Benchmarking values If the analyst wishes so, the scores coming from his/her analysis can be compared with those obtained from the METPEX test sites, as those seen in the previous figures. The first 8 light blue bars in each chart refer to the 8 test sites, anonymised and called City A, City B,, while the dark blue bar collects the mean of the specific indicator values over all these cities. Then, the mean value of the observations referring to the FIA network of motorists are shown (10th bar). Finally, the orange bar is the overall mean value considering all the observations from the 8 cities plus the FIA network. The number of observations composing the sample of each city for that particular indicator is reported at the bottom of the figure. The y-axis of the plots goes from 1 to 5, like the scores given by the users to the evaluated items, higher values being associated with higher overall satisfaction ratings. It must finally be said that the mean value of the indicator for a given city is showed only if the relative sample is made up of at least 10 observations. 6

7 City quality profiles In the deliverable, a further example of possible use of the METPEX tool is provided. In fact, the profiling of the METPEX test sites through radar graphs that report the values of all indicators pertaining to different travel modes, users groups, journey phases or quality issues is presented. Five radar charts are produced, each one grouping together some or all indicators pertaining to a specific evaluation dimension. In each figure, the mean values of the indicators computed in each specific city are connected through blue areas creating a coloured polygon. The orange line represents the mean value of the indicators for the observations of all dataset. For example, these are the results obtained for the indicator computed for one of the cities: Figure 4 Modes Overview for City A 7

8 Figure 5 Users Groups Overview for City A Figure 6 Quality indicators overview (Part I) for City A QUAL1: Design of transport stations QUAL2: Design of transport interchanges QUAL3: Design of transport stops QUAL8: Provision of information on arrivals and departures QUAL9: Public Transport Staff QUAL10: Quality of travel information during journey QUAL11: Quality of pre-trip information before the journey QUAL13: Quality of ride QUAL14: Safety and security while travelling QUAL15: Support for intermodal travels QUAL16: Motorised vehicle users needs QUAL18: Reliability and on-time performance QUAL19: Service availability QUAL21: Value for money of services 8

9 QUAL5A: Quality of crossings QUAL5B: Physical interactions among modes QUAL6B: Non-discriminatory service and protection of data QUAL7A: Accessibility for different user groups QUAL7B: Social dimension of services QUAL12A: Overall quality of transport infrastructure QUAL12B: Vandalism and graffiti QUAL17A: Tickets regulations and flexibility QUAL17B: Practical aspects related to ticketing QUAL20A: Ergonomy QUAL20B: Design for specific user groups Figure 7 Quality indicators overview (Part II) for City A ON-PR2: Trip-specific aspects ON-PT1: Staff behaviour ON-PT3: Trip-specific aspects ON-PT4: Service operations characteristics PT-CI1: Communication aspects PT-CI2: Information design PT-CI3: Punctuality and reliability PT-MR1: Infrastructural design PT-MR2: Relevant features for mobility restricted PT-MR3: Service operations PT-MR4: Ticketing and other issues PT-WW1: Infrastructures design PT-WW2: Trip-specific aspects PT-WW3: Safety and security Figure 8 Journey phases overview for City A This is an example of an effective way of communicating to stakeholders and policy makers the main areas where the investigated service performs relatively well or relatively badly. Results According to the stability analysis results, when the number of observations available in the test site is sufficiently large, the definition and the structure of the factors coming out from a city are comparable with those from the total dataset. This is a very important result, since the identification and definition of the indicators here proposed can be considered valid also beyond the 9

10 dataset that was used to generate them, despite the fact that the Principal Component Analysis is a completely empirical and data-driven exploratory analysis technique, as previously mentioned. In the comparison of the indicators definitions according to PCAs from the test sites and from the total dataset conducted in Deliverable D5.3, however, some differences have been still noticeable. The variability of the cities due to elements such as the geographical position, the dimensions, the number of inhabitants or the transports provision, which is one of the strength of the METPEX project, certainly influences the identification of the indicators when considering only specific test sites. Anyway, as the results showed, some variables come to be a sort of common baseline in the composition of the indicators, irrespective of the portion of dataset under consideration. Conclusion & Opportunities for Further Research On the basis of the extensive testing of the above reported indicators, we can therefore conclude that the indicators here proposed can give a sound initial assessment of the perceived quality of different transport services, according to the perspectives of different users groups, or focusing on specific quality components or phases of the journey experience. Furthermore, such assessment can be matched against the results that were found for the different METPEX test sites. A more advanced approach would imply running a confirmatory factor analysis to amend some of these indicators or to make them more fit to specific evaluation instances, on the basis of some expert judgment. However, also if such additional work is needed, results here presented can give a useful input for the initial specification of the improved measurement model. 10