Benchmarking and evaluation of railway operations performance Gemma Nicholson, David Kirkwood, Felix Schmid, Clive Roberts Birmingham Centre for Railway Research and Education, UK 25 March 2015
Outline Motivation Quality of Service (QoS) framework QoS quantitative evaluation Example of application ON-TIME project Conclusions and future work
Motivation Differing perspective depending on the stakeholder Evaluation of quantity and quality of operational behaviour Whole system approach Decomposition of system into properties Measure the effect of changing system properties, i.e., quantitative evaluation
Quality of Service framework Key measures Quantitative output KPIs Railway system System properties Railway system decomposition Influencing factors
QoS framework applications The QoS framework broadly has 4 applications: benchmarking of simulation tools comparison of tables or operational control systems visualisation of delay propagation linking of real operational data and microscopic simulation
QoSQE methodology
The full Quality of Service framework
Key measures The full framework KPIs Railway system System properties Influencing factors QoS framework outline
QoSQE performance indicators deviation area crew freight tonne km to recover delays at destination rolling stock passenger km journey no transfer interchange crowding maximum deviation station arrival delays energy consumption track Transport volume Journey Passenger Connectivity Resilience Punctuality Energy comfort Resource Railway system
TV key measures freight tonne km passenger km Transport volume journey no transfer Journey Transport volume interchange crowding deviation area to recover maximum deviation delays at destination passenger seat km = number of km travelled number of seats available on the service station arrival delays freight tonne km = number of km travelled freight train cargo capacity in tonnes energy consumption Passenger Connectivity total values Resilience Punctuality Energy comfort selected O-D pairs during T crew rolling stock track Resource Railway system
JT key measures freight tonne km Journey deviation area to recover delays at destination crew rolling stock passenger km journey no transfer interchange maximum deviation station arrival delays energy consumption The average journey crowding [seconds] of all journeys that make scheduled stops at O and D, in that order track Transport volume Journey Passenger Connectivity Resilience Punctuality Energy comfort Resource Railway system
CN key measures freight tonne km Connectivity deviation area to recover delays at destination crew rolling stock passenger km journey no transfer The average interchange interchange of maximum all interchanges station at I for crowding journeys that both depart O and deviation arrive at D arrival during delays the simulation T energy consumption track Transport volume Journey Passenger Connectivity Resilience Punctuality Energy comfort Resource Railway system
RS key measures freight tonne km passenger km Transport volume journey no transfer Journey Resilience deviation area to recover delays at destination Based on the system deviation measurement: interchange maximum station maximum deviation crowding during period T [seconds] deviation arrival delays to recover [seconds] deviation area [seconds 2 ] energy consumption Passenger Connectivity Resilience Punctuality Energy comfort crew rolling stock track Resource Railway system
PT key measures freight tonne km Punctuality deviation area to recover delays at destination crew rolling stock passenger km journey no transfer interchange maximum deviation station arrival delays energy consumption At station S, during crowding period T, the sum of arrival delays to all services departing S during T track Transport volume Journey Passenger Connectivity Resilience Punctuality Energy comfort at intermediate stations at terminal stations Resource Railway system
EG key measures freight tonne km Energy deviation area to recover delays at destination crew rolling stock passenger km journey no transfer For a interchange given O-D pair, the average maximum energy consumed station per crowding service for all services that both deviation depart from arrival O and delays arrive at D during period T energy consumption track Transport volume Journey Passenger Connectivity Resilience Punctuality Energy comfort Resource Railway system
RU key measures freight tonne km passenger km Transport volume journey no transfer Resource interchange crowding deviation area to recover maximum deviation delays at destination RU1: track : the average number of trains passing a point per hour during period T station arrival delays energy consumption RU2: rolling stock: the total number of rolling stock units in use Journey during period T Passenger Connectivity Resilience Punctuality Energy comfort crew rolling stock track Resource Railway system
Example: ONTIME project Project aim: an improvement in capacity by reducing delays and improving traffic fluidity Evaluation and comparison of traffic management approaches For minor perturbations For major perturbations
QoSQE applied in ON-TIME Benchmarking Network, table, period Simulation Evaluation Simulation Timetable delay delay delay (pre described scenarios) Simulation + real- perturbation management method delay introduced delay introduced Comparison between simulation and schedule at service level Key measure values Outputs Visualisation of KPIs
ONTIME case studies Iron Ore line, Sweden and Norway East Coast main line, UK Section of Dutch network, The Netherlands Minor perturbations e.g. temporary speed restriction, signal failure Major perturbations e.g. complete line blockage
ONTIME case studies Iron Ore line, Sweden and Norway East Coast main line, UK Section of Dutch network, The Netherlands Minor perturbations e.g. temporary speed restriction, signal failure Major perturbations e.g. complete line blockage
deviation area crew freight tonne km to recover delays at destination rolling stock passenger km journey no transfer interchange crowding maximum deviation station arrival delays energy consumption track Transport volume Journey Passenger Connectivity comfort Resilience Punctuality Energy Resource delay Railway system Capability Dependability Rolling stock Infrastructure Timetable Operational rules Traffic management Operational management Human factors System maintenance Environmental factors dynamic performance stations pattern train operational rules priority resource allocation planners condition monitoring technical protection facilities static characteristics track sections allowance infrastructure operational rules conflict detection incident management dispatchers maintenance plan environmental incident handling signalling buffer traffic operational rules conflict resolution drivers power network recovery crew operational rules delay management passengers communication network third parties passenger information system
IOL simulator benchmarking Comparison between tabled and simulated event s for IOL simulation
IOL simulator benchmarking stations NoNk NoSms NoRom NoKat NoBjf Rgn Kja Vj Lak Ka Bln Akt Ak Soa Kpe Sbk Tnk Bfs Rsn Rut Kv Pea Kia Rsi Apt Svv Kmb Kra tabled but not logged in simulation 0 5 10 15 20 25 30 35 tabled train service indices table matches simulation 1600 1000 500 100 50 10 1 Difference between tabled and simulated departure [seconds]
IOL quantitative evaluation Quantification of key measures 1. Demonstration of resilience KPI using in-built simulator logic only 2. Key measure results using perturbation management algorithms
NoNk 04005 09901 41905 09903 09171 09905 NoSms NoRom NoKat 9922B NoBjf Rgn Kja Vj Lak 9921B stations Ka Bln Akt Ak Soa Kpe Sbk Tnk Bfs Rsn 9919B 9175B Rut Kv Pea 9926B 09904 09906 45902 09176 0 1 2 3 4 5 6 7 [hours]
NoNk 04005 09901 41905 09903 09171 09905 NoSms NoRom NoKat 9922B NoBjf Rgn Kja Vj Lak 9921B stations Ka Bln Akt Ak Soa Kpe Sbk Tnk Bfs Rsn 9919B 9175B Rut Kv Pea 9926B 09904 09906 45902 09176 0 1 2 3 4 5 6 7 [hours]
240 180 deviation (min) 120 60 0 1 2 3 4 5 6 7 (hours)
NoNk 41905 09901 09903 09171 09905 stations NoSms NoRom NoKat NoBjf Rgn Kja Vj Lak Ka Bln Akt Ak Soa Kpe 04005 9921B 9175B 9922B Sbk Tnk 9919B Bfs Rsn Rut Kv Pea 9926B 09904 09906 45902 09176 0 1 2 3 4 5 6 7 [hours]
240 180 deviation (min) 120 60 0 1 2 3 4 5 6 7 (hours)
240 180 deviation (min) 120 60 0 1 2 3 4 5 6 7 (hours)
240 Resilience KPI 180 deviation (min) 120 60 maximum deviation deviation area 0 to recover 1 2 3 4 5 6 7 (hours)
Key measure results with perturbation management Punctuality
deviation (s) 16000 14000 12000 10000 8000 6000 4000 2000 0-2000 FCFS 1 2 3 4 5 6 7 (hours) Resilience deviation (s) 16000 14000 12000 10000 8000 6000 4000 2000 0 Algorithm 1 16000 Algorithm 2 deviation (s) 14000 12000 10000 8000 6000 4000 2000 0-2000 1 2 3 4 5 6 7 (hours) -2000 1 2 3 4 5 6 7 (hours)
Summary of the other measures JT O-D Reference simulation FCFS Algorithm 1 Algorithm 2 Mean journey [min] Narvik - Bergfors 162.5 (100%) 205.0 (126%) 164.0 (101%) 165.6 (102%) RS FCFS Algorithm 1 Algorithm 2 Deviation area [s 2 ] 1.519e+08 1.426e+08 1.257e+08 (100%) Maximum delay [s] 15445 (94%) 15136 (83%) 10189 (100%) (98%) (66%) Time to recover [s] - - - EG O-D Reference simulation Total energy consumption [kwh] Narvik - Bergfors 33714 (100%) FCFS Algorithm 1 Algorithm 2 33983 33770 33957 (101%) (100%) (101%) RU2 # trains 00:00:00-07:00:00 Reference simulation FCFS Algorithm 1 Algorithm 2 Passenger Freight Passenger Freight Passenger Freight Passenger Freight 0 15 0 15 0 15 0 15
Conclusions The QoSQE showed that the perturbation management approaches applied in ON-TIME resulted in improvements in particular to the resilience and punctuality KPIs less significant, but still positive outcome for the other KPIs Since the built-in simulator dispatching logic is used in the benchmark simulations, no quantitative conclusions can yet be drawn about the effects of perturbation management systems in real railway networks General implications Extension to platform independence Applicable to the assessment of operational data
Further work Formalisation of the key measures Application in other situations future projects using real operational data e.g. evaluation of traffic on the Birmingham Cross City line
Thank you Benchmarking and evaluation of railway operations performance Gemma Nicholson, David Kirkwood, Felix Schmid, Clive Roberts Birmingham Centre for Railway Research and Education, UK g.l.nicholson@bham.ac.uk