Validation and verification of specification models

Transcription

1 Validation and verification of specification models Test4Rail, Braunschweig Dr. Oliver Lemke V2.0

2 Agenda Introduction Needs Process Conclusion 2

3 SIGNON business activities Planning Engineering Technical Consulting Signalling systems Telecommunications Power supply Systems Software Safety Studies Methodology Processes 3

4 Introduction Using models and (semi-)formal languages like SysML have become more common over the last years for specifying railway systems: NeuPro DB s standardisation of interlocking architecture in Germany EULYNX European counterpart EULYNX ( is an initiative of 12 European infrastructure managers (IMs) to harmonize interlocking architectures This is accomplished by creating unified operator s specification documents for the supply industry 4

5 NO_OPERATING_VOLTAGE SysML state machine diagram Simulation interface F_EST_SubS_TDS - Behaviour [SubSTDS STD1] when( T1_Power_On_detected )/ when( T2_Power_Off_detected )/ OPERATING_VOLTAGE_SUPPLIED INITIALISING when( T4_Booted )[D13_Start_Time_synchronisation_is_initiated]/ T12_Interrupt_Safe_Communication_Protocol_Connection := true; when( T7_Invalid_or_missing_Configuration_Data_carrier )/ BOOTING FALLBACK_MODE when( T10_Safe_Communication_Protocol_Connection_disconnected )/ when( T9_Process_Data_Interface_connection_established )/ OPERATIONAL T12_Interrupt_Safe_Communication_Protocol_Connection := true; Specification document 5

6 Agenda Introduction Needs Process Conclusion 6

7 Needs The claim is, that model-based, (semi-)formal specifications improve the specification quality by: Being correct Being consistent Being unambiguous But this is only true if, the underlying model is unambiguous, consistent and correct As models can be reused for system acceptance tests, integration tests and various simulations, these requirements for model quality are even aggravated. Hence assuring a high level of quality for the models becomes essential. 7

8 Agenda Introduction Needs Process Conclusion 8

9 Process - simplified CENELEC V-model P1 Validation P5 P9 Verification Implementation phase 9

10 Process - small V-model for specification phase User requirements Formalised requirements Validation STM acceptance Verification State machine (STM) implementation 10

11 F_EST_SubS_TDS - Behaviour [SubSTDS STD1] INITIALISING when( T1_Power_On_detected )/ when( T4_Booted )[D13_Start_Time_synchronisation_is_initiated]/ BOOTING NO_OPERATING_VOLTAGE OPERATING_VOLTAGE_SUPPLIED when( T10_Safe_Communication_Protocol_Connection_disconnected )/ when( T9_Process_Data_Interface_connection_established )/ OPERATIONAL when( T2_Power_Off_detected )/ T12_Interrupt_Safe_Communication_Protocol_Connection := true; when( T7_Invalid_or_missing_Configuration_Data_carrier )/ FALLBACK_MODE T12_Interrupt_Safe_Communication_Protocol_Connection := true; User Reqs. Formalised reqs. State machine implementation State machine acceptance Knowledge Env. Stimulus A System Response a Stimulus B Informal documents Stimulus C Response d Scenarios as SysML sequence diagrams (ca scenarios per subsystem) SysML state machines in modelling tool Executable simulator Creation - Modeller Verification - Tester Validation - Stakeholder 11

12 Process - verification Verification step 1 (Black-Box-Verification): Verify that state machines (STM) react as specified in sequence diagrams (SD) Implemented by stimulating the executable simulator according to the SDs and reading back its responses, comparing them to the responses defined in the SDs Test execution is highly automated through GUI testing tools (e.g. Ranorex) Verification step 2 (White-Box-Verification): - Verify that STM does not add implicit behaviour not specified in sequence diagrams - Checked by verifying that all SDs fully cover the STM according to defined coverage criteria, e.g. full state and transition coverage - Unmarked states and transitions are not covered by sequences and therefore describe additional behaviour - Generate SDs covering the missing elements in STM and discuss with stakeholder 12

13 F_EST_SubS_TDS - Behaviour [SubSTDS STD1] when( T1_Power_On_detected )/ NO_OPERATING_VOLTAGE when( T2_Power_Off_detected )/ OPERATING_VOLTAGE_SUPPLIED INITIALISING when( T4_Booted )[D13_Start_Tim e_synchronisation_is_initiated]/ T12_Interrupt_Safe_Communication_Protocol_Connection := true; when( T7_Invalid_or_m issing_configuration_data_carrier )/ BOOTING FALLBACK_MODE when( T10_Safe_Communication_Protocol_Connection_disconnected )/ when( T9_Process_Data_Interface_connection_established )/ OPERATIONAL T12_Interrupt_Safe_Communication_Protocol_Connection := true; 13

14 Process - validation Knowledge Informal documents Operational stakeholders validate the STM against the input documents by using their own test cases. This assures diversity and coverage of domain knowledge. 14

15 Agenda Introduction Needs Process Conclusion 15

16 Experiences Statistics for a SysML model of the interface between interlocking and axel counting system. Model Number of SDs as formalised requirements: 45 Number of states in the STMs: 15 Number of transitions in the STMs: 38 Detected errors (after multiple manual reviews and quality checks) Verification - functional errors (STM does not match SDs): 5 Verification - implicit behaviour (STM contains behaviour not specified in SDs): 3 Validation - functional errors: 3 16

17 Conclusion The process presented is able to improve the specification quality The quality of model-based specifications is typically higher than of text-based specifications The additional benefit of using formal verification techniques must be evaluated, as the efforts for applying them in the real world are still very high Thank you! 17