Testability of Dynamic

Transcription

1 System Engineering in the Energy Testability of Dynamic and Maritime Sectors: Towards a Real-Time Systems Solution Based on Model-Centric Processes Lionel Briand slant.org

2 Software Challenge in the Oil and Gas Industry Today s Risk Reality: The offshore industry is facing rapid product innovation with more and more software embedded systems The evolution: From Mechanical operation Computer-based operation with integrated control & monitoring systems Implications: Increased system complexity, More and more integrated sub-systems, Components delivered by different suppliers, Complex operation DNV s concern is whether the offshore industry has been able to keep up with the evolution and adopt processes that enables them to cope with the level of the technological innovation 2

3 Example Control System I Heave compensation and tensioning Compensate for vertical movements for Marine riser Drilling Give tension to Marine riser system Guideline tensioners Podline tensioners Components: Compensator, Accumulators, Wire ropes, Sheaves, Compressors, APV s or NPV s, Piping, Control, Active electric motors 3

4 Example Control System II Drilling Control & Monitoring Systems Derrick Pipe Handling Typical Robotic Motion Control screen picture 4

5 Systems of Systems: Ship Source: Kongsberg 5

6 System Development Lifecycle System design System development coordination Owner NEED System Selection Component design Component implement. Component testing Integration & verification Validation & acceptance Operation Syst. Int. Operator Component design Component implement. Component testing Component design Component implement. Component testing Supplier COTS selection COTs purchase COTS testing Owner (OW) - Ensures that recommended practices (RP) are followed by suppliers and integrator. RP part of contract. System Integrator (SI) - Follows the RP included in the contract to perform its tasks - Includes the RP in the contracts established with component suppliers - Controls that RP are followed Supplier (SU) Follows the RP included in the contract to perform its tasks Operator (OP) - Follows the RP included in the contract to perform its tasks Verifier - Controls if RP are followed by SI, SU,OP (depending of its mandate) Many suppliers Integration complex and costly COTS/SOUP certification 6

7 DNV s Preliminary Study Source: Major issues include: - Safety certification - System integration 7

8 Current Safety Practices Safety Standards Recommended practices & Processes Hazard analysis (FTA, FMEA) Safety analysis (manual, informal) Safety testing (not systematic) Risk analysis Safety certification (safety case) 8

9 Standards Main instrument that governs system safety practice Public law, government regulations, industry committees sponsored by associations, societies, and institutes MIL-SD-882D: Standard Practice for System Safety IEC 61508: Functional Safety of Electrical/Electronic/ Programmable Electronic Safety-Related Systems ISO 17894: General principles for the development and use of programmable electronic systems in marine applications DO-178B: Software Considerations in Airborne Systems and Equipment Certification General guidelines, good to create a mindset, an organizational culture Do not ensure safety or even help achieve sufficient confidence Little (direct) evidence they are effective by themselves 9

10 DNV-OS-D202 OFFSHORE STANDARD (Excerpt) 201 All relevant actions shall be taken during manufacturing of software for a complex system to ensure that the probability of errors to occur in the program code is reduced to an acceptable level Relevant actions shall at least include 202 The actions actions to: taken to comply with 201 ensure that the programming of applications shall is be based documented on complete and valid implemented, specifications and ensure that software purchased from other execution parties has an of acceptable these actions track record shall and be is subject to adequate testing retraceable. The documentation shall impose a full control of software releases include and versions a brief during description manufacturing, of all installation tests that onboard and during the operational phase apply to the system (hardware and ensure that program modules are subject to syntax and function testing as part of the manufacturing process software), with a description of the tests that minimise the probability of execution failures. are intended to be made by sub-vendors, those to be carried out at the manufacturer s works and those to remain until installation onboard. 10

11 Processes and Recommended Practice Process standards: CMM, CMMI, SPICE (IEC 15504), IEC Establish good practices and prerequisites for effective development organizations But not sufficient to ensure safety 11

12 Safety Certification Often more expensive than development Can t directly measure safety Preserves chain of safety evidence Multiple forms and sources of evidence Safety case: A documented body of evidence that provides a convincing and valid argument that a system is adequately safe for a given application in a given environment (Adelard) 12

13 Safety Case Benefits Explicit claims, assumptions, decisions Can be reviewed Decrease cost of certification if produced during development Challenges Expensive when done after the fact Informal documents often imprecise, incomplete, hard to assess & analyze 13

14 Modeling Safety Arguments ASCAD - Adelard Safety Case Development York s Goal Structuring Notation Claim Inference rule Helps structure arguments But does not necessarily help ensure the precision and completeness of arguments Does not prescribe the type and degree of detail of information needed No analysis support Inference rule Argument structure Evidence Evidence Sub-claim 14

15 Hazards Assump;ons about the Environment Chain of Evidence for a Safety Case Cer3fica3on Requirements from Industry Standards System Safety Requirements So6ware Safety Specifica;ons Assump;ons about system interac;ons with environment Assump;ons about non so6ware system components Conformance Test Cases Source Code Verify Conformance Safety Relevant Design Decisions 15

16 Simplified Safety Case Safety system requirements: Reverse thrust should not be allowed while the plane is airborne Software safety specification: Reverse thrust should not be triggered when wheels are not spinning Assumption: wheels not spinning (above threshold) => airborne Review reveals wrong Design: triggerreverse() operation in Thruster class has pre-condition: assumption! self.aircraft.wheel.spinning (OCL) Source code: no other statement accesses the thruster driver than statements in triggerreverse() precondition checked as code assertion on entry + exception handling Test case report: triggerreverse() called when wheels not spinning (simulation) raised exception 16

17 The Mobile Offshore Drilling Unit using a DP system could float off beyond a dangerous limit. Strength of wind, waves and current. The thrusters to counter-act the forces of nature should fire within x seconds. Software response time of Y milliseconds for triggering thrusters. Dedicated task for detecting sway, yaw, surge and triggering the thrusters. Thruster proper;es adequate Test Strategy Assump;ons about how to interpret sensor data. Trace analysis Task pre-emption, priority, deadline, etc Software response time < Y milliseconds Static and dynamic code analysis Dynamic Positioning System

18 Support for Safety Cases Goal: cost-effective, trustable safety cases Relevant safety information must be precisely described and stored, A model-driven in a structured approach form is that required: can be queried, easily modified, Augment and system analyzed models to generate with safety safety cases Traceability information between various pieces of safety information, Generate from safety safety-relevant system requirements evidence to system test results, from is system key, to models decrease the cost of certification Likely to be, to some degree, domain Hard to achieve with document-based safety cases specific 18

19 Model-Driven Engineering Claim: Safety-critical model driven development must be model-driven, using modern standards and technologies, to help better integrate safety and development processes Model-driven engineering (MDE) is a software development methodology which focuses on creating models, or abstractions, more close to some particular domain concepts rather than computing (or algorithmic) concepts. It is meant to increase productivity by maximizing compatibility between systems, simplifying the process of design, and promoting communication between individuals and teams working on the system. 19

20 MDE: State of the Art Standard modeling notation for software: UML Specialized, standard modeling notation for systems based on UML: SysML Software modeling notation for embedded, RT systems: MARTE (UML profile) Good, extensible modeling environments for scalable modeling, model transformation, constraint checking, etc. Little to support modeling safety case information Little support for managing traceability among elements of safety case 20

21 Questions What information should be collected and at what degree of detail? How can it be captured in a systematic fashion? How can confidence in such information be quantified and confidence for overall safety claims be quantified? How can safety certification decisions be supported based on such information? 21

22 Safety Modeling Example UML class diagram profile to help capture evidence that safety standards are met Information modeling based on analysis of DO-178B safety standard: we identified 27 concepts, 48 attributes and relationships UML profile (Class diagram here): 32 stereotypes Example: Navigation Controller (NC) subsystem of an aircraft s navigation system G. Zoughbi, L.C. Briand, Y. Labiche, A UML Profile For Developing Airworthiness-Compliant (RTCA DO-178B) Safety-Critical Software, ACM/IEEE International Conference on Model Driven Engineering Languages and Systems (MODELS 2007), Nashville, USA,

23 Targeted Analysis Tracing model elements to requirements Ensuring that all functional requirements are accounted for Ensuring that all safety requirements are accounted for Reactions are traceable to safety requirements and events Validating software model to ensure safety Ensuring that all critical events are detected and handled appropriately Generating airworthiness certification information System partitions Software levels for subsystems and classes Safety monitors, events, and reactions Reactions are traceable to events that cause them Hardware/Software interfaces 23

24 Modeling Event and Reactions 24

25 <<SafetyCritical>>{CriticalLevel=C} <<Monitor>>{Kind-Safety, MonitoredEntity=Controller, DetectableEvent=, EventHandler= } <<Handler>>{HandlableEvent=, PerformedReaction = } <<Rationale>>{Reference= SREQ4, } <<Interface>>{IsBetweenHardwareAndSoftware =, InterfaceFor= } UML safety profile instance (class diagram 25 only)

26 Model-based Safety Analysis Guided hazard analysis (FTA) from models: System models (e.g., SysML) Environment models Need to check safety-relevant properties on requirements, specifications, and design models, in early stages Standard rule conformance (Example above) Safety properties violations Dealing with incomplete, incorrect models Guided, interactive, structured inspections 26

27 Example Analysis System requirements + Environmental assumptions Hazards Undesirable Events and Conditions Relevant Modules Behavioral models (e.g., state machines) Can undesirable events and conditions be obtained with the developed behavioral models? e.g., paths in a state machine We cannot expect complete, precise postconditions for all state machine actions 27

28 Safety Testing Automation SOFTWARE UNDER TEST (SUT) Sensor Input SUT SIMULATION Model the environment / context: - System & application states - At level of abstraction perceived by the SUT - Using a modeling notation as similar Simulated as possible as the one used for Sensors designing SUT Simulated Application Effector Output Effector Hardware Test strategy Hazard Analysis Dunn, 2007 Fault Library Model failure modes System Response Archive

29 Context Modeling UML System Sequence Diagrams modeling system-level interactions 29

30 Domain Model State models of sensors or effectors

31 Challenges Adapt modeling notations to address both system modeling and certification needs Analysis of safety properties in early stages of development based on incomplete, possibly incorrect models => search-based techniques Quantifying confidence in safety evidence Safety engineering is multidisciplinary Scaling up to systems of systems? 31

32 Conclusions The safety process must be integrated into the development process Safety case data collection and analysis must be an integral part of development activities Model-driven development is a solution: Safety-relevant information modeled is part of system models Models are a basis for safety-relevant analyses and inspections Models can drive safety testing Models enables support for traceability New modeling standards (e.g., SysML), methodologies, and tools make all this easier 32