Verification vs. Validation

Transcription

1 Alex Grove

2 Verification vs. Validation Validation: are we building the right product? Verification: are we building the product right? We have discussed RTL verification before In DO-254 this is called analysis Validation: a mostly manual step where review is fundamental to this process. Significant amount of documentation & spreadsheets. Today we are talking about Validation as well as Verification 2

3 So what is Safety Critical? Where the electronic system could cause actual harm Not just life & limb, also the environment Sectors and standards Avionics (RTCA DO-254/ED-80) Industrial (IEC 61511) Nuclear (IEC 62566) Automotive (ISO 26262) Transportation / Railway 3

4 Certification, Compliance, & Qualification Certification: the aircraft/engine Independent audit with regularity body (EASA,FAA) Compliance: the devices, CCA, LRUs, Systems Comply to a given industry standards Design Assurance Levels (DAL) vary depending on function and its criticality Evidence required to demonstrate compliance Qualification: typically applied to tools 4

5 Developing for Safety-Critical Safety is the highest priority The device/system must perform reliably and function as intended based on the requirements under normal and foreseeable abnormal conditions Design Assurance Confidence based on the elimination of uncertainty Good design practices are common DO-254 is widely considered as best practice Requirement to comply to industry standards The business driver for DA activities 5

6 Developing for Safety-Critical Requirements are central to the development process Design activities are based on the requirements Requirements are the start of the design phase Requirements inform the Designers what they need to design Verification activities are based on the requirements Requirements are the start of test planning phase Requirements inform the Verification Engineers what they need to verify Requirements are validated throughout the development process DO-254 defines a requirements capture/allocation process DO-254 requires all derived requirements to be validated 6

7 The Requirements Challenge Requirements are often poorly defined What if the requirements are badly written? What if the requirements are not verifiable? What if the requirements change? Requirements-Based Verification (RBV) addresses three major issues Ensures that requirements are correct, complete, unambiguous, verifiable and logically consistent Ensures that the design/system correctly implements the requirements Ensures that the final product meets the requirements 7

8 Requirements-Based Verification Requirements Capture Reviews Analysis Tests FPGA 8

9 RBV Methods for Safety-Critical Reviews Requirements Review and Validation Design Review Test Plan Review Test Results Review Planning Identify requirements to be verified Define functional coverage Analysis Traceability (correlation of data) Impact Analysis and Change Control HDL DRC CDC Analysis Simulation Coverage Analysis Static Timing Analysis Testing FPGA Level Tests FPGA/Board Integration Tests DO-254 Randall Fulton: hardware flies, not simulations 9

10 Avionics V&V Example Requirements Capture Conceptual Design Detailed Design Implementation Physical Testing * Aldec White Paper: Managing Validation and Verification Activities for DO

11 Avionics V&V Example 11

12 1 2 Example Graphical View 12

13 Some Common Questions How do we track and record the review activities? How do I build traceability? Do I have a test for each requirement? Am I meeting the coverage goals? What design and verification elements are impacted due to a requirement change? Have I tested all of the requirements? How do I track the progress? How long before we are done? 13

14 1 4 Managing RBV Activities with Spec-TRACER Spec-TRACER Requirements Lifecycle Management for FPGAs and SoCs Facilitates requirements capture, review, validation, coverage analysis, impact analysis and traceability Traceability to HDL design, testbench, coverage results, log files and waveforms Facilitates test plan creation and tests management 14

15 Conclusion Today s safety-critical FPGAs/SoCs continue to increase in size and complexity Industry standards recommend requirement based approach for safety-critical FPGAs/SoCs to ensure high-reliability and safe products By adopting such an approach ensures: Requirements are correct, complete, unambiguous & logically consistent Design/system correctly implements the requirements Design/system meets the requirements Managing the activities for RBV can be an overwhelming task what s needed is a systematic approach to manage them - this is the motivation behind Aldec s Spec-TRACER solution 15

16 Online Demonstration Videos 16

17 Q & A Direct Offices Corp. Headquarters N. America Ph sales@aldec.com Europe Ph Israel Japan China India Taiwan sales-eu@aldec.com sales-il@aldec.com sales-jp@aldec.com info@aldec.com.cn sales-in@aldec.com sales-tw@aldec.com Training /training Support /support Blog /company/blog Solutions Riviera-PRO Advanced Verification Platform Active-HDL FPGA Design and Simulation ALINT-PRO Design Rule Checking HES HW/SW Validation Platform Spec-TRACER Requirements Lifecycle Management DO-254/CTS FPGA Level In-Target Testing HES-7 SoC/ASIC Prototyping RTAX/RTSX Prototyping Microsemi Rad-Tolerant Devices 17