Achieving ISO Compliance in Silicon (And Beyond?)

Transcription

1 Test and Verification Solutions Achieving ISO Compliance in Silicon (And Beyond?) Mike Bartley, TVS mike@testandverification.com 1

2 Agenda Some background on asuresign ISO26262 And the implication for requirements Advanced verification Requirements sign off is not so obvious Supporting hierarchy and cross-discipline And the supply chain? 2

3 Background TVS was established in 2008 The TVS headquarters are in Bristol, UK with Offices in India, France and Germany 80 verification engineers and software testers engaged around the world TVS developed a verification management tool in 2010 Aimed at verification data management Linking verification activities to requirements 3

4 D o w n s t r e a m U p s t r e a m ISO and Requirements Management ISO Stipulates The management of safety requirements includes managing requirements, obtaining agreement on the requirements, obtaining commitments from those implementing the requirements, and maintaining traceability. Requirements Stakeholder Requirements (Customers and internal) Product Requirements Intent to implement Safety Requirements SOC and Module Specs Intent to verify Verification & Test Plans Proof of implementation Verification & Test Results 4

5 Why do we need a new tool? Good tool support for requirements tracing Doors, Reqtify, Enterprise Architect, Jira, But users report limited support for testing What do they need for requirements signoff? Capture the mapping of requirements to signoff activities (not just directed tests) ability to manage changes Automate recording of tests results recording of the configuration data reporting of requirements signoff status Document the sign off of the requirements 5

6 Median peak number of verification engineers Functional Verification Trends Industry evolving its functional verification techniques Assertions Constrained-Random Simulation 37% 41% 64% 69% 2007 Wilson Research Group and Mentor Graphics 2010 Functional Verification Study, Used with permission Code coverage 48% 72% 2010 Functional coverage 35% 40% 72% 0% 10% 20% 30% 40% 50% 60% 70% 80% Listen to the 2012 survey Harry Foster at DVClub April 8th 30% 29% % % 15% 10% 5% 19% The adoption of formal property checking has grown by 53% 0%

7 The mechanics of an advanced test bench Test Functional Coverage Coverage Checker Monitor Stimulus generator constraint addr data Assertions Driver Design Under Test assert Coverage Active Passive Code Coverage 7

8 What are the implications for Requirements Signoff? Just mapping a requirement to a directed test is NOT sufficient Requirements need to map to Tests Directed Constrained random with a particular seed Coverage Code, functional and assertion Checkers Dynamic and Static Need to automate Test pass and fail Coverage collection and reporting Checker pass and fail All linked to configuration management data 8

9 Complex Bi-Directional Mappings Req1 Feat1 Feat1.1 Goal1 Metric1 Feat1.2 Feat1.3 Goal2 Goal3 Metric2 Metric3 Goal4 Metric4 Metric5 Req2 Feat2 Metric6 75% 50% 0% asuresign Export Reqs Status as XML: Req1 [, ] Req2 [, ] Metrics can be: From HW verification From Silicon validation From SW testing 9

10 asuresign Requirements Signoff Reqs Regression Scripts UCIS Regression Tests Reqs Signoff Test Holes Reqs History Resources 1. Staff 2. Hardware Reqs Configuration System Bug Tracking Test History 1. Versions 2. Pass/Fail asuresign Do not disturb your current regression flow 10

11 Cross Domain Considerations Requirements can be signed off via a range of different disciplines Pre-Silicon Verification Across hierarchy Software testing Pre-Silicon Post Silicon Post-Silicon Functional verification of features Validation of electrical features 11

12 Some features can only be verified at SoC Low power features Voltage islands Power down and power up Frequency scaling The interrupt system HW/SW co-verification Digital and Analog integration Chip wide ECC Lock step CPUs 12

13 Cost Saving: Remove Over Engineering Across Hierarchy and Across Disciplines Requirements can be tested at Block, Subsystem and SoC level Across multiple disciplines Manage requirements signoff at those levels and disciplines

14 Up the supply chain? asuresign supports Software testing Lab testing??? Can we pass asuresign databases up the supply chain? SQL databases? XML files? 14

15 Summary Electronics in cars More of it and more complex! ISO26262 mandates requirements signoff Hardware verification is a complex task Complex mappings from requirements to signoff Needs high levels of automation Requirements management tools not good at Supporting complex signoff Cross domain signoff Single view required across all domains Identify over/under engineering & risk based testing And Beyond?