SoC Planning, Management, Reporting, Auditing, and Signoff

Transcription

1 SoC Planning, Management, Reporting, Auditing, and Signoff By Cadence Accurately monitoring progress on complex integrated circuit designs (or SoCs) has become more difficult as the designs have increased in complexity, leading to surprises from backwardslooking reporting and management processes that do not forewarn coming crises. The Cadence Metric-Driven Verification (MDV) Methodology provides a more uniform and standardized method of reporting progress towards closure in meeting specification requirements. Driven by the Metric-Driven Signoff Platform, the solution provides transparent program status, data-mining capabilities, and progressive access to the details of issues that have potential schedule impact, by providing program managers and oversight officers the information to achieve the desired goal of first-pass success. Contents Introduction...1 Why Change? (or, What Goes Wrong for Many Today?)...2 Elements...2 Program Management and Oversight Reporting...4 Insights for Oversight...5 Interactive Navigation: Peeling the Onion...6 Security Features: Authentication, Authorization, Audit...8 Summary...8 Further Information...8 Introduction First-pass success is the desired outcome for any system-on-chip (SoC) design. However, accurately monitoring progress on complex integrated circuit designs (or SoCs) has become more difficult at the same rate as the designs have increased in complexity (nominally measured by transistor count). This fact has led to many surprises from backwards-looking reporting and management processes that do not forewarn coming crises. In the ultra-highvolume, ultra-high-monetary-stakes markets in commercial electronics, new approaches have been put into practice that greatly improve the accuracy, timeliness, and transparency of progress reporting by leveraging design collateral data already being generated. This white paper outlines the Cadence solution for formalizing and automating a more uniform and standardized method of reporting progress towards closure in meeting specification requirements. This capability is important to all personnel responsible for managing, oversight and timely delivery of complex systems. The Cadence MDV Methodology has been proven in hundreds of programs and used on some of the most complex integrated circuits ever put into high-volume production. The solution provides transparency into the program status and data-mining capabilities that enable progressive access to the details of issues that have potential schedule impact. This system can be used within the context of most global program tracking and management systems, allowing SoC designers to achieve the desired goal of firstpass success. Management and oversight responsibilities for large SoC programs has become a more stressful and risky proposition as system complexity has continued at super-linear rates. Some of the basic requirements to improve the management of these programs are: Timely access to status and progress Accuracy/objectivity of information Aggregation of metrics data from across multiple verification platforms and sites

2 Transparency into underlying details Uniformity of content and format of reporting Traceability back to requirements Additionally, care must be given so as not to burden already overhead-laden programs with more processes whose sole function is keeping management in the loop. It has been shown in the commercial electronics industry that by leveraging the data generated when executing a metric-driven verification methodology that includes automation for planning, execution automation and reporting, a vast majority of the oversight requirements can be met without further effort. The key to higher quality program management and oversight is then to ensure that following the industry s best practices for SoC verification is a requirement. Because more than half of all SoC respins are due to functional failures, demanding application of best practices in verification just makes good dollars and sense. There are many documented case studies that have measured the benefits of adopting a full Cadence functional verification methodology. Customer reports show up to 60% verification effort reduction, and on-going track records of right-the-first-time SoC programs. Why Change? (or, What Goes Wrong for Many Today?) From a program management and oversight perspective, failure to diagnose the severity of current issues or foresee many upcoming problems prevents effective mitigation strategies from being executed prior to crisis level. Many programs that have extreme delays and ultimately fail already suffer from overhead-rich program structure and processes that hamper efficient use of front-line engineering resources. Adding additional reporting requirements, reviews, and processes simply starves the energy that should be applied to actual progress. These added costs in the name of getting it right the first time simply results in a further reluctance to consider SoCs for many applications where they would be ideal. Instead, the risks of SoC design are traded for alternate implementation fabrics that do not penalize as greatly for design re-work, but that have inferior size, weight, and power (SWAP). So, if so much energy is being expended on progress and issue reporting, what is going wrong? Our experience has shown that there are typically a number of systematic contributing factors that hinder the effectiveness of what are probably otherwise great policies and processes. These factors include: No automated link to up-to-date program goals No live/automated link between updated requirements and daily engineering tasks No reporting for issues in early triage Incomplete recording of actual functional coverage when multiple verification platforms are used for a project Difficult to share/interpret tool reports and understand trends over time Lack of mechanisms to objectively estimate time to completion and rate of closure This is just a sampling of the shortfalls found in many programs that have contributed to overall failure or lengthy/ costly delays. Elements An essential and integral part of the Cadence MDV Methodology is the platform that provides several critical functions to codify and provide uniformity to the verification process. As shown in Figure 1, these functions include: Creation/import of verification plans Ability to cross-link design requirements to verification plan elements to cover, test, and check metrics Ability to score, rank, and ID redundancy in a test suite Management of verification regressions across multiple verification platforms Dashboards for reports and regression control Merging/unification of data across verification engines and sites Management of underlying data in SQL database for tracking large-scale data trends over time Open, customizable, and extensible environment 2

3 Create Plans, Link to TB and Specs Analyze Metrics, and Coverage Launch Jobs, View Results, Triage Data Track Progress, Submit Reports Dashboards Reports, Launch www Planning Analysis Regression Tracking Web vapi Batch Data I/F ALM Custom ABVIP AVIP = Verification Management API Planning Execution Data Management VIP vplan Incisive Simulation vplan vplan JasperGold Formal Palladium XP / Z1 Tracking Regressions Metrics / Views Tracking Data Web Data Source Saved Coverage Regressions Data User Defined Views Used Defined Metrics / Perspectives Figure 1: Functions of the platform Coverage Files Log Files The underpinnings of a flexible, extensible solution come from its basic architecture. The Cadence platform uses a client-server architecture on top of a unified database, as shown in Figure 2. Local Area Network User Interface Clients Server Verification Database Figure 2: client-server architecture The client-server structure enables: Multi-user solution (100 users per server) Plus web access Manages multiple projects concurrently Multiple users per server Integrates Cadence verification engines Aggregates data across multiple sites Web is designed to run on any computer or tablet that supports browsers and HTML 5 (current versions) Remote access for project teams: status, dashboards, centralized reports Remote access for engineers: regression management, triage charts Simplified server administration: configuration, maintenance 3

4 The Cadence MDV Methodology is a cyclic process that drives programs to closure (Figure 3). The process begins with the creation of a plan that details the verification goals and the corresponding metrics that will execute in the different verification environments. Running on simulation, emulation, and formal verification platforms will establish the goals have been met. The tests are run, and the coverage and metrics are collected at the same time that the failures enter into a triage and debug subcycle. The data from the individual job runs are automatically aggregated into a central database that can be used for report and dashboard view generation. The reports can contain live links into more detailed reports that pinpoint failure type, cause, and location. Provisions are made for the inclusion of external data as well. Web Portal, Dashboards, Reports, Plan Regression Execution External Data Integration Runner / DRM Integration Project Tracking Multi-Engine Coverage Analysis Failure Analysis / Confirmation Figure 3: platform s advanced verification methodology control cycle Program Management and Oversight Reporting Based on the connection between the daily engineering activities and the top-level plan, management tracking data can be extracted at any time to get up to the minute progress reports (Figure 4). The live links also serve to provide the mechanism to dig deeper into the source and nature of the open issues that stand in the way of closure. Management Dashboards Report Portal Triage Charts Web Clients (VMG002) Figure 4: Typical management dashboard view generated by the platform The underlying reporting works via a snapshotting capability that can be configured for each program. The snapshot data includes time/date information so that charting of progress can be reviewed as a function of time (Figure 5). This provides a valuable trajectory towards closure view of the program. 4

5 Snapshot Table Snapshot Data Chart Snapshot Data Over Time Figure 5: Snapshot data for progress-over-time reporting The key capabilities for management and oversight provided by are: Timely access to status and progress Accuracy/objectivity of information Transparency into underlying details Uniformity of content and format of reporting Insights for Oversight Not all effort turns into progress. Being able to spot inefficient expenditures of time, energy, and resources is a hallmark of good program oversight. Of course the obvious gaps in progress are easy to spot with reporting. Figure 6 shows the (customizable) color coding for metrics reporting. The red flags are flagged red. The metrics report provides links back to the verification, where further diagnosis is possible for each test associated with each design feature. In essence, the platform provides a data-mining continuum from the high-level dashboard views down to, if necessary, the lowest level of actual design and verification artifacts. Figure 6: metrics report 5

6 Those obvious gaps in coverage are an important starting point for understanding actual program status. However, deeper insights are also possible through interpretation of trend data. From our experience on many high-complexity SoC programs, the following practices and observations are effective in ensuring programs stay on track: First priority: Tracking how much of the verification plan has been implemented Second priority: Tracking how much of the functional coverage has been closed in order to know how we well the project is tracking to verification plan Set milestones (not just the final goal) and have visibility into how well the project is tracking to verification plan goals for a given milestone Look at the code coverage to ensure the implementation is thoroughly exercised If some code can t be covered, the team needs to run unreachability analysis to ensure there is no dead code that should be removed If there is high functional coverage, but low code coverage, the team probably did not do a good job planning and has a weak vplan and functional coverage metrics Don t focus on the test suite growing, but do expect to see a growing number of simulation cycles over time to give some indication how much the verification is running Observe code churn metrics by measuring revision control check-ins over time, it should go down over time. A lot of code churn late in the project usually means more bugs being inserted, which will extend verification Use more formal verification since it can be more effective/efficient than simulation for different types of design features, but understand what it covers so no duplicate efforts or assumptions are made Know how much the system-level software tests exercise the chip when running on a Palladium platform, so it can be understood how much they contribute to the verification plan goals Following these program oversight practices has been a key ingredient in achieving first-pass success, on schedule, in numerous SoCs. Interactive Navigation: Peeling the Onion Getting the details you need to make accurate progress assessments can be slow as you try to peel the onion to get at the core of issues that jeopardize program success. The turnaround time for each information request can be lengthy and prolonged by misconstrued requests. Having direct access to actual program data (versus sanitized/distilled information) in a form that is easily and interactively navigable can eliminate days or weeks of further uncertainty. The platform (Figure 7) offers capabilities to enable program managers and oversight officers a way to get answers themselves, based on the actual project collateral. The client-sever architecture has a web-based interface designed to run locally or remotely on any computer or tablet that supports browsers and HTML 5 (current versions). Figure 7: platform offers broad options for web access 6

7 The platform s interactive user interface operates directly on the project data that drives the functional verification process. As shown in Figure 8, it is easy to traverse from top-level requirements, down into sets of tests and individual tests to examine detailed status for every part of the project. Hierarchical organization and color-coding help speed the process of methodical analysis. Figure 8: Verification plan view with grading annotations The operation of the clients are based on the same up-to-date data sources to ensure the consistency and integrity of the status reporting. The combination of hierarchical organization (Figure 9) and interactivity (Figure 10) provides the capability to dig where it is needed and not be inundated with unnecessary data where it is not. Figure 9: Client interface is interactive to enable deeper discovery Figure 10: Web client interface mirrors interactive report views of client 7

8 Security Features: Authentication, Authorization, Audit Ensuring integrity in the process and reports is an essential element to strong management and oversight functions. offers some basic authentication, authorization, and auditing capabilities that dovetail well into most program requirements on this front. Authentication Private user or password for each user Leverage corporate LDAP data or user configured by admin as user/password data tables platform will support both, or any one of the methods individually Authorization Once authenticated, authorization permissions are used for three levels System-level permissions: Licenses control Add/remove license Projects control - Add/remove projects Users/groups control - Add/remove/update user and groups Project-level permissions: Set which group/users have access to the project Create sessions (Include collect, compact, dynamic, rerun) Create tracking configuration Add/edit/delete user-defined attributes Scoped permissions (individual entities, specific to session and TC) Read User can only see the session or TC Write (include read by default) User can also update the specific session or TC Manage User can also delete data Summary Strong program management and oversight is predicated upon the availability of timely, accurate information. Automatically generated reports from daily engineering activities have the benefit of timely availability as well as the accuracy of actual status metric measurement. Further, because of the automation, the overhead normally needed to collect, collate, and summarize status can be substantially reduced. So, rather than adding effort in the program schedule, the engineering teams can expend a larger percentage of their time on program closure. The Cadence Metric-Driven Verification Methodology, driven by the platform, has been used on hundreds of successful SoC programs, including some of the largest and complex ever designed. The extensibility of the solution makes it adaptable to diverse sets of requirements and use within most global program management systems. Further Information Learn more about Cadence s Metric-Driven Verification Methodology at cadence-www/global/en_us/home/tools/system-design-and-verification/system-design-and-verification-flows/ metric-driven-verification-signoff.html. Cadence Design Systems enables global electronic design innovation and plays an essential role in the creation of today s electronics. Customers use Cadence software, hardware, IP, and expertise to design and verify today s mobile, cloud and connectivity applications Cadence Design Systems, Inc. All rights reserved worldwide. Cadence, the Cadence logo, and the other Cadence marks found at are trademarks or registered trademarks of Cadence Design Systems, Inc. All other trademarks are the property of their respective owners /16 SA/LL/PDF