Usage Model-Based Testing for Hardware In the Loop THE SOLUTION TO BOOST YOUR TEST EFFICIENCY

Transcription

1 Usage -Based ing for Hardware In the Loop THE SOLUTION TO BOOST YOUR TEST EFFICIENCY

2 ALL4TEC Since 1998 French: 45 experts Turnover 2009: 4 M Expertise offer: QUALITY SAFETY ENGINEERING SYSTEM ENGINEERING TESTING EMBEDDED SYSTEMS

3 QUESTION YOUR TEST DEPARTMENT What is your cost dealing with various: Environment s Platforms Tools Specification Approaches Measurement Devices How long do you spend trying to: Reuse thru different engineers Reuse thru different projects Reuse thru different car brands How hard do you fight to: Estimate the quality Make documents, reports Calculate the functional coverage Imagine you can use all this invest for: INCREASING TRUST YOU HAVE IN YOUR PRODUCTS

4 MODEL BASED TESTING PROCESS Management Usage Usage Editor Cases Traceability Coverage -Cases Generator Campaign Analysis Scripts Adaptor Execution Platform Under Reliability Mean Time To Failure Intensity of Platform API Runner Hardware In the Loop Under

5 WHAT IS MaTeLo? Formal Specification Product point of view Enhanced Coverage Usage Cases Traceability Coverage Fully Measured process environment Scripts Compatibility Adaptor Reliability Traceability Execution Platform Under Mean Time To Failure Intensity of Better Engineering Productivity

6 WHAT IS EXAM? Formal Cases Specification Abstraction Layers Better Usage of Simulators & Devices Centralized Repository Method designed & consolidate Automotive Ready Library Focused on Reuse & Collaboration

7 MATELO USAGE MODEL (1) ALL POSSIBLE USAGES SCENARIO (3) FORMAL REQUIREMENT TEST SPECIFICATION (2) DYNAMIC REPRESENTATION (4) CONSISTENCY BETWEEN 1. TESTS 2. REQUIREMENTS 3. USE CASES

8 TESTING SCOPE Process Black Box testing Field Application Functional ing Integration ing Acceptance ing Under Software Product

9 PRODUCT SET MaTeLo Editor: Based Specification Design the usage model Qualify the possible path with profiles Assign test operations and requirements MaTeLo or: Based Cases Generation Define the test strategy Generate the test cases EXAM: Automation Manage the test cases on the test repository Manage the test environment heterogeneity Execute an log automatically the test sequences Start Close MaTeLo Campaign Analysis: Based Quality Insurance Analysis of the Campaign Quality & Reliability assessment

10 Usage Cases Traceability Coverage MaTeLo EDITOR Usage Design Scripts Adaptor Execution Platform Under Reliability Mean Time To Failure Intensity of

11 MATELO SCREENSHOT

12 MATELO USAGE MODELS Probability on every Choices BEGINNING Association of Configuration of Operations TRANSITION MACRO CHAIN STATE END

13 MODEL TRANSITION = TEST STEP Select_Gear(5) Accelerate(100%) Check_Speed(185) Doors Req. EXAM operations Stimulations Inputs Stimulation Classes of Equivalence Input ports Networks signals Variables GUI objects Oracle Outputs = f(inputs) Output ports Network signals Environ. Variables Wave Form Expected Results Checks Outputs

14 MATELO PROFILE Profiles can be embedded to qualify the usage model Operational profiles profiles Data distribution Usage path probability 87 (+/- 25)

15 Usage Cases Traceability Coverage MaTeLo EDITOR Management Scripts Adaptor Execution Platform Under Reliability Mean Time To Failure Intensity of

16 REQUIREMENTS MANAGEMENT IBM Doors MaTeLo Req Library MaTeLo Usage CSV (Reqtify, Excel ) Import Update Association Object -Transition -Input, Expected Result -Value, Class of Equivalence XML Requirement Role -Necessary -Sufficient MaTeLo Project

17 Usage Cases Traceability Coverage MaTeLo TESTOR Cases Generation Scripts Adaptor Execution Platform Under Reliability Mean Time To Failure Intensity of

18 MaTeLo TEST STRATEGY Most probable approach Field application approach Arcs coverage approach Usage approach Start Start Start Start Custom profile Usage profile Close Close Close Close FREQUENCY RISK, CRITICISM UPDATE COVERAGE USAGE (Determinist, Stochastic) DEFINE THE TEST STRATEGY, BY CHOOSING Algorithm Profile Part of model to test Generate

19 MaTeLo TESTOR: HTML TEST PLAN State Input Requirement Expected Result

20 Usage Cases Traceability Coverage MaTeLo EDITOR Bench Management Scripts Adaptor Execution Platform Under Reliability Mean Time To Failure Intensity of

21 MaTeLo FOR EXAM Repository SUT Interface Operations Stimulation Measurement Administration Sub Sequences Name UUID Description Association Association Automatic Cases Automatic call of Operation Automatic parameterisation association Case description generation Usage model respect Generate

22 Usage Cases Traceability Coverage EXAM Extended & Flexible Automation Scripts Adaptor Execution Platform Under Reliability Mean Time To Failure Intensity of

23 EXAM SCREENSHOT

24 ABSTRACTION LAYERS MaTeLo Third Party Equipment Usage Scenario Description << derive >> Case Specification << generate >> Flow Control Device Driver << control >> Under Markov-Chain Usage UML Sequence Diagram Python Precompiled Application Hardware In the Loop Bench

25 COLLABORATIVE ARCHITECTURE Multi Users Environnement Cases Implementer Python Code Generator Repository Project, Libraries, Campaign

26 Usage Cases Traceability Coverage EXAM ler Cases Description Scripts Adaptor Execution Platform Under Reliability Mean Time To Failure Intensity of

27 EXAM UML TEST SEQUENCE BEGINNING Operation Sub Sequence END

28 EXAM INTERFACE CONCEPT Case Description INTERFACE Case Implementations CLASS Sub Sequence Python Code Interface Implementation Virtual Function Virtual Car Sub Sequence Environment Access

29 Usage Cases Traceability Coverage EXAM ler Cases Management Scripts Adaptor Execution Platform Under Reliability Mean Time To Failure Intensity of

30 EXAM ABSTRACTION MANAGEMENT Variable Mapping Short-name Mapping Simulation Abstraction Parameter Set Composing Embedded Network Abstraction Configuration Concrete Case Abstraction Platform Abstraction

31 EXAM ABSTRACTION MANAGEMENT Possible Usage Scenarios Strategic Cases Case 1 Case 2 Case n HIL Plateform Available Configurations Network Functions Tools Env. dspace NovaSim Carts ASAM HIL API Proprietary CAN_1 CAN_2 LIN_X FlexRay_1 FlexRay_2 Manual Gear Auto Gear Hand Free ACC StartStop CANoe CANape INCA MS Excel Diag Tool Gasoline Diesel Turbo Hybrid Electric

32 SYSTEM CONFIGURATION 1 Possible Usage Scenarios Strategic Cases Case 1 Case 2 Case n HIL Plateform Available Configurations Network Functions Tools Env. dspace NovaSim Carts ASAM HIL API Proprietary CAN_1 CAN_2 LIN_X FlexRay_1 FlexRay_2 Manual Gear Auto Gear Hand Free ACC StartStop CANoe CANape INCA MS Excel Diag Tool Gasoline Diesel Turbo Hybrid Electric

33 SYSTEM CONFIGURATION 2 Possible Usage Scenarios Strategic Cases Case 1 Case 2 Case n HIL Plateform Available Configurations Network Functions Tools Env. dspace NovaSim Carts ASAM HIL API Proprietary CAN_1 CAN_2 LIN_X FlexRay_1 FlexRay_2 Manual Gear Auto Gear Hand Free ACC StartStop CANoe CANape INCA MS Excel Diag Tool Gasoline Diesel Turbo Hybrid Electric

34 Usage Cases Traceability Coverage EXAM Runner Cases Automatic Execution Scripts Adaptor Execution Platform Under Reliability Mean Time To Failure Intensity of

35 EXAM TEST RUNNER AUTOMATIC TEST SCHEDULER

36 Usage Cases Traceability Coverage EXAM Report Manager Verdict and Analyze Runs Scripts Adaptor Execution Platform Under Reliability Mean Time To Failure Intensity of

37 ANALYZE TEST RUN REPOSITORY TEST CAMPAIGN RESULT & VERDICT MANAGEMENT

38 EXAM TEST RECORD MANAGEMENT REAL TIME RECORD ANALYSIS AND VERDICTS

39 Usage Cases Traceability Coverage MaTeLo TCA Campaign Analysis Scripts Adaptor Execution Platform Under Reliability Mean Time To Failure Intensity of

40 Usage Cases Usage Scripts Adaptor Execution Platform Cases Under Scripts Adaptor Execution Execution Platform Platform Under Usage Cases Scripts Adaptor Under Traceability Coverage Reliability Mean Time To Failure Traceability Intensity of Coverage Reliability Mean Time To Failure Intensity of Traceability Coverage Reliability Mean Time To Failure Intensity of Usage Cases Scripts Adaptor Execution Platform Under Traceability Coverage Reliability Mean Time To Failure Intensity of TEST CAMPAIGN PROCESS Usage Target Quality Criteria 1. Coverage 2. Reliability Usage Cases New Cases Traceability Generation Coverage Metrics Quality/Version Trust Indicators Cases Traceability Coverage Scripts Adaptor Reliability Execution Platform Mean Time To Failure New Scripts Package Under Intensity of Bugs Adaptor Execution Platform Under Reliability New version Mean Time To Failure Intensity of

41 COVERAGE CAPABILITIES States coverage Transitions coverage S0 T 0-3 coverage T 3-4 (Req_34) Inputs: Req 341 Signal_A: [0-10;11-50;51-100] Req_3411; Req_3412; Req_3413 Expected Result Signal_B: > 47 & <= 62 Req_342 T 0-1 T 0-2 S3 T 3-4 Classes of Equivalence coverage S1 T 2-3 T 3-5 S4 Global coverage S2 S5 Indicators by SUT versions Unitary by version Cumulated over all SUT versions

42 TEST CAMPAIGN REPORT Coverage Database MaTeLo Campaign Analysis Coverage Results Database Trust Indicators

43 MATELO & EXAM Usage Cases Scripts Adaptor Execution Platform Under Traceability Coverage Reliability Mean Time To Failure Intensity of 1) Provide all needed test layers 1) Usage 2) Case Description 3) Case Implementation 2) Optimize all your test investments 1) Hardware In the Loop & tools 2) Engineers 3) Provide an open platform focused on 1) Formalization and abstraction 2) Reuse and collaboration 3) Automatic generation 4) Optimize the test campaign for 1) Functional Coverage 2) Product Availability

44 QUESTION? Booth 1720 Technical contact Sales contact Wiki & Forum & Documentation