ERIGrid Holistic validation procedure and test specification

Transcription

1 ERIGrid Holistic validation procedure and test specification Kai Heussen Center for Electric Energy - Department for Electrical Engineering Technical University of Denmark Workshop Designing and Validating the Future Intelligent, Electric Power Systems September 6, 2017, Fuldatal/Kassel, Germany

2 Coordinated Voltage Control Validate this!

3 The Vision HOLISTIC SYSTEM VALIDATION

4 Holistic System Validation Specification Testing I. System validation: alignment of Specifications & Testing II. Integrated hardware & software testing Validate systems not components. &Debugging III. Tests that combine multiple domains e.g. Power, Comm. and Automation IV. Systematically design tests & integrate results from various experiments for a holistic assessment i.e. combine simulation, co-simulation, HiL, PHiL, CHIL, different Labs,

5 System validation - a holistic procedure "Holistic testing is the process and methodology for the evaluation of a concrete function, system or component (object under investigation) within its relevant operational context (system under test), corresponding to the purpose of investigation

6 A generic experiment / validation - Defines components of a system - Includes: Object of investigation - Defines functions of a system - Requirements define Test Criteria System Configuration Use Case Purpose of Investigation - Experimental design - Test procedure Input controllable & uncontrollable input parameters Configuration Test/ Experiment Object of investigation Experimental setup Boundaries of experiment Testing Tools Data Exchange Test Case Target Test Criteria Performance indicators/ Test metrics experiment assessment: Quality of experiment: error type; uncertainty; quantification of error

7 Design of Experiments efficient test design due to sampling-oriented approach Target measures / metrics e.g. average control error Test/ Experiment design sampling space on a need-to-know basis e.g. 3 levels of package loss rate, 20 levels of disturbance, Input Params Choose samples Target

8 Holistic Test vs. Component Test Component Test e.g. Inverter MPPT test, anti-islanding No interactions with the system Usually open loop test (predefined voltage, frequency etc setpoints are applied to the DuT) Holistic/System Test Combining several tests (testing process) Using simulations PV simulator DC Hardware inverter under test AC AC gid simulator Testing a system rather than just component Central Controller (optimization) RTDS Simulated Network PH IL PV simulator PV inverter Images: Panos Kotsampopoulos (NTUA); DoA Tutorial 9/14/2017 8

9 The basics HOLISTIC TEST DESCRIPTION

10 THREE levels of specification Holistic test description Use Cases Test Objective(s) Generic Test Case (TC) Map/split Test Specification Test specification(ts) Test specification Specific Map/split Experiment Experiment Experiment Specification Specification Specification (ES) Lab ERIGrid 3rd General Assembly - NA5 status 16/06/

11 2 research infrastructure (RI) capability profiles 2 Holistic testing procedure different mapping steps 1 Mapping from SC, UC, TO to holistic test case 1 Use Scenario Use Case(s) & Generic Case(s) System Configuration Use Cases Holistic Test Case Test Objective x 35 Process steps x mapping step 2 3 Mapping of holistic test case to (sub-) test specifications, i.e. specific (sub-) test system(s) Mapping of (sub-)tests to RI and specify experiments Sub-test specification Experiment Specification Exp. in RI 1 Division into individual (sub-)tests Subtest spec. 2 Exp. Spec. Exp. in RI 2 Sub-test specification n Experiment Specification Experiment in RI n RI 1 RI 2 RI n 6 holistic test evaluation test refinement Mapping of results of experiments to represent holistic result Mapping (preliminary) results to test adjustments 4 Exchange of data and results Mapping between tests resp. experiments ERIGrid 3rd General Assembly - NA5 status 16/06/

12 Key Questions to be answered for test specification: WHY TO TEST? WHAT TO TEST? WHAT TO TEST FOR? HOW TO TEST? Validate this! External presentation 14/12/

13 Key Questions to be answered for test specification: WHY TO TEST? WHAT TO TEST? WHAT TO TEST FOR? HOW TO TEST? Validate this! External presentation 14/12/

14 Test System & Domain System under Test (SuT): is a system configuration that includes all relevant properties, interactions and behaviors (closed loop I/O and electrical coupling), that are required for evaluating an OuI as specified by the test criteria. OuI Object under Investigation (OuI): the component(s) (1..n) that are subject to the test objective(s). Remark: OuI is a subset of the SuT. Domain under Investigation (DuI): Identifies the domains of test parameters and connectivity relevant to the test objectives. External presentation 14/12/

15 Test System Functions optimization in the controller state estimation OuI Use Cases define Functions Of Systems (IEC approach) DER P,Q control measurements OLTC tap control SuT Functions under Test (FuT): the functions relevant to the operation of the system under test, as referenced by use cases. Function(s) under Investigation (FuI): the referenced specification of a function realized (operation-alized) by the object under investigation. Remark: the FuI are a subset of the FuT. External presentation 14/12/

16 Key Questions to be answered for test specification: WHY TO TEST? WHAT TO TEST? WHAT TO TEST FOR? HOW TO TEST? External presentation 14/12/

17 Purpose of Investigation (PoI) o Verification o Validation o Characterization Modeling / Understanding Scoring / Performance Test objectives/poi: Characterization and validation of the DMS controller 1. Convergence of the optimization (validation) 2. Performance of the optimization under realistic conditions (characterization) 3. Accuracy of the state estimation (characterization) External presentation 14/12/

18 Designing Test Criteria Detailing Sequence Test objective PoI Test Crit. Test criteria: How to break down the PoIs? Target metrics (criteria): list of metrics to quantify each PoI Variability attributes: controllable or uncontrollable parameters to disturb SuT Quality attributes (thresholds): test result level or quality of the TM required to pass or conclude the testing. Target metrics: convergence (when/how often?), 1.2. How fast?, 1.3. solution quality Voltage deviation 2.2 number of tap changes, 2.3 network losses 3. Voltage, P, Q estimation errors Variability attributes: Load patterns (realistic, annual variation; applies to criteria 1-3); Communication attributes (packet loss, delays) Quality attributes (thresholds): 1.2: convergence within 2 sec (validation) 3.* estimation quality characterized with confidence 95% External presentation 14/12/

19 Key Questions to be answered for test specification: WHY TO TEST? WHAT TO TEST? WHAT TO TEST FOR? HOW TO TEST? External presentation 14/12/

20 2 research infrastructure (RI) capability profiles Test Specification & Design 1 3 SC UC Test Case SuT, FuT, PoI, TC Test Specification Test Test Design, I/O I/O 4 map 5 Experiment Specification Experiment in RI TO Research Infrastructure (RI) test evaluation 6 Given: Purpose of Investigation (PoI) & Test Criteria System & Domain categories and relations To Specify: 7 test refinement Precise system (specific system configuration) Which variables to manipulate & which to measure How to quantify the test metrics (based on test data) Sampling of the input spaces (design of experiments methodology) Combination and interpretation of the outputs The test design / procedure. Mapping to actual lab setup (experiement setup) External presentation 14/12/

21 Detailing test setup & Mapping to the Lab Scoping & specification of test system. Separate specification of lab implementation

22 Can I just say, that it's very nice to get these questions sorted out now, rather than when you're sitting down and have to implement something. You would then usually go "Oh shit, how does this work again? ERIGrid participant FIRST APPLICATION EXPERIENCE

23 Collaboration with ELECTRA ELECTRA Web-of-Cells (WoC) concept large set of Use Cases: distributed control concurrent development & lab implementation Challenge: how to track & convince that ongoing experiments actually validate the ELECTRA WoC Result (intermediate): Gap analysis based on Test Criteria & System configuration vs. ELECTRA goals organization of test case clusters; collaborative design of test formulation

24 ERIGrid Transnational Access: Preparation & Documentation External Lab users apply description procedure E.g. DiNODR distribution network oriented application of demand response currently ongoing in SYSLAB The preparation work helped us a lot. Except minor changes in the plan and configurations due to a number of device, communication and control unavailabilities, we are following our test and experiment specifications. The template is also useful for our user team to exchange ideas in an organized and effective way. - Alparslan Zehir (DiNODR) EU H2020 Programme GA No

25 Conclusions & Future work A clear vision for holistic validation First results: 3-level - Test Description template & guidelines Multi-Domain System Configuration description (CIM compatible) Several successful applications & encouraging feedback Future work: Further exemplify, simplify & detail description method Develop & apply full holistic validation procedure

26 Thank you for your attention! Get your copy of the ERIGrid Test Description here

27 2 research infrastructure (RI) capability profiles professional sequence Overall Specification & mapping procedure Generic System Configuration Use Cases Test Objective 1 3 Test Case System under test, Object under Investigation, Functions, Purpose of Investigation, Test Criteria map to specific test systems Test Specification Test Design, Test System Configuration, Input & Output 4 map to concrete lab setup test evaluation 7 test refinement learner Start anywhere. Build specification iteratively. 5 Experiment Specification Experiment Design, Experiment setup Experiment in RI 6 Research Infrastructure (RI)

28 IEC TR Guideline for definition of Basic Application Profiles (BAPs) using IEC Compliance Validation Accordance of the whole implementation with specified requirements or standards. However, some requirements in the specified standards may not be implemented. [SOURCE: CEN-CENELEC-ETSI SG-CG Report on Interoperability CEN_9762_CLC_9624 clause 12.1 Terms and definitions] 3.2 Conformance Accordance of the implementation of a product, process or service with all specified requirements or standards. Additional features to those in the requirements / standards may be included. [SOURCE: CEN-CENELEC-ETSI SG-CG Report on Interoperability CEN_9762_CLC_9624 clause 12.1 Terms and definitions] 3.3 Conformance test Verification check of data flow on communication channels in accordance with the standard conditions concerning access organization, formats and bit sequences, time synchronization, timing, signal form & level and reaction to errors. The conformance test can be carried out and certified to the standard or to specifically described parts of the standard. The conformance test should be carried out by an ISO 9001 certified organisation or system integrator. [SOURCE: IEC ] External presentation 14/12/

29 Holistic Test Case Example TEST CASE: Narrative: For a DMS controller in development stage (simple implementation) the performance of the DMS algorithm and controller should be evaluated under realistic conditions. This test, could be seen as the last step before installing the DMS in the field. SuT: DMS, DER, OLTC, transformer, distribution lines, telecom network OuIs: DMS_controller DuI: Electric power and ICT FuT: DER P,Q control, measurements, OLTC tap control, comm. via ICT FuI: optimization in the controller, state estimation Test objectives/poi: Characterization and validation of the DMS controller 1. Convergence of the optimization (validation) 2. Performance of the optimization under realistic conditions (characterization) 3. Accuracy of the state estimation (characterization) Test criteria how to formulate these objectives? Potential Test setups: Pure simulation (e.g. co-simulation) Combination of virtual & physical interfaces and simulated components. PHIL and CHIL Full hardware setup Target criteria - Variability attributes: - Quality attributes

30 Holistic Test Case Example TEST CASE: SuT: OuIs: DMS_controller; DuI: Electric power and ICT FuT: FuI: optimization in the controller, state estimation Test objectives/poi: Characterization and validation of the DMS controller 1. Convergence of the optimization (validation) 2. Performance of the optimization under realistic conditions (characterization) 3. Accuracy of the state estimation (characterization) [ ] Test criteria how to quantify these objectives? Target criteria: convergence (when/how often?), 2. How fast?, 3. solutions quality (how suboptimal etc.?) 2. Voltage deviation of all the nodes from 1 pu, number of tap changes, network losses 3. Voltage, P, Q estimation errors Variability attributes: Load patterns (realistic, annual variation; applies to criteria 1-3); Communication attributes (packet loss, delays) Quality attributes (thresholds): 1.2: convergence within 2 sec (validation) 3.* estimation quality characterized with confidence 95%