Technical Paper Session 11 - Strategies to Improve Building Models and Operation

Transcription

1 2016 Winter Conference Technical Paper Session 11 - Joshua New, Ph.D. Oak Ridge National laboratory newjr@ornl.gov Strategies to Improve Building Models and Operation Paper #5 - Suitability of ASHRAE Guideline 14 Metrics for Calibration Orlando, Florida

2 Learning Objectives Objective 1 - Describe the current state of testing for building model calibration. Objective 2 - Explain the major components of the Trinity testing framework. ASHRAE is a Registered Provider with The American Institute of Architects Continuing Education Systems. Credit earned on completion of this program will be reported to ASHRAE Records for AIA members. Certificates of Completion for non-aia members are available on request. This program is registered with the AIA/ASHRAE for continuing professional education. As such, it does not include content that may be deemed or construed to be an approval or endorsement by the AIA of any material of construction or any method or manner of handling, using, distributing, or dealing in any material or product. Questions related to specific materials, methods, and services will be addressed at the conclusion of this presentation.

3 Acknowledgments Aaron Garrett JSU Amir Roth DOE BTO Zheng O Neill UA

4 Outline/Agenda Publication edits Context Trinity Test Limitations Web service implementation Purpose of this paper Results of a large Calibration study

5 Publication Edits Trinity test implementation of BESTEST-EX method to evaluate calibration, whether manual or automatic ASHRAE Guideline 14 definition (b) of calibration process of reducing the uncertainty of a model by comparing the predicted output of the model under a specific set of conditions to the actual measured data for the same set of conditions. We use the word calibration herein whether it is to actual measured data or to simulation output (as a surrogate to measured data)

6 Autotune Automatic calibration of models to data E+ Input Model... 6

7 Autotune Performance DOE Office of Science DOE-EERE: BTO Industry and building owners Results ASHRAE G14 Requires Autotune Results High Performance Computing Different calibration algorithms Machine learning big data mining Large-scale calibration tests Features Calibrate any model to data Calibrates to the data you have (monthly utility bills to submetering) Runs on a laptop and in the cloud 30+ Publications: Open source (GitHub): Monthly utility data Hourly utility data Residential home Within 30 /day (actual use $4.97/day) CVR 15% 1.20% NMBE 5% 0.35% CVR 30% 3.65% NMBE 10% 0.35% Results of 20,000+ Autotune calibrations (15 types, tuned inputs each) Other error metrics Leveraging HPC resources to calibrate models for optimized building efficiency decisions Tuned input avg. error Hourly 8% Monthly 15% 3 bldgs, 8-79 inputs

8 Trinity Test what is it? True model defined by the user for a specific test case; the answer key used to quantify accuracy of the calibrated model Calibration (edits) simulation output as a surrogate for measured data

9 Reproducibility! Advantages No specific, unique buildings of interest No faulty or unshared data used for calibration No variation in definitions or metrics No sole focus on simulation output Proliferation in calibration literature Necessarily unique Largely irreplicable Essentially incomparable

10 Limitations Cleanroom approach which has removed all realworld noise from the calibration process No: sensor drift, missing data, utility data measured at different times, unaccounted for occupancy/behavior changes, model/form uncertainty (but can allow study) Allows use of any weather file (TMY) For real-world application, you need AMY data No mapping of simulation output to measured data Temperature gradients: what point is Temp. of N wall? No sensor placement/material issues Test results equally weight all inputs/outputs, even though some matter more than others

11 Results IDF + CSV XML Thickness of metal siding? Calibrator: Between 0 and 0.5 and less than 1-B Oracle: 0.055

12 Website XML EPW CSV

13 Website/service

14 Results CV(RMSE)<30% NMBE<10% Exceeds G14!!! Metric Value Input error average Input error maximum Input error minimum 0.09 Input error variance CV(RMSE) CH4:Facility [kg](monthly) 9.95 CO2:Facility [kg](monthly) CO:Facility [kg](monthly) Carbon Equivalent:Facility [kg](monthly) Cooling:Electricity [J](Hourly) Electricity:Facility [J](Hourly) NMBE CH4:Facility [kg](monthly) CO2:Facility [kg](monthly) CO:Facility [kg](monthly) Carbon Equivalent:Facility [kg](monthly) Cooling:Electricity [J](Hourly) Electricity:Facility [J](Hourly) Electricity:Facility [J](Monthly) outputs

15 Purpose of this Study Are CV(RMSE) and NMBE the best metrics to use for calibration? What about no-cv: What about Mean Absolute Percent Error? What about (non-normalized) Mean Bias Error? What about Percent Absolute Error? maybe calibration using another metric would allow a calibration algorithm to reach lower input-side error (i.e. recover the real model of the building)

16 20,000 Building Calibration Study Restaurant Hospital Large Hotel Large Office Medium Midrise Primary Quick Office Apartment School Service #Inputs #Groups Secondary Small Hotel Small Office Stand-alone Super Strip Mall School Retail Market Warehouse TOTAL #Inputs #Groups

17 Results For the Strip Mall: If you use MAPE to minimize error to measured data, then you ll have the closest building match in terms of CV(RMSE)

18 Is there anything better? Output Variable Number of Buildings 5 InteriorEquipment:Electricity [J](Hourly) 10 7 InteriorLights:Electricity [J](Hourly) 9 Correlation to other properties showed that in most buildings (out of 15), calibrating to electrical usage of interior equipment and lights yielded better calibration results than any other building properties ASHRAE G14 extension to allow a tier-2 calibration using (increasingly feasible) submetering requirements would allow more accurate and useful models from calibration

19 Conclusions Trinity test allows replicable calibration studies and quantifies calibration performance An unsupported website and web service: A calibration study was conducted 20,000 calibrations, 15 DOE commercial buildings, each with calibrated groups CV(RMSE) and NMBE are as good as any of the proposed alternatives which is to say, BAD. Calibration to important subsets is proposed

20 Bibliography Energy Scalable tuning of building energy models to hourly data. Energy 84, ASHRAE Evolutionary tuning of building models to monthly electrical consumption. Transactions 119(2). Energy and Buildings Evaluation of weather data for building energy simulations. ENB 49(0), IBPSA Autotune E+ Building Models. IBPSA 37. NREL Building energy simulation test for existing homes (BESTEST-EX). NREL/TP ASHRAE ASHRAE Guideline 14, Measurement of energy and demand savings.

21 Questions? Joshua New