Session 3 Uncertainty Assessment Hand-on Training Workshop on National GHG Inventories - IPCC Cross-Cutting Issues 4-5-6 November 2015, Ankara Turkey
Introduction Perfect accuracy and certainty impossible Understanding and communicating uncertainties is an essential component of the GHG inventorying process. Use to help prioritize inventory improvement efforts
Topics Definitions of key terms and common causes of uncertainties Basis for uncertainty analysis and its key benefits Approaches to quantifying uncertainties Methods to combine uncertainties Strategies to reducing uncertainty
Definitions Random error Bias or Systematic error Accuracy and Precision Uncertainty Variability Confidence interval Probability density function Note: You do not need to memorize these terms, but you should have a good understanding of what they mean to follow the discussion in this lesson.
Definitions Accuracy: Agreement between the true value and the average of repeated measured observations or estimates of a variable. An accurate measurement or prediction lacks bias or, equivalently, systematic error. Precision: Agreement among repeated measurements of the same variable. Better precision means less random error. Precision is independent of accuracy.
Definitions Bias or Systematic error: Lack of accuracy. The mean or average of many separate measurements differs in a regular amount and direction from the actual value. It can occur because of failure to capture all relevant processes involved or because the available data are not representative of all real-world situations, or because of instrument error. Random errors: Random variation above or below a mean value. Random error is inversely proportional to precision. Usually, the random error is quantified with respect to a mean value, but the mean could be biased or unbiased. Thus, random error is a distinct concept compared to systematic error.
Definitions Variability: Heterogeneity of a variable over time, space or members of a population. Variability may arise, for example, due to differences in design from one emitter to another (inter-plant or spatial variability) and in operating conditions from one time to another at a given emitter (intraplant variability). Variability is an inherent property of the system or of nature. Uncertainty: Lack of knowledge of the true value of a variable that can be described as a probability density function (PDF) characterizing the range and likelihood of possible values. Uncertainty depends on the analyst s state of knowledge, which in turn depends on the quality and quantity of applicable data as well as knowledge of underlying processes and inference methods.
Definitions Confidence Interval: The true value of the quantity for which the interval is to be estimated is a fixed but unknown constant, such as the annual total emissions in a given year for a given country. The confidence interval is a range that encloses the true value of this unknown fixed quantity with a specified confidence (probability). Probability density function (PDF): The PDF describes the range and relative likelihood of possible values. The PDF can be used to describe uncertainty in the estimate of a quantity that is a fixed constant whose value is not exactly known, or it can be used to describe inherent variability. Throughout this lesson it is presumed that the PDF is used to estimate uncertainty, and not variability.
The Basics The emissions may be: Directly measured Estimated based on proxy data that is believed to accurately approximate emissions
Uncertainty types Scientific uncertainty: Arises when the science of the actual emission and/or removal process is not completely understood. Analyzing and quantifying scientific uncertainties is extremely problematic and is likely to be beyond the scope of project proponents and verifiers. Estimation uncertainty: Arises any time GHG emissions are quantified. Therefore all emissions or removal estimates are associated with estimation uncertainty.
Estimation Uncertainty Estimation uncertainty can be further classified into two types: Model uncertainty Parameter uncertainty
Model Uncertainty Associated with the mathematical equations (i.e., models) that characterize relationships between various parameters and emission processes. Due to the use of an incorrect mathematical model or inappropriate input into the model. Likely to be beyond the scope of GHG inventory team
Parameter Uncertainty Associated with quantifying the parameters used as inputs (e.g., activity data and emission factors) to estimation models Can be evaluated through statistical analysis, measurement equipment precision determinations, and expert judgment. Investigating and quantifying parameter uncertainties is the primary focus of inventory team Activity data and emission factors are parameters Types of parameter uncertainty Statistical (random) Systematic (bias)
Random! Statistical Uncertainty Where possible, can be identified through repeated sampling & subsequent statistical analysis For many parameters, it may not be possible to repeatedly sample or collect data under the same operating conditions Statistical uncertainty is generally assumed to average out over time Statistical uncertainty is the cause of noise in the data Statistical parameter uncertainty may be approximated by the rated precision of measurement equipment.
Statistical Uncertainty 40 35 30 25 20 15 True Values Measured Values 10 5 0
Systematic Uncertainty (Bias) Measured values consistently greater (or less) than the true value Biases cannot be detected through sampling of data and statistical analysis Biases can be identified through: Data quality investigations and other QA/QC measures Comparison of data with other independent datasets Biases do not average out over time, and therefore present a more serious problem than random uncertainties. Biases may increase (or decrease) over time if data quality falls (rises) and historical data is not able to be revised. Changes in biases over time are particularly troubling because of their impacts on emission trends and estimated emission reductions.
Systematic Uncertainty (Bias) 40 35 30 25 20 15 True Values Measured Values 10 5 0
Causes of Parameter Uncertainties Random errors in measurement devices (parallax error, temperature variation, etc.) Measurement devices can produce systematic biases Imprecise calibration, faulty measurement equipment, environmental factors, operator error, double counting, omission of data, etc. Parameters may also be based on non-representative samples Fuel samples taken monthly, while fuel deliveries occur weekly. Data does not account for process start-up and shut-down conditions, or irregular operating conditions.
Reporting and Combining Parameter Uncertainties Qualitative High, Medium, Low Semi-Quantitative Combine uncertainty values using ranking schemes (e.g., Data Attribute Reporting Systems) Quantitative models Error propagation (assumed data distributions are normal & all parameters are independent) Monte Carlo (complete flexibility)
Confidence interval Typical to use a 95% confidence interval 95% probability of enclosing the true, but unknown, value Enclosed by the 2.5th and 97.5th percentiles of a probability density function (PDF)
Example: Symmetric uncertainty PDF is a normal (symmetric) distribution Mean value is 1.0 2.5 th percentile of uncertainty is 0.7 97.5 th percentile of uncertainty is 1.3 1.0 ±30%, at a 95% probability range.
Example: Asymmetric uncertainty Asymmetric about the mean value of 1.0 2.5 th percentile of uncertainty is 0.5 97.5 th percentile of uncertainty is 2.0 1.0-50% to +100%.
Biases are often unknown Difficult to capture systematic errors (biases) because you are often not aware of them Biases that are known can be corrected for Biases may arise from: imperfections in the design of the GHG inventory (assumptions, methods selected etc.) simplifications and assumptions in calculation models used measurement techniques
Good practice for uncertainty analysis Potential causes of uncertainty that are not quantified should be described, particularly with respect to the design of the GHG inventory, models, and data and to make an effort to quantify them in the future
Collecting Uncertainty Information Parameter uncertainty information may be collected from: Calibration records for measurement devices Measurement equipment precision values supplied by equipment manufacturers in their manuals or other technical documentation Statistical analysis techniques where it is possible to make repeated measurements Scientific publications Expert elicitation While much focus is given to quantifying uncertainties, the qualitative explanation of the causes of uncertainties are more important! 25
Uncertainty Information Increasing uncertainty Census Survey Empirical data Expert judgment Complete data. If well designed, it should have small errors Data based on sampling. Errors should be quoted or determined Should have quoted errors derived from measurements Experts should give range of possible value or mean and uncertainty
Expert judgement Elicitation protocol is key to generating uncertainty estimates that are credible and defensible Includes procedures and techniques to motivate and condition the experts The 2006 IPCC Guidelines provide help
Key elements of an elicitation protocol Motivating: Establish a rapport with the expert, describe the context of the investigation, explain the most commonly occurring biases. Structuring: Clearly define the quantities for which judgements are to be sought. (e.g., resulting emissions or removals should be for typical conditions averaged over a one-year period). Conditioning: Work with the expert to identify and record all relevant data, models, and theory relating to the formulation of the judgements. Encoding: Request and quantify the expert s judgement on the value of the selected parameters. While doing this request information (quantitative and qualitative) on what they judge the uncertainties in these values. Verification: After you have analyzed input from experts, provide them feedback to confirm that you have properly encoded and interpreted their judgments. This process also gives the experts a chance to consider if they want to add anything to their answers. Document all information provided by experts.
Approaches to combining uncertainties Parameter uncertainties combined to provide: Uncertainty estimates for the emission or removal category Uncertainty estimates for the entire inventory in any year and The uncertainty in the overall inventory trend over time. 2006 IPCC guidelines include two approaches Approach 1: Error propagation Approach 2: Monte Carlo simulation The bulk of your effort and resources should be on improving inventory estimates rather than quantifying uncertainty.
Approach 1: Error propagation The same equation to estimate emissions is used to combine uncertainties for each category Assumptions and Requirements All parameter uncertainties are symmetric and in the form of a normal (Gaussian) probability distribution Is best applied where uncertainties are less than ±30%
Error propagation (Product Rule)
Error propagation (Summation Rule)
Error propagation (Summation Rule) Equations for calculating total emissions (E T ) for source category 1 (E 1 ) and 2 (E 2 ) E T = E 1 + E 2 or E T = E 1 -E 2
Uncertainty in trend - Sensitivities Emissions/removals estimates for base year and current year Type A sensitivity: the change in the difference in overall emissions between the base year and the current year, expressed as a percentage, resulting from a 1% increase in emissions or removals of a given category and GHG in both the base year and the current year. Primarily relevant for emission factors. Type B sensitivity: the change in the difference in overall emissions between the base year and the current year, expressed as a percentage, resulting from a 1% increase in emissions or removals of a given category and GHG in the current year only. Primarily relevant for activity data. Type A sensitivities: Uncertainties that are fully correlated between years Type B sensitivities: Uncertainties that are not correlated between years
Worksheet for uncertainty calculations using Approach 1 Input data
Keep in mind Approach 1 (error propagation) appropriate if Uncertainties are small (standard deviation/mean less than 0.3) Symmetric Uncorrelated If uncertainties are larger will tend to underestimate uncertainty
Approach 2: Monte Carlo simulation Simulation technique achieved by running multiple trials using randomly sampled inputs from a user defined distribution of values. This distribution for each input variable represents the uncertainty in that variable. Appropriate for estimating uncertainty when: Uncertainties of the individual components are large The distributions are asymmetric or not normal (non-gaussian) Correlations occur between some of the activity data sets, emission factors, or both Uncertainties are different for different years of the inventory.
Requirements for Approach 2 Monte Carlo Specify a probability density function, or PDF, for each input variable that reasonably represents the uncertainty in that variable. Obtained using statistical analysis or expert elicitation Based upon representative range of factors such as seasons and geography. Monte Carlo can deal with PDFs of any shape and width, as well correlations (both in time and between source/sink categories). Can deal with more complex models (e.g., the first order decay for CH 4 from landfills).
Illustration of steps for Monte Carlo simulation Step 1 Step 2 Select random value for AD Select random value for EF Select random value for AD Select random value for EF Step 3 Estimate emissions Add up all emissions Estimate emissions Step 4 Store values Calculate mean and uncertainty Mean and distribution constant? If Yes Finished If No Go to Step 2
Example of Monte Carlo results Results of a Monte Carlo simulation after: 1 iteration 50 iterations 100 iterations 1,000 iterations 10,000 iterations
Example of Monte Carlo results (1 iteration) Frequency 1,2 1 0,8 0,6 0,4 0,2 0 0,4 0,45 0,5 0,55 0,6 Emissions (Gg CO 2 eq)
Example of Monte Carlo results (50 iterations) Frequency 20 18 16 14 12 10 8 6 4 2 0 0,4 0,45 0,5 0,55 0,6 Emissions (Gg CO 2 eq)
Example of Monte Carlo results (100 iterations) Frequency 30 25 20 15 10 5 0 0,4 0,45 0,5 0,55 0,6 Emissions (Gg CO 2 eq)
Example of Monte Carlo results (1,000 iterations) 250 200 Frequency 150 100 50 0 0,4 0,45 0,5 0,55 0,6 Emissions (Gg CO 2 eq)
Example of Monte Carlo results (10,000 iterations) 2500 2000 Frequency 1500 1000 500 0 0,4 0,45 0,5 0,55 0,6 Emissions (Gg CO 2 eq)
Choosing an approach Where the conditions for applicability are met, Approach 1 and Approach 2 will give the same results. Approach 1 is less effort to apply. Although once Monte Carlo software is set up, it can be easy to apply Approach 2. But set up can be a great deal of effort. Approach 2 will provide a better results in terms of the thoroughness of the uncertainty estimates.
Reporting and documentation Uncertainty documentation should address for each parameter: Causes of uncertainty What uncertainties were quantified Source of any data used as a basis to estimate uncertainty Methods used to combine uncertainties Background information for expert judgment analysis Explanation of any correlation found between inputs Explanation of any special consideration unique to the country/inventory Explanation of differences in results between Approaches 1 and 2
Using and interpreting uncertainties
Uncertainty assessment in practice In the context of national GHG inventories. Who are the users of uncertainty information? How do you collect uncertainty information? How can and should they use it? How do you communicate the results of your uncertainty assessment work to stakeholders?
Users
Users of uncertainty information Uncertainty information provided by a Party under the UNFCCC may be used by: Scientific and research community Policy makers and negotiators Public and other stakeholders IPCC and other methodology developers UNFCCC/KP Expert Review Teams Inventory compilers
Stakeholders in uncertainty data What do these groups want to use uncertainty information for? How and should these groups use and interpret GHG inventory uncertainty information?
Scientific and research community Wants Statistically valid and detailed uncertainty values for all variables May assume uncertainty estimates have origins similar to experimental data Reality Need to understand the role expert judgment and elicitation play Potential for biases in uncertainty data between categories and countries
Policy makers and negotiators Wants Simple to understand assessments of data quality. ( Can we trust it? ) To use quantitative uncertainty estimates as an eligibility or enforcement tool Reality Uncertain estimates should in most cases NOT be used for policy making (we will discuss)
Public and other stakeholders Wants Enormously diverse Often don t care or understand Reality Need to educate as to the difference in uncertainties at national level versus at a project or facility level.
IPCC and other methodology developers Wants Basis for assessing quality of methods (methodological uncertainty) and factors (parameter uncertainty for guidelines and EFDB Reality Need to understand how uncertainty estimates were developed what types of uncertainties included and excluded And what are the causes of the uncertainties
UNFCCC / Expert Review Teams Wants Data consistent with reporting requirements Tool to assess rigor of data quality management process Reality Goals realistic, as long as they follow UNFCCC reporting guidelines that clearly indicate that uncertainty data should not be used to compare across Parties.
Inventory compilers Wants Data suppliers: feedback on their data quality Inventory managers: quality management metric and tool Reality Wants are often realistic, as along as it is understood that uncertainty estimates may not be comparable
Collection of information
Parts of the uncertainty assessment process 1. The rigorous investigation of the likely causes of data uncertainty and quality 2. the creation of quantitative uncertainty estimates and parameter correlations for each variable 3. the mathematical combination of estimates using a statistical model (e.g., first-order error propagation or Monte Carlo) 4. the selection of inventory improvement actions in response to results of uncertainty assessment 5. Communication of uncertainty assessment results
Where to focus? We tend to focus on parts 2 and 3 (2) creation of uncertainty estimates and (3) their mathematical combination Often little attention is given to parts 1 and 4 (1) the investigation of causes and (4) inventory improvements Part 1 is best thought of as detective or investigatory work, results are largely qualitative Part 5, communication, is often done poorly, given the potential for misuse of results by stakeholders
Org chart for the process Overall Inventory lead Uncertainty assessment coordinator QA/QC Officer Source/sink category lead Source/sink category lead Source/sink category lead Source/sink category lead Outside expert
Roles and coordination Inventory Lead: overall director responsible for supervising the uncertainty assessment for the entire Inventory and communicating results
Roles and coordination Source Category Leads: responsible for making decisions and performing uncertainty assessment on their specific source or sink categories. determining the appropriate level of disaggregation for data collection and uncertainty model development, prioritizing the variables for input data collection efforts and allocating resources to collect uncertainty information, identifying experts for elicitation reviewing results of uncertainty analysis and identifying corrective actions and improvements
Roles and coordination The Uncertainty Analysis Coordinator: responsible for directing the assessment of uncertainty for entire Inventory obtaining data inputs eliciting expert judgments with Source Leads developing the uncertainty model developing quantitative uncertainty estimates Ensuring that qualitative information on causes of uncertainty is documented interpreting the results of the uncertainty analysis
Roles and Coordination QA/QC Officer directs the overall implementation of QA/QC supervising QA/QC staff overseeing the expert reviews ensuring the full and adequate implementation of QA/QC elements adequate qualifications of source category staff and contractors
Roles and Coordination Outside Experts independent individuals who may contribute data to the inventory estimation (i.e., data suppliers ), may be involved in improving / examining inventory methods, data, or report may serve as expert for elicitation purposes
What is uncertainty information? We often focus on the number ±9% But uncertainty information has qualitative as well as quantitative components. Although we focus on the quantitative, it is the qualitative information that is often the most useful. Do we care more about the number or about why the uncertainty is there in the first place?
Quantitative uncertainty information Types of data Distribution Standard deviation or standard error Their associated confidence intervals Upper and lower bound of variables and their associated cumulative probability levels In many cases, however, little or none of this quantitative uncertainty data is available for a variable.
Qualitative uncertainty information Descriptions of the causes or likely causes of uncertainties Understanding of how uncertainties related to data collection process References for uncertainty information Publications Background and qualifications of experts elicited Elicitation protocols
Setting up your uncertainty model: identifying variables The uncertainty estimation methodology based on inventory estimation equations Equations may be identical to inventory estimation methodology, or Equations may be a simplified version of the inventory estimation methodology The mathematical models underlying both inventory and the uncertainty methodologies must yield the same emission estimates. Levels of aggregation for variables are likely to be the main difference between the inventory and the uncertainty estimation methodologies.
Setting up your uncertainty model: identifying variables Uncertainty estimation methodologies tend to have fewer variable components (i.e., are less disaggregated). The level of variable disaggregation for uncertainty estimation for each variable in every source and sub-source category should be determined by: the availability of uncertainty data inputs, resource availability and the importance of the source/sub-source category (e.g., if it is a key category).
Collecting information: variable by variable Uncertainty data collected at the individual variable level (e.g., EF, activity data) The source category lead must work with uncertainty coordinator to define the appropriate level of disaggregation for uncertainty model Example: Cattle manure may be estimated by province. But one equation at a national level is used for the uncertainty model A single uncertainty value is then needed for the manure EF, and Monte Carlo model is simplified
Elicitation If published uncertainty data insufficient, then expert elicitation will likely be necessary Types of elicitation Informal interview : eliciting uncertainty information from the inventory experts that are directly involved in the inventory process (e.g., source category leads, contractors, etc.) Expert elicitation : formally eliciting uncertainty information from outside experts
Elicitation Informal interview is a modest process of elicitation, involving discussions with inventory experts that are intimately familiar with the inventory source category.
Elicitation A less formal elicitation or informal interview may be warranted where: the emission source is not a key inventory source category and so is a small contributor to inventory emissions only a small number of variables for the category have missing data knowledgeable outside experts on the underlying variables are unavailable resources for conducting a formal elicitation are limited
Integration with QC Tier 2 source-specific QC checks and investigations into input data quality and uncertainty investigations require contacting the same persons or organizations. QC and uncertainty assessment processes should be integrated! Investigations should be done as one Avoid duplicative questions and multiple points of contact.
When to collect? Uncertainty information may not need to be collected every year for every source or variable A plan should be developed specifying frequency Uncertainty and Tier 2 QC investigations should have one timeline Key categories prioritized in terms of immediacy and frequency Some variables may only need to be revisited every few years For each variable, the year in which uncertainty information is collected must be documented.
Trend uncertainty Estimate bias may not be constant from year to year It may exhibit a pattern (e.g., growing or falling) For example, if your data supplier cuts his budget for data collection, then the data he provides you may be getting worse over time. Assuming biases in estimates cancel out when estimating trend uncertainty, may be incorrect
Using uncertainty information
How should you use uncertainty information? National inventory users and stakeholders want to apply uncertainty information in a number of ways Two main types of applications 1. Comparative 2. Quality management Each application requires uncertainty information to have certain characteristics
Comparative applications Policy makers (as well as some other stakeholders) want to use uncertainty information to make decisions or support compliance systems. To compare: Countries Source categories Sectors Facilities Projects, etc.
Comparative applications For uncertainty information to be used for international compliance applications it must: 1. Be comparable across countries 2. Be relatively objective and subject to review and verification 3. Not be subject to gaming by countries acting in their own self-interest 4. Be administratively feasible to estimate 5. Be of high enough quality to warrant the compliance costs imposed on countries (e.g., through adjustments) 6. Attempt to address all types of uncertainty National GHG inventory uncertainty estimates in most cases do not have these characteristics!
Comparative applications Quantitative uncertainty estimates are often based on expert judgments Only through incredibly rigorous elicitation can you avoid significant subjectivity in expert judgment They are unlikely to be sufficiently comparable across countries, categories, parameters, or time, because of differences across the experts.
Why can t I compare? Expert judgments do not currently undergo any rigorous review Without a costly and rigorous review process, they will be ripe for manipulation and gaming It is currently impractical to ensure that uncertainty estimates across Parties are done in a consistently objective fashion. UNFCCC inventory review process is already enormously strained
Why can t I compare? The uncertainty in uncertainty estimates is typically vastly greater than the uncertainty in the inventory estimate itself.
Quality management One goal for inventory compilers is continuous improvement of emission and removal estimates. From a quality management perspective, uncertainty assessment is a structured way to investigate, conceptualize, and track data quality. From this perspective, uncertainty assessment is just part of your QA/QC program
Quality management Quantitative uncertainty estimates can be useful But in isolation, they do not provide information needed to isolate the causes of data quality problems so they can be corrected. An investigation focused approach to uncertainty assessment focuses on Parts 1 and 4 of the process (investigation and improvement). This approach also provides more verifiable information and justifications to support quantitative uncertainty estimates.
Quality management An investigation-focused approach requires that you work closely with data suppliers to: 1. Exchange information on the inventory s data quality requirements and data collection practices 2. Identify and understand activity data reporting and collection problems 3. Identify situations where there is a lack of empirical data for emission factors or other parameters 4. Identify situations where the variability in an inventory parameter is high 5. Identify situations where there is a lack of scientific consensus of the appropriate estimation method for an inventory parameter or category 6. Identify specific actions that can be taken to correct or mitigate problems
Quality management By working jointly with data suppliers you can Educate them on your needs Help them identify specific causes of uncertainty and the magnitude of their effect on data quality Create pressure for investments in data quality improvements (e.g., expanded data collection or more research)
Quality management The required characteristics of uncertainty information are less strict if they are only used prioritizing inventory improvements It is less critical that uncertainty estimates be objective and comparable because they do not have compliance implications
Summary You do not have to choose between an investigation-focused and Monte Carlo type uncertainty assessment approach. The former will obtain better results for the latter. However, where resources are limited, inventory quality will likely benefit the most if resources are focused on data quality investigations and improvements (Parts 1 and 4), rather combining subjective uncertainty estimates
Summary We wrongly assume that uncertainty estimation will automatically lead to inventory quality improvements In reality, we need a process designed to investigate and assess the causes of poor quality And a feedback process that leads to the implementation of measures for improvements.
Communication
Communicating uncertainty information A GHG inventory is not a purely scientific exercise. We acknowledge this by aggregating estimates of varying uncertainties without regard to rules of significant figures. It also has accounting and legal compliance characteristics Do accountants report their profit and loss statements with a ±35%? Most stakeholders will only use or pay attention to the point estimates in an inventory. When looked at, quantitative uncertainty estimates are, more often that not, misused
Communicating uncertainty information When communicating uncertainty information, focus on explaining the causes of uncertainties in key underlying variables, rather than quantified uncertainty estimates. Always include discussion of what measures are being taken to monitor and improve data quality If you must discuss aggregate quantitative uncertainty estimates, focus on the uncertainty in the trend, rather than annual totals
- son -
Exercise: Approach 1 - Propagation of error You have estimated emissions (in Gg CO 2 -eq) from a number of activities For this example we will consider 4 categories and different GHGs for two years: 2000 and 2010 You have obtained uncertainty values for the activity data and emission factors used Your data are listed in a table IPCC category Gas 2000 emissions 2010 emissions Activity data uncertainty Emission factor uncertainty 1.A.3b Road transportation Gasoline CO 2 4595 6089 ±2% ±2% 1.A.3b Road transportation - Gasoline CH 4 10 29 ±1% ±50% 2.1 Nitric acid production N 2 O 1595 1396 ±5% ±100% 3.A.2 Manure management N 2 O 539 422 ±15% ±160%
Prepare your data First, calculate the total emissions for both years of the inventory. IPCC category Gas 2000 emissions 2010 emissions Activity data uncertainty Emission factor uncertainty 1.A.3b Road transportation Gasoline CO 2 4595 6089 2% 2% 1.A.3b Road transportation - Gasoline CH 4 10 29 1% 50% 2.1 Nitric acid production N 2 O 1595 1396 5% 100% 3.A.2 Manure management N 2 O 539 422 15% 160% Total 6739 7936 Next, ensure that you use the absolute values of the uncertainties for both the activity data and the emission factors.
Step 1: Combine the uncertainties For each category combine uncertainties of the activity data and emissions factors. Column G is the combined uncertainty by category derived from the data in Columns E and F using the error propagation equation (Equation 3.2). The entry in Column G is the square root of the sum of the squares of the entries in Columns E and F. IPCC category G a s 200 0 emi ssio ns 201 0 emi ssio ns Activ ity data unce rtaint y Emis sion facto r unce rtain ty Com bine d unce rtain ty A B C D E F G= ( E 2 +F 2 ) 1.A.3b Road transportation Gasoline 1.A.3b Road transportation - Gasoline C O 2 C H 4 459 5 608 9 2% 2% 3% 10 29 1% 50% 50% 2.1 Nitric acid N 159 139 5% 100% 100%
Step 2: Calculate the percentage uncertainty for the latest year 1. For each category, calculate the uncertainty in Column G as a percentage of total emissions in the latest year (2010) (also referred to as contribution to the variance ) and record the value in Column H. The entry in each row of Column H is the square of the entry in Column G multiplied by the square of the entry in Column D, divided by the square of total at the foot of Column D. 2. Calculate the total contribution by summing the entries in Column H. For our example, the total is ΣH=0.0388. 3. Take the square root of the sum to estimate the uncertainty in the total inventory in the latest year. For our example, the total % uncertainty for the year 2010 is 19.7%. IPCC category Gas 2000 emission s 2010 emission s Activity data uncertainty Emission factor uncertaint y Combined uncertainty A B C D E F G= (E 2 +F 2 ) Contributio n to the variance H= (G D) 2 /(ΣD) 2 1.A.3b Road transportation Gasoline 1.A.3b Road transportation - Gasoline CO 2 4595 6089 2% 2% 3% 0.0005 CH 4 10 29 1% 50% 50% 0.0000 2.1 Nitric acid production N 2 O 1595 1396 5% 100% 100% 0.0310 3.A.2 Manure management N 2 O 539 422 15% 160% 161% 0.0073 Total 6739 7936 0.0388
Step 3: Estimate the trend uncertainty To determine uncertainties in the trend involves the following intermediate steps: 1. Estimate the Type A sensitivity 2. Estimate the Type B sensitivity 3. Estimate trend uncertainty due to EF uncertainty 4. Estimate trend uncertainty due to AD uncertainty 5. Combine the results in steps 3. and 4. above to estimate total trend uncertainty Notes: The Type A and Type B sensitivities are intermediate variables that simplify the calculation procedure. The results of steps 3. and 4. by themselves do not provide uncertainty information for specific categories. The results of these steps combined (step 5.) provide an indication of the trend uncertainties.
Step 3.1: Estimate the Type A sensitivity We first calculate the Type A sensitivity for each category as the percentage difference in emissions between the base year and the current year in response to a 1% increase in category emissions/removals in both the base year and the current year. Type A shows the sensitivity of the trend in emissions to a systematic uncertainty in the estimate. The equation for this is: Absolute value of: 0.01 D x + ΣD i (0.01 C x + ΣC i ) 100 (0.01 C x + ΣC i ) ΣD i ΣC i ΣC i 100 Where: Cx, Dx = column C or D value in category x ΣCi, ΣDi = Sum of column C or D values over all categories (rows) of the inventory For example, for Category 1.A.3b the equation Type A sensitivity value is calculated as follows: 0.01 6089 + 7936 (0.01 4595 + 6739) (0.01 4595 + 6739) 100 7936 6739 6739 100 = 0.0999
Step 3.1 (cont): Estimate the Type A sensitivity Performing the same calculation for all categories, you will get the following values for Type A sensitivity. IPCC category Gas 2000 emissions 1.A.3b Road transportation Gasoline 1.A.3b Road transportation - Gasoline 2010 emissions Activity data uncertainty Emission factor uncertainty Type A sensitivity A B C D E F I CO 2 4595 6089 2% 2% 0.0999 CH 4 10 29 1% 50% 0.0026 2.1 Nitric acid production N 2 O 1595 1396 5% 100% 0.0714 3.A.2 Manure management N 2 O 539 422 15% 160% 0.0315 Total 6739 7936
Step 3.2: Estimate the Type B sensitivity In Column J of the worksheet you can calculate the the Type B sensitivity as the percentage difference in emissions between the base year (2000) and the latest year (2010) in response to a 1% increase in category emissions/removals in the 2010 only. This shows the sensitivity of the trend in emissions to random error in the estimate. IPCC category Gas 2000 emissions 2010 emissions Activity data uncertainty Emission factor uncertainty Type A sensitivity Type B sensitivity A B C D E F I J= Abs(D/ΣC) 1.A.3b Road transportation Gasoline 1.A.3b Road transportation - Gasoline CO 2 4595 6089 2% 2% 0.0999 0.9035 CH 4 10 29 1% 50% 0.0026 0.0043 2.1 Nitric acid production N 2 O 1595 1396 5% 100% 0.0714 0.2072 3.A.2 Manure management N 2 O 539 422 15% 160% 0.0315 0.0626 Total 6739 7936
Step 3.3: Estimate the trend uncertainty due to emission factor uncertainty In Column K of the worksheet you multiply together the values in Columns I and F to estimate the uncertainty introduced into the trend in emissions by emission factor uncertainty, under the assumption that uncertainty in emission factors is correlated between years. If you determine that the emission factor uncertainties are not correlated between years then the Type B sensitivity (Column J) should be used in place of that in Column I and the result multiplied by the square root of 2. IPCC category Gas 2000 emissions 2010 emissions Activity data uncertainty Emission factor uncertainty Type A sensitivity Trend uncertainty due to EF uncertainty A B C D E F I K= I F 1.A.3b Road transportation Gasoline 1.A.3b Road transportation - Gasoline CO 2 4595 6089 2% 2% 0.0999 0.20% CH 4 10 29 1% 50% 0.0026 0.13% 2.1 Nitric acid production N 2 O 1595 1396 5% 100% 0.0714 7.14% 3.A.2 Manure management N 2 O 539 422 15% 160% 0.0315 5.05%
Step 3.4: Estimate the trend uncertainty due to activity data uncertainty In Column L of the worksheet you multiply together the information in Columns J and E to estimate the uncertainty introduced into the trend in emissions by activity data uncertainty, under the assumption that uncertainty in activity data is not correlated between years. If you determine that the activity data uncertainties are correlated between years then the Type A sensitivity value (Column I) should be used in place of that in Column J and the square root of 2 factor does not then apply. IPCC category Gas 2000 emissions 1.A.3b Road transportation Gasoline 1.A.3b Road transportation - Gasoline 2010 emissions Activity data uncertainty Emission factor uncertainty Type B sensitivity A B C D E F J= Abs(D/ΣC) Trend uncertainty due to AD uncertainty L= J E 2 CO 2 4595 6089 2% 2% 0.9035 2.56% CH 4 10 29 1% 50% 0.0043 0.01% 2.1 Nitric acid production N 2 O 1595 1396 5% 100% 0.2072 1.46% 3.A.2 Manure management N 2 O 539 422 15% 160% 0.0626 1.33% Total 6739 7936
Step 3.5: Estimate the total trend uncertainty Estimating the total trend uncertainty involves the following: 1. In Column M you calculate the uncertainty introduced into the trend in total emissions by the category in question. This is derived from the values in Columns K and L using Equation 3.2. The entry in Column M is the sum of the squares of the entries in Columns K and L. 2. The estimate of the total uncertainty in the trend is then calculated from the entries in Column M by using the error propagation equation. This total is obtained by summing the entries in Column M (ΣM=0.87%) and then taking the square root of the sum (9.32% for our example). The uncertainty in the trend is a percentage point range relative to the inventory trend. In our example, the 2010 emissions are 18% percent greater than the 2000 emissions (from 6739 in 2000 to 7936 in 2010), and since we have estimated the uncertainty as 9.32%, then the trend is 18% ±9.32% (or from about 9% to 27% increase) for the 2010 emissions relative to the 2000 emissions. IPCC category Gas 2000 emissions 2010 emissions Activity data Emission factor uncertainty Trend uncertainty Trend uncertainty Total trend uncertainty