Guideline for Performing Cost Benefit and Sustainability Analysis of Remedial Alternatives. Instructions for the Excel based Tool (Version 2.

Transcription

1 Guideline for Performing Cost Benefit and Sustainability Analysis of Remedial Alternatives Instructions for the Excel based Tool (Version 2.0) 1

2 Documentation Contents 1. Purpose and Background 2. Structure 3. Home 4. Setup 5. Cost Benefit Analysis (CBA) 1. Data Entry 2. CBA Results 6. Multi Criteria Analysis (MCA) 1. Data Entry 2. Weighting Concepts 3. Weighting 4. MCA Results 7. Summary Results 8. Troubleshooting Slide No CB&SA Tool Instructions v2.0 2

3 CRC CARE CBS&A Tool 1. Purpose and Background 3

4 Tool Purpose This tool is intended as an aid to conducting a CB&SA. CB&SA is a scalable, flexible, and iterative process that can be used for small or large, simple or complex, short term or long term projects or programs. The CB&SA can be applied at various stages of the decision making process and undertaken at varying levels of detail and rigour. The level of detail and rigour required will be determined by the practitioner based on the considerations of a range of factors, including: The size and complexity of the project; Available budget; Information requirements; Time constraints; and The extent to which risk and uncertainty may affect the selection of a preferred option. The CB&SA allows for different levels of effort and it will be at the practitioners discretion to determine the level of effort that is appropriate. CB&SA Tool Instructions v2.0 4

5 CRC CARE CB&SA Tool 2. Structure 5

6 CRC Care Tool Structure Cost Benefit Analysis Multi Criteria Analysis Home and Setup CBA Inputs CBA Results MCA Inputs MCA Results Summary Results Calculations 4 Sheets - Home navigation - Preliminary screening - Name project - Identify options - Specify costs and benefits - Specify indicators - Select participants for weighting Sheet Color Code 1-15 Sheets - One inputs sheet per option, up to a maximum of 15 options - Input costs and benefits for each option 2 Sheets - Total costs and benefits for each option - Benefit-Cost Ratio and Net Present Value for each option, at base, high, and low discount rates 5 Sheets - Sub-indicator ranges and units of measure - Score each option on each sub-indicator - Assign weights to each subindicator 4 Sheets - Summary scores overall and for each indicator group - Graphic comparison of MCA scores - Sensitivity results for each weighting participant 3 Sheets - Value-Cost Scatter graphic (if CBA results are not weighted in MCA) - Combined CBA- MCA results in table and graphic form - Test of results sensitivity to CBA weight 2 Sheets - Calculations of MCA results with consensus and individual weights - Calculations of MCA score with sensitivity to weight of cost Home and Setup CBA inputs MCA inputs Summary Results Calculations CBA results MCA results CB&SA Tool Instructions v2.0 6

7 CRC CARE CB&SA Tool 3. Home 7

8 Home To Start: First, save the tool with an appropriate file name. When you open the file you will probably see a button here that says Enable Content. If so, click this button to activate macro that will setup the model in response to your checked preferences. (If not, macros should be automatically activated). The Home sheet provides navigation links to various locations in the Tool. The user can decide to navigate via the worksheet tabs in Excel or the navigation links above, as preferred. Click this button if you wish to over-ride the worksheet and cell protections that protect against data entry and spreadsheet modification errors. Click this button to re-enable protection once modification is complete. CB&SA Tool Instructions v2.0 8

9 CRC CARE CB&SA Tool 4. Setup 9

10 Preliminary Screening Throughout the Tool, user inputs will be indicated by light yellow cells, as shown on this sheet. The PrelimScreen sheet provides a way to pre-screen options and determine whether there is sufficient cause to complete the full CBS&A analysis. Note: If a preliminary screening of the project and options has already been completed, the user my bypass this screen and advance directly to Setup1. 1. Enter a name for your project. This name is used on other sheets in the tool via a link to this cell. 2. Identify key threshold indicators. 4. For each threshold indicator and option identified, select Y, N, or?. 3. In this row, enter titles for up to 15 options to be screened. Note, maximum of 50 characters allowed for each title Once Preliminary Screening has been completed, the Continue to Setup 1 button will advance the user directly to the next sheet in the analysis process. CB&SA Tool Instructions v2.0 10

11 Setup 1 1. If you did not complete the Preliminary Screening sheet, you must enter a name for your project here. This name is used on other sheets in the tool via a link to this cell. 5. Place a check in the box and, if not completed on the Preliminary Screening sheet, enter a name for each option in your analysis here. The Next button, located on the bottom centre of each setup sheet, is a navigation aid that takes the user to the next setup sheet. The Home button is located on the upper left side of each sheet in the Tool. This button takes the user directly back to the home navigation sheet from anywhere in the Tool. 2. Enter the first year of the analysis. For example, if the project begins in 2017, enter 2017 into the box. Entry here must be a year. 3. Enter the lifetime of the project for your cost benefit analysis. Entry must be between 1 and 100 years. 4. Specify a discount rate for the cost benefit analysis (in percent). In addition to a base discount rate, select a higher and lower bound discount rate for sensitivity analysis. CB&SA Tool Instructions v2.0 11

12 Setup 2 2. Specify the non-monetary indicators to be used in your MCA analysis. First, select the main indicator types to be included in your analysis. The tool is designed to include Social, Environmental, and Economic indicators, with room for one additional main indicator type. It is highly recommended that you include Social, Environmental, and Economic indicators in your analysis. The tool has preselected each of these main indicator types. 1. Specify the costs and benefits to be considered in your CBA analysis. For each cost and benefit, click on (place a check mark) into the check box in column D and enter a name into the input field in column F. Each main indicator type can accommodate up to 20 sub-indicators. It is required that a minimum of one sub-indicator is selected for each main indicator selected. The tool is designed to accommodate multiple subindicators per main indicator category. Indicator weights and scores are to be provided for sub-indicators only. Once main indicator types are confirmed, select the preferred number of sub-indicators for each main indicator type by checking the box and entering a name for each in column L, P, T, or X. CB&SA Tool Instructions v2.0 12

13 Setup 3 1. In Step 1, select one of the options for incorporating the CBA results into the MCA analysis as described. See the Section 5.5 of the guidelines for a discussion of why one of or the other approach might be appropriate for your analysis. Note that testing the sensitivity of the MCA to CBA weight is available for both selections. However, the V-C Scatter diagram is only available if Do not assign a weight to the CBA results is selected. 2. Select which CBA result metric to use in your MCA analysis. By default, the model is set to use the net present value of benefits. 3. Identify the individual (or group) that will provide weights in your MCA analysis. (Note, in practice these individuals may not be known during the model set-up phase: it is acceptable to complete this Step later in the analysis e.g., after the scoring of options for each indicator is complete.) For each weight to be provided, check the box and enter both a full name and initial. Full names are limited to 50 characters, and initials are limited 3 characters. CB&SA Tool Instructions v2.0 13

14 CRC CARE Tool 5.1 CBA Inputs 14

15 Entering Data for Indicators A CB_Op# sheet will be available for each option selected on Setup1. The data entry shown below should be completed on a separate sheet for each option. The sheet below shows year 1 to 11 inputs on the CB_Op2 sheet for Option 2 s CBA inputs. Note: The tool will automatically provide rows for the benefits and costs specified by the user in the Setup 2 sheet. Input annual costs and benefits to included in the CBA analysis. Yellow cells indicate where to enter data. As shown is this example, there are zero benefits for to include in the CBA analysis for option 2, and all costs occur in Note: The CBA inputs will be directly entered into this sheet by the user. Users should prepare backup documentation to clarify the sources and calculations used as a basis for the inputs. That documentation should be referenced in any reporting of CB&SA results. CB&SA Tool Instructions v2.0 15

16 CRC CARE CB&SA Tool 5.2 CBA Results 16

17 CBA Results The CB_results sheet provides a summary of the CBA results and benefit and cost summary inputs by option. Results are shown for two metrics: the Net Present Value of Benefits, and the Benefit- Cost Ratio for the base discount rate, and at the upper and lower discount rates as entered in the Setup1 sheet for sensitivity analysis. Note that in this simple example, no benefits have been identified in the CBA analysis, making the BCR for each option zero. If your analysis has no monetized benefits, NPV should be used in analyzing CBA results. CB&SA Tool Instructions v2.0 17

18 CBA Chart The CB_charts sheet provides a graphical comparison of the results metrics shown on the CB_results sheet. The upper bar graph shows the NPV of Benefits for each Option, and the lower bar graph shows the Benefit-Cost Ratio of each Option. Note in this example no results are shown in the BCR graphic. This is because no benefits have been identified for any of the options, making the BCR for each option zero. This dropdown menu can be used to select the data displayed in the results charts. The user can select to display results at base, high or low discount rates, or show all rates at once for comparison. Option NPV and BCR at base discount are shown in green, high discount rate results in yellow, and low discount rate results in blue. CB&SA Tool Instructions v2.0 18

19 CRC CARE Tool 6.1 MCA Inputs

20 Entering Data for Indicators The MCA_inputs sheet is where the user establishes the scale that will be used as the basis for scoring each non-monetary indicator. Note here if preferred direction for indicator is high or low (i.e., is it good that the indicator is high or low?). For any new indicator, specify the units of measure for that indicator here. As noted in Section 5.1, users should provide off-line, verbal documentation of scale endpoints for qualitative scales (i.e., write in words what is meant by a 1 and what is meant by a 5. CB&SA Tool Instructions v2.0 20

21 Entering Data for Indicators On the MCA_scoring sheet, the user must score each option on the selected sub-indicators. Data for each sub-indicator is entered as shown below. Specify the best and worst case score for each indicator. Option scores outside of these endpoints will be highlighted in red as shown above for Option 2 and subindicator 3A. If your analysis has scores that fall outside the worst-best range, you must adjust the worst-best outcome accordingly. As shown in the example above, sub-indicator 3A best case scale endpoint must be adjusted to 2,200 so that the Option 2 score does not fall outside of the assigned endpoints. If helpful or necessary, make notes on score calculation or document source for each input here. CB&SA Tool Instructions v2.0 21

22 CRC CARE CB&SA Tool 6.2 MCA Weighting Concepts

23 Weighting Indicators Weights represent the relative value of each indicator in a particular decision context In the car purchase example below, cost, which only varies by $100, is probably less important than colour (which varies from two tone to zebra Weight: x% y% stripe) The relative value of one indicator versus another is dependent both on the inherent importance of the indicator, and its variability across the range of options A concept called swing weighting (described in next slide) is used to capture this distinction Colour Cost $17,000 $17,100 CB&SA Tool Instructions v2.0 23

24 Swing Weighting An approach called swing weighting is used to assign weights to indicators. Swing weights refer to the swing in value that occurs as an indicator is varied from one end of its performance scale to the other. For example, if two scales range from 0 to 100, swing weights ask how the swing from 0 to 100 on one preference scale compares to the 0 to 100 swing on another scale. To assign swing weights, consider the best and the worst projected outcomes for each indicator and evaluate an option that includes the worst possible outcome for all indicators. Then ask which indicators you would swing from worst outcome to the best outcome to make the biggest improvement in the desirability of this option, and rank that indicator as number one. This process is repeated to identify the next indicator they would swing from worst to best and rank that indicator as number two, and so on for all indicators. Once all indicators are ranked, assign a weight of 100 to the highest ranking indicator and a relative percentage of this weight to the other indicators (e.g., an indicator whose swing was perceived as half as important of that of the highest ranking indicators would be assigned a weight of 50). CB&SA Tool Instructions v2.0 24

25 CRC CARE CB&SA Tool 6.3 MCA Weighting

26 Weight Print Form First, print a weighting form for each participant providing weights, ensure everyone completes the weighting process completely and accurately as indicated below. Third, Score each of the #1 ranked sub-indicators from on relative importance. The subindicator(s) with a rank of #1 should be assigned a weight of 100. The other sub-indicators should be assigned a weight in descending order by rank, with the weight being proportional to the difference in value. For example, if the swing in value for the sub-indicator with a rank of #2 is close to that of the highest ranked sub-indicator, it might be assigned a weight of 80 or 90. If the swing in value for #2 is viewed to be half that of #1, assign a weight of 50. Second, rank the subindictors with the largest swing in value assigned a rank of 1, the sub-indicator with the second largest swing in value assigned a rank of 2, and so on. Ties are acceptable, i.e., if the swings in value for 2A and 4A are valued equally, they may be assigned the same rank. CB&SA Tool Instructions v2.0 26

27 Weight Input Input the weights provided by each participant into the Wt_input sheet into the yellow cells. Entries should made in the appropriate column for each appropriate participant (as designated by the tool-populated initials). The tool calculates the mean, standard deviation, maximum, and minimum weight assigned for each indicator Note that the weights provided by each participant differ, with 100 indicating the indicator with the largest value swing for that participant, and the remaining weights distributed proportional to 100 as measures of relative value. The tool calculates an initial Consensus weight for each indicator, which is a normalization of the mean results that makes the highest mean equal to 100. In this example, each consensus weight is the mean * 100 / 92. After discussion, the person operating the tool may type over the calculated consensus weights if the group would prefer a consensus that s different than the normalized mean weights. CB&SA Tool Instructions v2.0 27

28 Weight Summary The Wt_summary sheet provides a summary of the weights on a percentage basis. There are no user inputs on this sheet. The tool provides formulas that calculate the percentage weight of each sub-indicator for each participant. The percentages each sum to 100% for each participant. As shown, percentage weights are provided for both indicator categories and sub-indicators. For analyses with many subindicators in a particular category, the category weights provide a representation of the relative value assigned to each category. CB&SA Tool Instructions v2.0 28

29 CRC CARE CB&SA Tool 6.4 MCA Results

30 MCA Results The MCA_Results sheet shows MCA results in total and for each category. The MCA results are the weighted average of normalized scores times weights for all indicators. Note: Scores of 0 result for indicators that have a score equal to the worst possible outcome for that indicator. CB&SA Tool Instructions v2.0 30

31 MCA Chart The MCA_Chart sheet shows a stacked bar graph for the results provided in the MCA_Results sheet. Note that NPV or BCR will appear as an indicator in this graphic only if you have chosen to assign a weight to the CBA results in step 1 of Setup 3. CB&SA Tool Instructions v2.0 31

32 MCA Sensitivity Results The S_Results sheet shows the sensitivity of the MCA scores to changes in weights. The final MCA scores for each participant providing weight are shown, as well as the ranked Options for each participant. Note, in this particular example, the results are relatively insensitive to changes in weights. CB&SA Tool Instructions v2.0 32

33 MCA Sensitivity Chart The S_Chart sheet provides a graphical interpretation of the data from S_Results, showing a scatter plot of the sensitivity of MCA scores to the weights assigned by each individual. CB&SA Tool Instructions v2.0 33

34 CRC CARE CB&SA Tool 7. Summary Results

35 Value Cost Chart The V-C Scatter sheet provides a comparison of the project options with MCA result (value score) on the y-axis and CBA result (NPV) on the x-axis. In this graphic, preferred options are ideally located in the upper right quadrant of the graph indicating relatively high NPV and high value. Note that this sheet appears only when the user chooses not to assign a weight to the CBA results in Step 1 on the Setup 3 sheet. CB&SA Tool Instructions v2.0 35

36 CBA Weight Sensitivity Results The CBA_Wt_Sens sheet shows the total MCA score for each option with varying weights placed on the BCA results in 10% increments. CB&SA Tool Instructions v2.0 36

37 CBA Weight Sensitivity Chart The dropdown menu can be used to select what discount rate is used in the calculation of CBA results. This allows the user to test the sensitivity of their results at high and low discount rates. The CBA_Wt_Chart sheet gives a graphical representation of the MCA-CBA sensitivity test from the CBA_Wt_Sens sheet. CB&SA Tool Instructions v2.0 37

38 Other Sheets Sens_Calcs: shows the formulas used to calculate MCA scores for each weighting sensitivity. The calculations in this sheet are used as inputs to the CBA_results, CBA_charts, S_results and S_Charts sheets. %Sens_Calc: shows the formulas used to calculate sensitivity of CBA and MCA scores to percentage of overall score and feed the CBA_Wt_Sens and CBA_Wt_Chart sheets. CB&SA Tool Instructions v2.0 38

39 CRC CARE CB&SA Tool 8. Troubleshooting

40 Troubleshooting The tool was built on.xls in Office 2013 using Form Control VBA. Though built to function in multiple operating environments, the tool has only been tested extensively in PC Office 2013: if using an older version of Excel or on an Apple Macintosh it s possible that some of the tool setup functions or formulas may not work properly. If you see MCA scores for categories that are negative, most likely you have a score that is outside the scale endpoint ranges. See slide 19 for a discussion how to adjust scale ranges. The tool will accommodate up to 15 options: if you have more than 15 options, use a screening methodology similar to that discussed in Section of the guidelines to narrow the set of options used for the analysis. The tool will accommodate up to 15 monetized benefits, 15 costs, 4categories of indicators and 20 sub indicators in each indicator category. This should be more than sufficient to characterize any remediation problem. If it seems more benefits, costs, or indicators may be relevant, do some further tests to see if some benefits/costs/indicators can be combined for a more generalized representation of effects. CB&SA Tool Instructions v2.0 40