Forests Branch. Final Report. Independent Forest Audit Terms of Reference Review. December 2011

Transcription

1 Forests Branch Final Report Independent Forest Audit Terms of Reference Review Resources and Labour Audit Service Team Ontario Internal Audit Division Ministry of Finance Serving:

2 Table of Contents EXECUTIVE SUMMARY PART A OBSERVATIONS RELATING TO THE ENGAGEMENT OBJECTIVES Approach for the Review of the IFA Terms of Reference Objectives of the Independent Forest Audit Managing the Independent Forest Audit Process Planning the Independent Forest Audit Conducting the Independent Forest Audit Independent Forest Audit Reports Independent Forest Audit Action Plans PART B OTHER OBSERVATIONS Managing the Independent Forest Audit Process Independent Forest Audit Reports Independent Forest Audit Action Plans Other Topics APPENDIX A ACKNOWLEDGEMENTS APPENDIX B BACKGROUND APPENDIX C ENGAGEMENT OBJECTIVES AND SCOPE APPENDIX D APPROACH USED FOR THIS ENGAGEMENT APPENDIX E CONSIDERATIONS FOR THE MINISTRY APPENDIX F IFAPP GUIDING PRINCIPLES APPENDIX G DISTRIBUTION LIST

3 EXECUTIVE SUMMARY Introduction The Crown Forest Sustainability Act, 1994 (the Act or CFSA) requires that the Minister of Natural Resources conduct a review every five years of the forest management unit to ensure the licensee has complied with the terms and conditions of the licence. The Minister s review is supported by an independent forest audit (IFA) of the management unit. Ontario Regulation 160/04 (the Regulation) regulates the IFA. The Regulation requires that at least once every five years the terms of reference that guide the process for the IFAs be reviewed. In 2011 the (MNR or the Ministry) asked the Ontario Internal Audit Division of the Ministry of Finance (Internal Audit) to undertake this review of the IFA terms of reference. This engagement is part of the approved audit plan for the Ministry for the fiscal year The objective of this engagement was to review the Independent Forest Audit terms of reference, as documented in the Independent Forest Audit Process & Protocol, to assess whether it achieves the requirements of Crown Forest Sustainability Act, 1994; the conditions of the Class Environmental Assessment Approval for Forest Management on Crown Lands in Ontario (2003) as amended; and the requirements of the regulated manuals including the Forest Management Planning Manual; as these pertain to the Ontario Regulation 160/04. Conclusion We assessed the IFA terms of reference and conclude that it substantially achieves the requirements of the Regulation under the CFSA. Where appropriate, Internal Audit has made suggestions for the Ministry to consider. Overall Management Comments / Action Plan: MNR acknowledges the report prepared by Ontario Internal Audit Division and would like to thank Internal Audit for their review. The areas that have been identified for our consideration will assist us in our continual evaluation and improvement of the IFA process. Page 1

4 Conformance with Institute of Internal Auditor s Standards This consulting engagement has been conducted in conformance with the Institute of Internal Auditors International Standards for the Professional Practice of Internal Auditing. Original signed by Ray Masse Director Resources and Labour Audit Service Team March 22, 2012 Date Page 2

5 PART A OBSERVATIONS RELATING TO THE ENGAGEMENT OBJECTIVES Approach for the Review of the IFA Terms of Reference Our review of the IFA terms of reference was guided by the requirements of the Regulation and a review of the audit process, which includes managing the audit, planning the audit, conducting the audit, communicating of audit results, and completing action plans and status reports to address the recommendations. The consultative approach used by Internal Audit was as follows: 1. Identified key stakeholders involved with the Independent Forest Audit process in order to understand their roles and responsibilities with regard to the IFA process. 2. Key stakeholders were notified that Internal Audit was conducting the required review of the IFA terms of reference and that there would be opportunities for them to provide Internal Audit information regarding their experience with the IFA process. 3. Reviewed key documentation in order to obtain the necessary background knowledge about the IFA process and to ensure that all legislative requirements for this review were known. 4. Developed a survey organized around the requirements of the Regulation in order to obtain information regarding the IFA process. 5. Conducted onsite meetings with representatives from the Forest Futures Trust Committee, IFA Auditors, Sustainable Forest Licence Holders, and from the Ministry. 6. Facilitated three one-day discussion workshops in order to engage a range of stakeholders in discussions regarding the current IFA process. Details about this consultative approach are contained in Appendix D Approach Used for this Engagement. 1. Objectives of the Independent Forest Audit All effective audits require stated objectives which a professional and independent auditor uses in the conduct of the audit. The terms of reference which guides the IFA process is documented in the Independent Forest Audit Process & Protocol (IFAPP) which was initially developed in The Forests Branch, Policy Division of the Ministry is responsible for the development of the IFAPP which was last revised in February In the case of the IFAs, the Regulation specifically defines the audit objectives for the conduct of the independent forest audits. For example the audit objective derived from the Page 3

6 first requirement of Section 4 of the Regulation is whether the forest management planning activities comply with the Forest Management Planning Manual and with the Act. The IFAPP is currently organized around eight guiding principles (see Appendix F). Most of the principles can be linked directly or indirectly to one of the requirements of the Regulation. However, there are some principles (i.e. the Commitment and the System Support principles) although they are important components of the IFA process, do not relate directly to a requirement. By having the objectives of the IFA audit protocol based on the aforementioned guiding principles, it is not apparent whether the IFA audit program effectively addresses the requirements of the Regulation. Furthermore, an auditor is not required to specifically assess and conclude on each requirement ( objective ) of the Regulation. We ascertained from the onsite meetings, from the Survey, from the workshop discussions, and from the review of the IFAPP that the objectives of the IFA have expanded beyond the core requirements of the Regulation for example, providing an assessment of forest sustainability. Additional objectives may be valid and in response to Ministry priorities regarding the management of Crown management units; however, the additional audit work required seems to be buried amongst the guiding principle objectives. Consideration for the Ministry: C1. The Ministry should consider enhancing the IFA process by aligning the objectives to the requirements of the Regulation. 2. Managing the Independent Forest Audit Process The successful management of an audit process consists of clearly defined roles, responsibilities, and accountabilities for all stakeholders, procurement practices that ensure the audits are carried out by professional and independent auditors, and the management of that process supports the independence of the auditors. A. Roles and Responsibilities A well-managed process requires that the roles, responsibilities, and accountabilities for all the key stakeholders are known, documented and align with the objectives of the process. Without clearly defined and communicated roles, the likelihood increases that there is duplication of effort, that inappropriate decisions are made, and that processes may bottleneck by too many participants. As well, in the case of an audit process, the assigned roles and responsibilities must support the transparency and the independence of that process. Page 4

7 There are distinct phases of the IFA process including: the administration and delivery of the RFP process; the content and delivery of IFA training for the auditors; the review and approval of Audit Plans; the attendance requirements for the Pre-Audit meetings, Site Selection meetings, Field audits, Closing meetings; the factual verification of reports; the acceptance of the final report; the review and approval of Action Plans; and the review and approval of Action Plan Status Reports. The IFAPP outlines roles and responsibilities for the eleven key stakeholders that have been identified. The roles and responsibilities for the identified stakeholders in the IFAPP are organized chronologically and not by the phases in the IFA process nor are they categorized to differentiate between what is required and what is optional. We concluded from the information obtained at the onsite meetings, from the Survey, from the workshop discussions, and from the review of the IFAPP that there may be opportunities to define the involvement of stakeholders at various stages of the audit process. Consideration for the Ministry: C2. The Ministry should consider enhancing the IFA process by further defining the roles and responsibilities for each stakeholder during each of the key phase of the IFA process in the IFAPP. B. Procuring Audit Services Qualification Requirements Through the Request for Proposal (RFP) process, management must ensure that the services of professional, knowledgeable, and independent auditors are procured. The RFP must clearly indicate the qualifications of the audit team members; and the objectives, the scope, the estimated time commitment, deadlines, and information for specific forest management units with respect to the audit engagement. For the procurement of audit services for the IFAs, the Ministry is the administrator for this RFP process. The RFP document is the responsibility of the Ministry who works closely with Ontario Shared Services. The evaluation and award of the RFP is the responsibility of the Forest Futures Trust Committee (FFTC). Once the IFAs have been awarded, the FFTC manages the relationship with the auditors. Page 5

8 In the case of the IFAs, the minimum qualifications and the team composition are stipulated in the Regulation. Specifically the Regulation stipulate the criteria for determining the independence of the auditor, that the IFA will be conducted by a team of a minimum of three independent auditors, and that at least one of the auditors will be a registered professional forester. In addition to the Regulation, the RFP defines the skill sets and experience of the audit team, and that there is a minimum of two registered professional foresters in the audit team. We ascertained from the onsite meetings, from the Survey, and from the workshop discussions that there is a small pool of known auditors that meet the requirements stated in the RFP document. The information that we obtained also indicated that efforts to open up the conduct of the IFAs to a broader base of auditors have occurred but it appears that there is still a perception that the qualifications and requirements have not changed and are still restrictive. One of the comments summarized the information that we had received: There is a small pool of firms conducting IFA s and the pool of auditors is just as small (some people work for multiple firms). I understand that the MNR and the FFT has examined this topic. Efforts to diversify the audit pool should continue to be explored. Consideration for the Ministry: C3. The Ministry should consider enhancing the IFA process by developing strategies to expand the number of auditors that would be qualified to conduct an IFA. C. Procuring Audit Services Information Requirements for Proposals To enable the procurement of audit services that will meet the objectives and purpose of that engagement, an effective RFP document should outline the objectives, the scope, the estimated time commitment, and deadlines for the audit engagement. With the IFA process, in addition to the objectives, scope, time commitments, and deadlines, the prospective auditors need information for the specific forest management units being audited. The RFP document includes a spreadsheet prepared by the Ministry that provides information regarding specific challenges associated with each forest management unit. The auditors submit their bid for an IFA based on the specific challenges identified in the spreadsheet for that forest management unit and the general timelines outlined in the IFAPP (Section 3.0). However, we ascertained from the onsite meetings, from the Survey, and from the workshop discussions that the model timelines in the IFAPP are estimates and Page 6

9 as such there is uncertainty around the amount of staff time involved for the completion of an IFA. Consideration for the Ministry: C4. The Ministry should consider enhancing the IFA process by providing additional information and time commitments for the purposes of the RFP thereby enhancing the procurement of audit services. 3. Planning the Independent Forest Audit The effective planning of an audit includes an audit plan developed by the auditors based on the objectives and requirements of the audit; the communication of that audit plan; and the flexibility to amend the audit plan as may be required. The auditors are accountable for developing an audit plan that meets the objectives of the audit. Good audit plans are detailed and realistic with timelines to be followed in order to accomplish the stated audit objectives for that audit. The auditor will consult with the auditee and/or the procurer of the audit services in developing the audit plan; however, the audit plan cannot be influenced or approved by the auditee if the auditor is to maintain his independence in the audit process. In the IFA process, once the auditors have been awarded the IFA an audit plan is prepared based on the proposed Approach and the Project Plan & Schedule documents that were submitted in response to the RFP process, and on the auditors preliminary information gathering and analysis. The IFAPP dictates the eleven mandatory components for the audit plan prepared by the auditors. The audit plan for the IFA is submitted to the auditees, the LCC, and the FFTC prior to the Pre-Audit Meeting where the audit plan is presented, discussed, and finalized. Although there is consultation with the stakeholders, these stakeholders do not approve the audit plan the audit plan remains the responsibility of the auditors. The decisions with respect to sample selection for audit purposes remain those of the auditors regardless of the consultation with the auditee. In the IFA process, the site selection by the auditors is completed once the audit plan has been finalized and as such it is not documented in the audit plan. Once the site selection(s) are made, the auditee is required to prepare information packages (i.e. Field Binders) for the auditors. The IFAPP provides a list of items for the auditors to consider regarding sample design as well as the requirement that sampling must be planned, designed and reported from the Page 7

10 perspective that samples of each type of operation (e.g., harvesting, renewal, road construction, etc.) will need to be done at a minimum of 10% of the level of operations conducted during the audit term. We concluded from the information obtained at the onsite meetings, from the Surveys, and from the workshop discussions that, with respect to sampling, there was concern that the minimum, of 10% had become a standard not a minimum; and that intensities and methodologies used by the auditors for sampling design, as well as the sites selected for the audit work, were not necessarily representative of the forest management activities. Although analysis of the sampling percentages for the IFAs conducted in 2009 and in 2010 seem to indicate that the minimum of 10% has not become a standard, the comments received with respect sampling from the onsite meetings, from the Surveys, and from the workshop discussion indicate otherwise: Need to look at putting in a mechanism that forces the audit team to properly assess field sites. Although representative samples are identified an auditor essentially gets credit for sampling these areas by setting foot into them or flying over them. Feeling amongst many people in industry and government is that not enough time is being spent in the field. Often brief snapshots of sites may not be indicative of overall performance. When forest management units have been amalgamated and are audited; the minimum sample size of 10% does not always reflect the larger forest unit. I feel that more time should be spent in the time actually looking at work conducted in the field. Our forest is a large unit (almost 2 million ha) and the field portions seem rushed. The framework for the IFA process includes the Pre-Audit Meeting where concerns regarding the sampling design (i.e. intensities and methodologies) can be brought to the attention of the auditors and/or the FFTC and discussed. It is at this stage that concerns should be raised rather than at a later stage where an issue resolution process would be employed. The IFAPP makes the suggestion that the auditors should hold a site-selection meeting and there are some auditors that include this type of meeting as part of the IFA. We ascertained from the onsite meetings that when this type of site-selection meeting was held it received positive feedback from the participants: There is adequate consultation with the Auditor prior to their sample selection there is an opportunity to voice any concerns. The Auditors do provide the sites that they will be sampling and there is an opportunity to provide input. Page 8

11 Consideration for the Ministry: C5. The Ministry should consider enhancing the IFA process by ensuring that all auditees are aware of the site selection process. 4. Conducting the Independent Forest Audit The effective conduct of an audit has clear and measurable objectives; an audit plan to meet those objectives; unbiased evidence gathering and analysis; a timeframe with sufficient data and/or results that allows the auditors to make assessments and conclusions; good communication; and the timely completion of the audit work and of reporting. The timing of the audit needs to be when activities that are being measured and assessed are completed. Forest Management Plans (FMP) provide the authority to carry out forest management activities including road access, timber harvest, and forest renewal, tending and protection treatments. The Forest Management Planning Manual and the legislation guide and prescribe the development of these 10-year FMPs. The processes to develop, review, submit, and approve a 10-year FMP take approximately three years. We concluded from the information obtained at the onsite meetings, from the Surveys, and from the workshop discussions that the forest management planning process is a wellestablished process; that forest management plans within the same planning cycle and region are developed in the same manner; that within the forest management planning process there are internal reviews and check points; and that there are several layers of review and approval. The first four requirements in Section 4 of the Regulation are audit objectives with respect to the FMP. In simple terms these four audit objectives equate to answering the following questions: i. Did you follow the FMPM and the Act when developing the plan (i.e. did you follow the process you were supposed to follow); ii. Do the activities in the plan comply with the Act, the plan and guidelines (i.e. are the planned activities in accordance with the Act, etc.); iii. Did the activities effectively meet the objectives of the plan (i.e. did you get the results that you planned); and iv. How did the activities compare to what was planned (i.e. did you do what you were supposed to do). Page 9

12 The third objective cited above is the requirement of the Regulation to assess the effectiveness of forest management activities in meeting the forest management objectives set out in the forest management plan, as measured in relation to the criteria established for the audit. This audit objective established by the Regulation assumes that the management objectives set out in the FMP are clear, measurable, and have defined timelines in other words, it assumes that these FMP management objectives are auditable. We ascertained from the onsite meetings, from the Surveys, and from the workshop discussions that the management objectives set out in the FMP are not always specific, measurable and have defined timelines therefore are not always auditable. Establishing auditable FMP management objectives is not necessarily a requirement in the development of the FMP. One such example of the difficulties with the auditing of the management objectives of the FMP is confirmed in the following comment: There can be some difficulty assessing if the Plan objectives have been met as they are often worded in a way that makes it difficult as to know what the plan actually is. One of the most important objectives is renewal and I have seen objectives that state to effectively regenerate areas to reach FTG in a manner that is consistent with the regeneration standards in the SGRs with the indicator being the percent of harvest area assessed as free growing and this will be reported through the year 7 and 10 annual reports. Based on this, how can the IFA determine that activities are meeting this objective? Consideration for the Ministry: C6. The Ministry should consider enhancing the IFA process by ensuring that individuals involved with the forest management planning process are aware of the IFA implications; how the forest management objectives are stated in the Forest Management Plan. 5. Independent Forest Audit Reports Audit reports communicate the purpose, scope, findings and recommendations of an audit. The process for the distribution, review and approval of the various types (i.e. phases) of the audit report must support the purpose objective for that type of report. The format for an audit reports is not only dependent on the type of audit and its objectives but must also consider its audience. Once the audit work has been completed, the communication of the findings, the recommendations, and conclusions are accomplished by the reporting phase of the audit process. At each step of the reporting process each type of report is given a purpose and is Page 10

13 distributed only to those stakeholders who have accountabilities regarding the purpose for that type of report. Typically the reporting process begins with an initial report, such as a draft report, that has the underlying purpose to verify factual content; and its audience is the auditee. A good and efficient factual clearance process involves only the stakeholders, in this case the auditees, who are knowledgeable and have the information to verify the factual content of the draft report. Once factual clearance has been completed either another draft report or a final report is issued. In the case where subsequent draft report(s) are issued, those reports should only be issued to the stakeholders that can expedite the process to the next phase. Usually the purpose of the final report is to communicate information regarding the audit including the objectives, scope, findings and recommendations to shareholders, regulatory agencies, or the general public depending on the type and objectives of the audit. In the case of the IFA process, the IFAPP outlines the requirement of three formal reports Draft Report, Final Draft Report, and Final Report that are issued to various stakeholders. The current timelines for the issuance of the IFA Reports, based on the model schedule in the IFAPP are: the Draft Report issued within 60 days of the start of the audit; the Final Draft Report is issued within the next 30 days; and the Final Report is issued within the following 30 days. In other words, the timeline for the model schedule is 120 days from the start of the field audit to the issuance of the Final Report. In the IFA process the purpose of the Draft Report is factual clearance thereby assisting the auditors with clarity and accuracy of the report; however in accordance with the IFAPP this report is not only submitted to the SFL holder, the Regional Operations Division of the Ministry, and the Forests Branch of the Ministry typically the auditees; but also to the FFTC personnel. After this report is submitted the auditors have a Draft Report Review Meeting where attendance is required by a number of individuals representing the SFL holder, the District Office of the Ministry, the Regional Operations Division of the Ministry, and the Forests Branch of the Ministry. As well there is an option for the FFTC to not only observe but to participate. Once the Draft Report Meeting has taken place, the auditees then have one week to provide any written comments to the auditors; and those comments are distributed to auditees as well as the FFTC representatives. One of the better practices for any process, including that of vetting factual content of a report, is to only involve those who can advance the process. Page 11

14 In the IFA process, the next report that is issued by the auditors is the Draft Final Report. The primary audience for this report is the FFTC and the purpose of their review is with respect to the auditors contractual obligations. An information copy of the Draft Final Report is provided to the SFL holders, and the Regional Operations Division and Forests Branch representatives of the Ministry for their review of the new content. The final report in the IFA process is the Final Report which the auditors issue to the following stakeholders SFL holder; MNR Regional Director and the representative for the Regional Operations Division; MNR District Manager and the representative for the District; MNR Director, Forests Branch and the representative for the Forests Branch; LCC; and FFTC. Consideration for the Ministry: No items for consideration. 6. Independent Forest Audit Action Plans One of the fundamentals of an effective audit is the inclusion of a process that enables the assurance that the recommendations made by the auditor are acted upon by the auditee. The elements of such a process include the documentation of the specific activities that are to be implemented to address the recommendation, who has ownership or accountability for addressing the recommendation, the timeline required for the activities to be implemented, how the implementation is to be monitored and/or reported, and the review of the actual activities that were implemented to ensure that these address the original recommendations. The IFA process includes the development of Action Plans by the auditee which helps ensure that the recommendations from the Final Report are addressed within a committed time frame. The IFAPP requires that an Action Plan be developed for each recommendation, and that it is prepared and submitted by the District Manager within two months of receipt of the Final Report. The approved Action Plan is then posted on the Ministry s external website once the Final Report has been tabled in the legislature. As a result there is usually several months between finalizing the Action Plan and posting it on the Ministry s website. The Action Plan that is developed to address the recommendations in the IFA Report has all the elements of a good process and includes the following components as directed by the IFAPP: Page 12

15 the action required to clearly address the recommendation the organization and position title responsible for completing the action the deadline date(s) for the action to be completed a method for tracking the progress of the action This process is further enhanced by requiring two separate Action Plans, based on accountability, for the recommendations made in the Final Report a Management Unit Action Plan for findings directed at the SFL holder and/or MNR District, and, if required, a Provincial Action Plan for finding directed at Corporate MNR. The final element of the process is to ensure that the recommendations from the previous audits have been addressed by having an independent review of the actions carried out by the auditee(s). For the IFA process this component is directed by the Regulation requirement that the auditor shall assess the effectiveness of any action plans implemented to remedy shortcomings revealed by a previous audit. This review is conducted five years (during the next scheduled IFA) after the original audit. In the interim the auditee is required to complete an Action Plan Status Report detailing the progress of the implementation of the Action Plan within two years of the approval of the Action Plan. This Action Plan Status Report is prepared by the MNR District Manager and submitted for review and approval by the MNR Regional Director. The comments received at the onsite meetings and at the workshop discussions indicated that although there is a requirement for the auditors to review the Action Plans from the previous IFA, the review is conducted at a high level and characteristically involves the review of the Action Plan Status Reports. Consideration for the Ministry: No items for consideration. Page 13

16 PART B OTHER OBSERVATIONS Through our review of the IFA terms of reference and our communication with stakeholders, observations which do not directly relate to the objective of this engagement were made and are detailed in this Part B of this report. 7. Managing the Independent Forest Audit Process A. Issue Resolution Process A formal issue resolution process outlines the process to forward concerns, the format to use to forward the concerns, the parties involved with reviewing the concerns, the timeline for a response to the concerns, whether there is an opportunity for appeal, and who is responsible for managing the process. A strong issue resolution process must identify the scope of issues that can be brought forward in order that the independence of the auditors and their ability to state their professional opinion is not compromised. In the case of the IFA, there is an informal process whereby if the auditee or other stakeholders have any concerns with the independent forest audit, then they have the option to forward their concerns to the FFTC who maintains an open-door policy. As well, the FFTC reviews the Final Draft Report to ensure that the IFA auditor has met the contractual obligations of the service agreement. We ascertained from the onsite meetings, from the Survey, and from the workshop discussions that there is no formal issue resolution process and that the auditees have expressed concerns regarding site selection, sampling and occasionally concerns about audit findings and recommendations. There are opportunities within the IFA process during the Pre-Audit meeting, the field audit, the Closing meeting, and during the review of the Draft and Final Draft Reports where concerns can be brought forward and addressed. Consideration for the Ministry: C7. The Ministry should consider enhancing the IFA process by formalizing the current issue resolution process while maintaining the independence and the objectivity of the IFA auditors. B. Feedback on the Independent Forest Audits The management of the procurement process should include a mechanism to monitor the delivery of the services which were contracted. Tools are required to monitor both the tangible deliverables (i.e. audit plans and reports) and the non-tangible components (i.e. Page 14

17 performance). The component of monitoring performance typically includes formal feedback on the program delivery. In the case of the IFA, the FFTC, who is responsible for evaluating RFP submissions and monitoring the quality of that service regarding program delivery and contractual obligations, has an open-door policy where auditees are free to provide their feedback regarding the independent forest audits. In addition members of the FFTC also attend field audits where feedback is obtained though conversations with some or all of the auditees present. We concluded from the information obtained at the onsite meetings, from the Surveys, and from the workshop discussions that there is no formal feedback mechanism and that not all the feedback from the auditees is being communicated to the FFTC. The Ministry used Client Surveys for the IFAs conducted in 2009 and in 2010 to obtain feedback from the auditees and recognize that this is a step towards obtaining regular feedback from all stakeholders. Consideration for the Ministry: C8. The Ministry should consider enhancing the IFA process by incorporating a formal periodic feedback process. 8. Independent Forest Audit Reports A. Report Process Draft Report Comments As previously mentioned, in the IFA process the purpose of the Draft Report is factual clearance thereby assisting the auditors with clarity and accuracy of the report. After this report is submitted the auditors have a Draft Report Review Meeting where attendance is required by representatives of the various auditees. The IFAPP states that the auditees should endeavour to provide written comments on the Draft Report to the auditors prior to the Draft Report Review meeting. Once the Draft Report Meeting has taken place, the auditees then have one week to provide any written comments to the auditors; and those comments will be distributed to all parties who attended the Draft Report Review Meeting as well as the FFTC representatives. One of the better practices for any process, including that of vetting factual content of a report, is to only involve those who can advance the process. Page 15

18 We ascertained from the onsite meetings, from the Surveys, from the workshop discussions, and the review of the IFAPP that the reporting process does not make it mandatory for the auditee to provide written comments on the Draft Report prior to the Draft Report Meeting and as such not all the Draft Report Meetings are as effective as they could be because the auditors may not be aware of concerns with the report prior to the meeting. As well there are participants at the meeting who are not directly responsible for factual clearance. Their presence may contribute to any inefficiencies of the meeting. Considerations for the Ministry: C9. The Ministry should consider enhancing the IFA process by requiring that Draft Report comments are provided in writing to the auditors prior to the scheduling of the Draft Review Meeting thereby contributing to the usefulness of the meeting. C10. The Ministry should consider enhancing the IFA process by ensuring that only participants at the Draft Report Meeting are responsible for report factual clearance thereby contributing to the efficiency of the meeting. B. Report Process Factual Clearance Although the reporting process requires that audit findings and recommendations be communicated to the auditee at the Closing Meeting which is required to take place on the last day of the field visit, this communication is verbal and is not necessarily detailed. Typically the detailed written findings and recommendations are not available to the auditee until the Draft Report is issued. The format of the Draft Report is the same as the Final Report and as such includes all the sections of the Final Report as well as the Appendices. The principal purpose of the Draft Report is factual clearance and the vetting is primarily with respect to the findings and recommendations because that is the area of interest for the auditee, and that is the area where there is the greatest risk that information may have been misinterpreted or not provided. With this in mind another format may be considered to communicate the information regarding findings and recommendations. One format which was discussed at one of the discussion workshops is utilizing the current Appendix 1 Recommendations of the IFA Report, instead of the Draft Report, as the document to communicate the findings and recommendations to the auditee for the purpose of factual clearance. The use of Appendix 1 Recommendations for the purpose of factual clearance would reduce the time required by the auditors to prepare a document for the purposes of factual clearance when compared to preparing a complete report, it would help ensure that Page 16

19 Appendix 1 Recommendations contains all the information regarding recommendations that is used by the auditee in the development of Action Plans, and it would enable the earlier communication of recommendations to the auditee. The following comments highlight the need for an improved process which keeps the auditee informed of any changes to the recommendations: In the IFAPP there should be a process to highlight changes made to draft audit reports... Need to formalize this process so all are in the loop when a decision has been made to remove an initial recommendation. Between the Draft Final and the Final Report recommendations significant changes were made to the recommendations with no further evidence gathered... It is evident that the auditors were influenced by outside opinion rather than evidence. If the Draft Report is replaced with the issuance of the Appendix 1 Recommendations document, the meeting date to discuss factual content and the recommendations should be earlier than the current 60 days from the beginning of the audit; and it still should be mandatory that auditees provide written comments to the auditors prior to the meeting. Consideration for the Ministry: C11. The Ministry should consider enhancing the IFA process by utilizing a document that serves the sole purpose of communicating the findings and recommendations for factual clearance to the auditees. C. Report Format Streamlining The management of the reporting phase of the audit process is aided by the use of mandatory content requirements and/or audit report templates. The use of standard audit report formats serves several purposes addressing the objectives of the audit, ensuring information and format consistency, and meeting expectations of the audience for the report. The typical format for a final audit report includes an executive summary, a statement of the objective(s), detailed findings and recommendations, and conclusion(s) regarding the stated objective(s). In the case of the IFA process, the IFAPP requires that the Final Report format includes the four sections and the seven appendices. There are three areas of the Final Report where the audit findings and recommendations are included: the Table of Recommendations and Best Practices; the Audit Findings; and Appendix 1 Recommendations. We ascertained from the onsite meetings, from the Page 17

20 Surveys, and from the workshop discussions that there may be opportunities to remove duplication within the report. Consideration for the Ministry: C12. The Ministry should consider enhancing the IFA process by either eliminating or combining areas of duplication within the Final Report with the goal to streamline the report. D. Report Format Balanced Reports The Final Report includes the reporting of the best practices in place in order to tell the whole story ; in other words have a balance report that recognizes best practices in addition to making recommendations for improvement. The IFAPP defines Best Practices as practices so identified should be exceptional, not situations in which the forest manager is simply meeting a good forest management standard. Highly effective novel approaches to various aspects of forest management may represent best practices. Similarly, applications of established management approaches which achieve remarkable success may represent best practices. We concluded from the information obtained at the onsite meetings, from the Surveys, from the workshop discussions, and from the review of Final Reports that the reports are not always well balanced with respect to any positive forest management activities, including best practices that have occurred during the IFA audit period. Balanced reports, that acknowledge what is going right with the management of the forest management unit as well as having recommendations where improvements are required, not only provide information regarding better practices and solutions to other auditees but also communicate to the Ontario public a more representative depiction of that management unit. Consideration for the Ministry: C13. The Ministry should consider enhancing the IFA process by providing additional reporting guidelines thereby producing more balanced Final Reports. 9. Independent Forest Audit Action Plans The IFAPP outlines the due date, review process, and submission and approval for the Action Plans. The IFAPP acknowledges the enhanced IFA process which currently has separate Action Plans developed for the management unit and provincial recommendations. However, the IFAPP has not differentiated the review, submission and approval process for a Page 18

21 Management Unit Action Plan from that of a Provincial Action Plan. Consequently the review, submission and approval process for the Management Unit Action Plan requires the same level of scrutiny as that for a Provincial Action Plan which is more rigorous than the approval required for the management unit s Forest Management Plans. Consideration for the Ministry: C14. The Ministry should consider enhancing the IFA process by revising the review and approval requirements for the Provincial Action Plans and the Management Unit Action Plans. 10. Other Topics A. Forest Sustainability Whether a forest management unit is managed by the Crown or by an SFL holder, the unit must be managed in accordance with standards and requirements established in the Act. One of the underlying requirements is that the forest management unit is managed in a sustainable manner. With respect to sustainability, the Act states that for the purpose of this Act and the regulations, the sustainability of a Crown forest shall be determined in accordance with the Forest Management Planning Manual. The 2004 and 2009 Forest Management Planning Manual cites that In accordance with the Crown Forest Sustainability Act, the sustainability of the forest will be determined in accordance with the approach described in the Forest Management Planning Manual. The approach requires the identification of measurable indicators in order to assess the effectiveness of activities in achieving management objectives and to assess the sustainability of the forest for the management unit. For the forest management plan, the determination of sustainability will be a conclusion that the forest management plan provides for the long-term Crown Forest health on the management unit, and has regard for plant life, animal life, water, soil, air, and social and economic values, including recreational values and heritage values. There are numerous definitions and principles regarding sustainability and consequently there are numerous approaches to evaluate sustainability. In Ontario for Crown forests, activities supporting sustainability are established and documented in the approved Forest Management Plan (FMP) for that forest management unit. The Ministry has an extensive open and consultative forest management planning process lead by a Registered Professional Forester with the assistance of a multi-disciplinary planning team and a local citizens committee, as well as with input from Aboriginal Page 19

22 communities, stakeholders and interested members of the public. Each forest management unit is required to have a FMP which is developed in accordance with the Act and the Forest Management Planning Manual. The FMP sets out long-term management direction that includes objectives and strategies for the sustainable management of that forest management unit. Furthermore, the Act states that The Minister shall not approve a forest management plan unless the Minister is satisfied that the plan provides for the sustainability of the Crown forest, having regard to the plant life, animal life, water, soil, air and social and economic values, including recreational values and heritage values, of the Crown forest. The question with respect to the IFAs is how the requirements of the Regulation dictate that the sustainability of the forest management unit is to be assessed. The answer is in the following requirements of the Regulation: a) assess to what extent forest management planning activities comply with the Forest Management Planning Manual and the Act; b) assess to what extent forest management activities comply with the Act and with the forest management plans, the manuals approved under the Act and the applicable guides; c) assess the effectiveness of forest management activities in meeting the forest management objectives set out in the forest management plan, as measured in relation to the criteria established for the audit; and d) compare the forest management activities carried out with those that were planned. The aforementioned requirements acknowledge that sustainability is supported by planning activities as defined by the Forest Management Planning Manual and that the FMP establishes the activities to support sustainability for a particular forest management unit. In other words, by virtue that the IFA auditor reviews and makes assessments on each of the requirements of the Regulation, sustainability is being reviewed. What is not stipulated as a requirement of the Regulation is that the IFA auditors need to make a separate conclusion relating to the sustainability. We ascertained from the onsite meetings, from the Survey, from the workshop discussions, and the review of the IFAPP that there are some concerns with the approach used by some of the IFA auditors to provide a conclusion on forest sustainability. Comments received encapsulated the concerns that we heard: The assessment of Forest Sustainability must be consistent with the Act definition of Forest Sustainability and the measures in place to determine it. In some cases the auditors are interpreting forest sustainability as they please. The assessment of these criteria needs to be strictly compared to the definition in the Act. Page 20

23 The definition of forest sustainability, criteria to assess it and the time frame associated with the sustainability conclusion need to be clarified. Forest Sustainability is not directly identified in the Regulation itself so further clarity is necessary to assist auditors and to define the audit scope and assessment requirements. Any evaluation must be factually / evidence based and opinions must be fully supported by qualified individuals. The public needs an unbiased professional opinion on whether a plan was developed according to the rules and if the approved plan is being implemented appropriately. A separate forum / avenue needs to be provided for auditors who wish to challenge question the rules themselves (FMPM / CFSA / Guides) and these concerns should not be identified in the audit of individual forests as they are beyond the scope of local forest managers to address. What is sustainability? It is easy to prove sustainability when each new plan can reduce habitat by dropping to some percentage (BNV) below the benchmark, and wood supply can be elevated in the short term and then allow a drop of 40% in the relatively near future and show no recovery, and the final test of sustainability is a feasible solution. When auditors (and others) question this as sustainable, they are going up against the fact that the Ministry approved the plan as sustainable. Are the auditors wrong or does the FMPM requiring a rewrite to create a more defined playing field. Regardless of what we personally believe, there are formal requirements that have to be followed when practicing in Ontario, and would be expected to be audited against such requirements. In practice, the Forest Management Plan was prepared in accordance with the requirements of the day, while the Audit Team (or specific members of the Audit Team) took the liberty to audit to a standard that was predetermined by their own personal opinions. The Independent Forest Audit Process and Protocol (IFAPP) should be based on the premise that the forest management plan has undergone an extensive review process, resulting in a sustainable forest management plan. The auditor should not be re-examining the sustainability conclusion reached in the management plan. Emphasis should be placed on assessing field results and the implementation of the management plan (i.e. is the plan being followed and are plan objectives being satisfied?). As we continued to express our frustration about preparing a Forest Management Plan under the requirements of the CFSA and the FMPM direction, then being audited to another standard through the subjectiveness of the supposedly objective IFA process... Page 21