Social Development Partnerships Program

Transcription

1 Internal Audit Services Audit of the Social Development Partnerships Program SDC-A E

2

3 Audit of the Social Development Partnerships Program Final Report Project No. 6563/02 Audit and Evaluation Directorate Policy and Strategic Direction Social Development Canada Project Team: Director General: Audit Director: Team Leaders: Audit Team: APPROVED: HRDC J.K. Martin G. Ross H. Forget / S. Lynch A. Batungwanayo A. Farley SDC J. Blain B. Sliter H. Forget / S. Lynch DIRECTOR: G. Ross July 26, 2005 Name Date DIRECTOR GENERAL: J.K. Martin July 26, 2005 Name Date September 2005 SDC-A E (également disponible en français)

4 Paper ISBN: X Cat. No.: SD31-5/2005E PDF ISBN: Cat. No.: SD31-5/2005E-PDF HTML ISBN: Cat. No.: SD31-5/2005E-HTML

5 Table of Contents 1.0 Executive Summary... i 2.0 Introduction Audit Findings Program Mandate/Strategy Accountability for Results Supporting Program Capacity Program Monitoring Conclusion Management Response Update, as at September, Management Response Overview Program Mandate/Strategy Audit Findings Management Response Accountability for Results Audit Findings Management Response Supporting Program Capacity Audit Findings Management Response Program Monitoring Audit Findings Management Response Conclusion Appendix A... A-1 Appendix B... B-1 Appendix C... C-1 Appendix D... D-1 Appendix E... E-1

6

7 1.0 Executive Summary The audit of the Social Development Partnerships Programs (SDPP) and the Child Care Vision (CCV) was undertaken as part of the annual audit plan of Internal Audit and Risk Management Services (IARMS) of the former HRDC department. Based on our preliminary survey of the program, it was learned that CCV s terms and conditions had expired and were subsumed under the SDPP on a transitional basis, pending program renewal (April 1, 2003). New agreements initiated in fiscal year and beyond were signed under the SDPP terms and conditions leaving very few active multi-year agreements under the previous CCV program authorities. Hence a decision was taken to audit the activities of both program areas as a single audit to be reported in as planned for SDPP. The projects selected for file review have start dates between September 1, 2001 and August 31, 2002 inclusive, all falling within the old terms and conditions, that is, the terms and conditions ending March 31, Except where noted, for the remainder of this report, the term SDPP refers to both programs, under the old terms and conditions. The field work for this audit was undertaken during the period May to September, This period, and the period immediately following the audit was characterized by significant change and planning activities in light of the new terms and conditions. Wherever applicable, Internal Audit and Evaluation has linked the results of this audit to key changes in program management. Specific references are made in Criteria 1.1, 1.2, 1.4, 2.1, 2.2, 2.5 and 4.1. The program focuses on vulnerable populations with activities focusing on: fostering collaboration, partnership, alliances and networks; generation, dissemination, and application of knowledge; and, strengthening the capacity of organizations of the social non-profit sector. As per the revised Terms and Conditions, SDPP s long term objectives are to: Increase the effectiveness of the non-profit sector in meeting the social development needs and aspirations of persons with disabilities, children and their families and other vulnerable or excluded populations; and Improve the quality and responsiveness of governments social policies and programs. The Program s more immediate objectives, against which the program will be evaluated, are to: Promote the generation, dissemination and application of knowledge on emerging social concerns, innovative solutions, best practices and social and economic outcomes as they relate to: persons with disabilities; children and their families; and, other vulnerable or excluded populations; Foster collaboration, partnerships, alliances, and networks to advance shared social goals and priorities; and Strengthen the capacity of organizations in the social non-profit sector with respect to governance, policy and program development, community outreach, organizational administration and management. i

8 The program is administered within the Social Development Sectors Branch, formerly the Income Security Programs (ISP) Branch, which is referenced throughout the report. The SDPP terms and conditions are also used by Strategic Policy Branch (SPB) via a Memorandum of Understanding. As part of the audit, we assessed the progress achieved in responding to the findings contained in the October 2000 Office of the Auditor General s report. Our assessment found that most of the OAG findings have been addressed by the program. The following issues, however, continue to be addressed: Contribution agreements frequently lacked key information; Deficiencies in handling payments; Inadequate project monitoring; Grant proposals not fully assessed in all cases; and Weak rationales for recommending the approval of grant projects. See Appendix E for details of our assessment. The objectives of the audit were to provide assurance that: SDPP is appropriately managed; and Risks are identified and appropriately addressed. The audit covered aspects of the management control framework with a focus on: accountability for results; quality of monitoring and file administration; and compliance with terms and conditions. Audit objectives, criteria, scope and methodology are included in Appendix A. As SDPP is a national, centrally-delivered program, the audit was conducted entirely at NHQ from May to September, The management framework was assessed for the program by the following means: Control self-assessment of the program by program management and selected staff; Interviews with program staff and management; Examination of relevant documentation including file reviews; and Analysis of information. The scope of the audit assessed the management framework and operational processes within SDPP and is based upon the 14 audit criteria as listed in Appendix A. This internal audit was conducted in accordance with both the Treasury Board Policy on Internal Audit and the Institute of Internal Auditors Standards for the Professional Practice of Internal Auditing. Based on the evidence examined in support of the audit criteria, we conclude that the Social Development Partnerships Program is being adequately managed and risks are being identified and addressed. More work is needed, however, to improve certain aspects of internal control as well as project and program monitoring. ii

9 The following recommendations for improvements are made. 1. SDPP management should require that all grant agreements be formalized, stipulating expectations and requirements of all parties involved, and that all grant projects be subject to a risk assessment; 2. Social Development Canada should develop an expanded grant section in the Grants and Contributions Operation Guide 1 ; 3. Social Development Canada should update the Grants and Contributions Operations Guide to clarify the application of the percentage overhead rates; 4. SDPP management should review and improve the claims verification practices at the payment stage of the project life cycle; 5. In order to maximize the program delivery skills of the program officers, SDPP management should consider alternatives to the current financial monitoring routines, bringing in a more formal approach to assessing recipient s financial and accounting status. This may be achieved by the utilization of a dedicated programwide financial officer to assist and accompany the program officers on the larger, more complicated assessments, on an as-required basis, as well as an adjustment to the risk assessments and monitoring plans, as noted in criterion 2.4. This would dovetail with specialization initiatives being considered by HRDC at the time of the audit; 6. SDPP management should improve activity and financial monitoring by ensuring that: Actual monitoring activities are conducted in accordance with the developed monitoring plan; Verification of claimed amounts to original documentation are done with more rigour, including verification of overhead rates, verification of per-diem salary rates to actual costs; A more predominant role is given to program activity monitoring; Costs claimed are linked to project milestones achieved. Refer to Appendix B, Management Action Plans for the identified actions and dates for completion. 1 This is a common guide shared by HRSDC; any changes require bilateral co-operation. iii

10 iv

11 2.0 Introduction The audit of the Social Development Partnership Program (SDPP) and the Child Care Vision (CCV) was undertaken as part of the annual audit plan of Internal Audit and Risk Management Services (IARMS) of the former HRDC. The CCV audit was scheduled for Based on our preliminary survey of the program, it was learned that its terms and conditions had expired and were subsumed under the SDPP on a transitional basis, pending program renewal. New agreements initiated in fiscal year and beyond were signed under the SDPP terms and conditions leaving very few active multi-year agreements under the previous CCV program authorities. Hence a decision was taken to audit the activities of both program areas as a single audit to be reported in as planned for SDPP. Except where noted, for the remainder of this report, the term SDPP refers to both programs. The SDPP is the result of the merging of two programs: the National Welfare Grants (NWG) and the Disabled Persons Participation Program (DPPP).The NWG began in 1962 to provide grants and contributions for social welfare research and development projects and human resource development in the social welfare field. The DPPP was established in 1985 with a mandate to facilitate the full participation of people with disabilities in the economic and social life of Canada. The program funded projects undertaken by non-profit disability organizations that were innovative, time-limited, results-oriented, and focussed on changing policies and practices that hinder the participation of disabled persons in community life. Reorganization of the federal government in 1993 resulted in both the NWG and DPPP coming under the mandate of HRDC. In 1998, SDPP was implemented as the new funding regime. It modeled an approach of partnerships between the government and the voluntary sector based on on-going consultative mechanisms to establish priorities, maintain accountability and respond to issues. The Child Care Vision program, established in 1995 was a successor to the Child Care Initiative Fund (CCIF). The primary objective of the CCV at this time was to support research and development projects that study the adequacy, outcomes and costeffectiveness of current best child care practices and service delivery models. A Treasury Board submission renewing the SDPP terms and condition incorporating CCV program activities was given approval January 30, 2003 to come into effect April 1, It makes the provision for agreements signed under the previously approved terms and conditions for SDPP and CCV to continue to operate under the previously existing authorities until completion. The renewal of the program s terms and conditions provided a vehicle for addressing the shortcomings identified in the October 2000 Report of the Office of the Auditor General. As such, clear reasoning behind funding choices, concise eligibility criteria, well defined program objectives and formal marketing of the program were key priorities for the renewal of the program. 1

12 The renewed program retains its focus on vulnerable populations, as well as on the key elements of the former programs, namely: activities focused on fostering collaboration, partnership, alliances and networks; generation, dissemination, and application of knowledge; and strengthening the capacity of organizations of the social non-profit sector. As per the revised terms and conditions, SDPP s long term objectives are to: Increase the effectiveness of the non-profit sector in meeting the social development needs and aspirations of persons with disabilities, children and their families and other vulnerable or excluded populations; and Improve the quality and responsiveness of governments social policies and programs. The Program s more immediate objectives, against which the program will be evaluated, are to: Promote the generation, dissemination and application of knowledge on: emerging social concerns, innovative solutions, best practices; and, social and economic outcomes as they relate to persons with disabilities, children and their families and other vulnerable or excluded populations; Foster collaboration, partnerships, alliances and networks to advance shared social goals and priorities; and Strengthen the capacity of organizations in the social non-profit sector with respect to governance, policy and program development, community outreach, organizational administration and management. SDPP is administered within the Income Security Programs (ISP) Branch through the Office for Disability Issues (ODI) and the Social Development Directorate (SDD) (now named the Community Development and Partnerships Directorate). The SDPP Terms and Conditions are also used by Strategic Policy Branch (SPB) via a Memorandum of Understanding to support SDPP s social development objectives and priorities and inform policy. It outlines roles and responsibilities to ensure clear accountabilities in delivery and management of the research and projects managed by SPB. Overall program accountability for SDPP, however, lies with the Assistant Deputy Minister of ISP. Of the $30.4 million program funds budgeted for , $12.5 million was allocated to ODI, $15.7 million to SDD, $0.6 million to SP and $1.6 million to Applied Research Branch (ARB). Of this funding, $6.3 million was allocated in the form of grants and the remaining $24.1 million was allocated in the form of contributions. For the past three fiscal years, the program has administered an average of 200 contribution agreements and 30 grants. An additional $9.7 million was allocated for operating funds. Approximately $11 million ($9 million program/$2 million operating) represents temporary funding arrangements. Program spending for totalled $30.4 million. Project funding (contributions) is available for: (i) the development and testing of models to improve the capacities of individuals to participate fully in society; (ii) to strengthen related services provided in communities; and (iii) for the development and distribution of information to client groups and key decision makers at the policy, program development and delivery levels. 2

13 Organizational funding (grants) is available for a specified period of time to selected national voluntary organizations that support activities in line with the department s mandate. Such funding is intended to assist in building the capacity of these organizations and encourage the viability of critical partners. As part of the audit, we assessed the progress achieved in responding to the findings contained in the October 2000 Office of the Auditor General s report. Our assessment found that most of the OAG findings have been addressed by the program. The following issues, however, continue to be addressed: Contribution agreements frequently lacked key information; Deficiencies in handling payments; Inadequate project monitoring; Grant proposals not fully assessed in all cases; Weak rationales for recommending the approval of grant projects. See Appendix E for details of our assessment. Audit Objectives The objectives of the audit were to provide assurance that: SDPP is appropriately managed; and Risks are identified and appropriately addressed. The audit covered aspects of the management control framework with a focus on: accountability for results; quality of monitoring and file administration; and compliance with terms and conditions. Methodology As SDPP is a national, centrally-delivered program, the audit was conducted entirely at NHQ from May to September, The management framework was assessed for the program by the following means: Control self-assessment (CSA) of the program by program management and selected staff; Interviews with program staff and management; Examination of relevant documentation including file reviews; and Analysis of information. The file review consisted of a sample of 40 project files. The sampling period covered agreements with a start date between September 1, 2001 and August 31, This period ensured that ground already covered by the Office of the Auditor General was not repeated and that adequate time had elapsed to incorporate program responses. (See Appendix D for Audit and Evaluation s results). All of the reviewed files were under the old terms and conditions. A sample of 40 files was felt to be sufficiently robust to provide assurance with respect to the audit objectives; however, it should be recognized that a sample of this size is not statistically valid and thus it will not allow statistical inference to the program as a whole. All file review findings of the audit were presented to appropriate program management and staff both at Human Investment Program Branch and Strategic Policy Branch for discussion and validation. Extensive interviews with SDPP management and staff were conducted during the spring and summer of

14 An Audit Advisory Committee was established for the duration of the audit. Advice was sought from SDPP management in the selection of members, with the Audit Director chairing the committee. The advice of the committee was sought to elaborate on the audit s scope, objectives, criteria and approach, as well as to provide another level of assurance with respect to significant matters and issues that were reported. Scope The audit assessed the management framework and operational processes within SDPP and was based upon the 14 audit criteria as listed in Appendix A. The files reviewed pertained to work conducted by the auditee between September 1, 2001 and August 31, 2002, under the old terms and conditions. This internal audit was conducted in accordance with both the Treasury Board Policy on Internal Audit and the Institute of Internal Auditors Standards for the Professional Practice of Internal Auditing. The planned analysis of the implementation of the management action plan in response to the summative evaluation report by EDD as mentioned in the SDPP Terms of Reference was not completed because the report was not finalized at the time of the audit. As mentioned in the Terms of Reference, the audit was to rely on the file review results of the Performance Tracking Directorate (PTD) as much as possible. It was necessary however, to alter this approach as the PTD files within the sample period were not completed files. That is, files reflecting projects which have finished all or most of the seven life-cycle steps, i.e. application, assessment, recommendation & approval, agreement, payment, monitoring, and close-out. Therefore, many of the responses to the questions regarding the later stages of the project life cycle were N/A, as many of the projects had not yet reached the payment stage. While a comparison is not appropriate, the PTD results are attached for information purposes, in Appendix C. 4

15 3.0 Audit Findings All significant audit findings are presented in this section in accordance with the audit objective(s) and criteria, which are described in detail within Appendix A: Audit Objectives, Criteria and Methodology. They include assurance statements on all of the criteria regardless of whether or not the performance expectations have been met. 3.1 Program Mandate/Strategy Audit Criterion 1.1: Program objectives are clearly stated, understood and measurable. Overall, we have found that the new terms and conditions more clearly state the program objectives, which were not clearly stated, nor well understood under the old terms and conditions. The majority of program officers, managers and staff interviewed, as well as the CSA results, implied that the SDPP objectives included in the old terms and conditions were very broad, unclear, and not well understood. However, when asked the same question of the new terms and conditions, the program managers stated that the objectives were much more focused, clear, and understandable. The program officers, however, stated that new objectives were still too broad. Information related to the measurability of the objectives is provided under criteria 2.2. Our review of the new terms and conditions and related documentation, as well as interviews and the control self assessment results indicate that this deficiency is being addressed. This is demonstrated in that the new program objectives are expressed in terms of expected outcomes, as well as containing a clarification of eligible activities. The described links among objectives, activities and expected outcomes contribute to this clarification. This criterion is, therefore, met. Audit Criterion 1.2: The program s policies, procedures and operational guidelines are clearly defined, timely, available and consistently applied. On the whole, we found the above criterion was met. Under the six-point action plan for the management of HRDC grants and contributions programs, the department developed a generic Operations Guide, and Model Files. These procedures apply to the delivery of the SDPP program. As well, there is an HRDC grants and contributions website encompassing all aspects of program delivery including policy, procedure, acts, regulations etc. SDPP also has its own Operations Guide for the management of the program. SDPP management has stated that the SDPP Program Policy and Operational Guidelines were viewed as a complement to the HRDC manual because the departmental manual focuses primarily on employment programs that are significantly different from the knowledge development and capacity building objectives of the SDPP. 5

16 During the file review phase of the audit, we utilized these resources extensively and found them, for the most part, useful, accurate, complete, and updated. There were a few inconsistencies in the operations guide however, which caused some degree of confusion on the part of the users, i.e. the treatment of overhead percentage rates. This is discussed further under audit criteria 2.4. Another item requiring clarification was the issue of whether or not grants require a formal agreement between the department and the recipient. During our file review, we found no evidence of risk assessments, nor formal agreements in any of the 13 grant files reviewed. The Operations Guide in section indicates that a letter of confirmation may be all that is required, while the Treasury Board Policy on Transfer Payments in section introduces the elements of risk and materiality into the equation, with the emphasis on the requirement of a written agreement, in situations of medium or high risk and materiality. The SDPP Terms and Conditions, section 8, or 13 in the newer version, specify a formal agreement is required, without mention of neither risk nor materiality. And, finally, FAS Grants and Contributions Policy, section 3.2 stated that the program s Terms and Conditions take precedence over the Operations Guide. While we can identify with the program officers confusion over this issue, the requirement of a formal agreement, as stipulated in the program s terms and conditions is evident, as is the requirement for a risk assessment, as stipulated in the Treasury Board Policy. The application of the policies, procedures and guidelines will be discussed under criterion 2.4. Recommendation 1: SDPP management should require that all grant agreements be formalized, stipulating expectations and requirements of all parties involved and that all grant projects be subject to a risk assessment. Audit Criterion 1.3: Planning and resourcing exercises are regularly undertaken to ensure that the program meets its objectives. Overall, we found that most activities constitute a sound basis for a more formalized planning and resourcing exercise, and therefore the criterion is considered to be met. Extensive interviews with management determined that planning and resourcing exercises only started in The Social Development Directorate and the Office for Disability Issues have developed planning activities which will roll up to the branch and departmental levels, and Strategic Policy produces both a business plan as well as a research plan. Priorities are determined via management retreats, consultations with key stakeholders as well as provincial and territorial counterparts. Program funds are supplemented through an annual departmental pressures exercise, which basically re-aligns B base funds along current priority lines, but this is problematic because of the timing of the release of funds through this process, usually in September, well after the project intake phase. At the time of our review, planning documents were either in draft form, or not completed. Consultations with stakeholders and central agencies are undertaken to discuss and determine the program s priorities. 6

17 At the time of our fieldwork, Child, Family and Community, (CFC), Community Engagement, (CE) and the Applied Research Branch (ARB) had all indicated their utilization of a call for proposals process for contribution project intake purposes. There was a special call related to Services in Early Childhood Learning and Care, and another, Development of Social Policy Capacity in the Voluntary Sector. The use of formal calls for proposals posted on the internet leads to a more transparent, targeted approach to program resource planning. SDD has also undertaken an external study to assist SDD and ODI in identifying approaches for improving the efficiency of their Call For Proposals process. During the file review process, the majority of grant files reviewed contained a letter inviting the current recipient to re-apply for the following year s funding, thereby, in our view, potentially restricting available funding for other potential applicants. This also raises a concern about the transparency of the intake process, when considered in light of the Program Evaluation report which mentions a high concentration of funding among small number of sponsor organizations. Audit Criterion 1.4: Management understands the risks facing the program and a risk mitigation strategy is in place. It was observed that overall, this criterion was met. SDPP underwent a risk self assessment (RSA) exercise in June, At that time, a set of program wide risks were developed, as well as a related set of mitigating strategies. The management team identified four risk areas that may impact the achievement of the program s intended outcomes: program impact; organizational capacity; long-standing relationship with partners; and organizational funding. Mitigation strategies were also identified to address these risk areas. Based on the results of this session, other sources of information and subsequent discussion, the overall risk presented by the program has been assessed by program officials to be moderate. During our interviews, management indicated the continued relevance of both these risks and their mitigating strategies and stated that the new Terms and Conditions were drafted with a view, in part, to addressing the identified risks. We have learned about additional strategies aimed at addressing the above risks, including the following: A strategic development unit is to be created, with a focus on the risk related to program impact ; Recruitment has changed, with a focus on generic skills; Using the call for proposals to be more open and transparent. We did not examine the implementation of the mitigating strategies. 7

18 3.2 Accountability for Results Audit Criterion 2.1: Roles and responsibilities are clearly defined, understood and transparent. Overall, the above criterion has been met. The rather complex management structure of SDPP demands a high degree of understanding of the roles and responsibilities among the numerous players in two branches and five directorates. At the time of the audit, the program was administered within the Human Investment Programs (HIP) Branch through the Office for Disability Issues (ODI) and the Social Development Directorate (SDD). The SDPP terms and conditions were also used by Strategic Policy Branch (SPB) via a Memorandum of Understanding to support SDPP s social development objectives and priorities and inform policy. The MOU outlines roles and responsibilities to ensure clear accountabilities in delivery and management of the research and projects managed by SPB. However, overall program accountability for SDPP lies with the Assistant Deputy Minister responsible for the ISP. The Program Policy Unit was established with a mandate to address two areas under a program renewal initiative: Implementation of the department s Six Point Action Plan for the grant and contributions programs within the directorate; Review and renewal of the directorate s grant and contributions programs. The manager of this area was in constant contact with the similar position in Strategic Policy. They shared the same SDPP working tools, established funding priorities, and gave the HIP input into the broader picture. Our interviews with program management indicated a reliance on this unit for clarification and policy direction, as well as a general satisfaction with the unit. There existed a degree of confusion, however, over the role of the program officers, specifically how they fit in the bigger picture and their relations with the Program Policy Improvement Unit. See also comments on Criterion 2.5 Audit Criterion 2.2: Performance indicators and mechanisms (including data-capture infrastructure) are timely, relevant, accurate and in place to measure and report on project and program performance and outcomes and are used for decision-making. We consider this criterion to be partially met. An RMAF and Logic Model has been developed to provide the basis of a performance measurement strategy for SDPP. This will be used to measure the success of the program in achieving its outputs and outcomes. This RMAF was approved March 28, The performance measurement strategy outlined in Appendix B of the RMAF outlines performance indicators for outputs, immediate, intermediate and long-term outcomes. 8

19 The information sources include project files, reports from funded organizations, focus groups and survey of funded organizations and opinion leaders. This forms an integral part of the ongoing performance measurement strategy. At the time of the audit, there were neither performance indicators nor mechanisms (including data-capture infrastructure) in place to measure and report program performance and outcomes. As part of the evaluation strategy, HRDC Program Evaluation committed to developing a strategy for collecting baseline data from Social Non-profit Organizations funded under SDPP. Subsequently, a contract for the development of a data collection strategy for the SDPP, as well as a detailed evaluation framework for the revised program was let on December 22, The contractor is to deliver a data collection strategy in consultation with a representative selection of social, non-profit organizations from both the SDD and ODI program areas (including representation of Official Language Minority Communities) to determine their capacity for collecting and maintaining those data elements identified in the SDPP RMAF as necessary to speak to the ongoing performance and evaluation indicators. We have reviewed the contract for the provision of the SDPP Data Collection Strategy and Evaluation Framework and are satisfied that it adequately addresses the formation of the data collection strategy put fourth in the RMAF. The contract is scheduled for completion by July 23, Audit Criterion 2.3: Relevant performance information is conveyed in reports to Parliament. We consider this criterion as partially met. Our review of the , and the Departmental Performance Report contained a listing of program objectives and a statement on broad qualitative information. This criterion should be met once the performance information becomes available as mentioned under criteria 2.2. Audit Criterion 2.4: Administrative and financial controls have been designed and implemented. There are financial and administrative controls in place and these are used; however, improvements are needed, and therefore we consider this criterion partially met. This was confirmed by the findings of the file reviews performed by the audit team (Appendix D). This review was based on a checklist of questions, pertinent to application, assessment, recommendation and approval, agreement, payments, monitoring and close-out phases of the lifecycle for grants and contributions. Although, a high compliance rate was achieved in a relatively high number of questions, we found weaknesses in the following administrative areas. All of the files reviewed were under the old terms and conditions: 9

20 Applications included descriptions of expected results that are measurable (15/27 compliant files); Inclusion of measurable and quantifiable objectives in the agreement (i.e. expected results clearly described) (14/27compliant files); Agreements include a schedule comprising project timeframes and milestones (20/27 compliant files); Agreements include an appropriate breakdown of expenditures (21/27 compliant files); Agreements include a forecast of cash flow (19/27 compliant files); Commitments completed prior to HRDC signing the agreements (15/27 compliant files); Clarity on why advance payments are made instead of cost reimbursement payments (9/27 compliant files); Completion of monitoring in accordance with risk assessment and monitoring plan (13/27 compliant files). We noted under criterion 1.1 that the old program objectives were not clearly stated and understood. This could explain the level of non compliance reflected in the first two findings above. During our review of the finance and administration processes, we have made the following general observations: The main source of operational direction is the departmental Operations Guide. This is the document that sets out the specific processes to be followed during the life-cycle management of either a grant or contribution project. The contribution life-cycle forms the main thrust of the guide and procedures. Grants, on the other hand, are mentioned when there is a procedure specific to grants only. The result of this is that grants are not presented in a life-cycle format, making it difficult to determine a specific set of grant guidelines or procedures. This has led to a lack of standardized direction for the management of the grant life-cycle. With a current increased emphasis on grants by the program, an enhancement of the grant life-cycle in the operations guide would be both timely and beneficial. As previously mentioned under criterion 1.2, there is a need for the Operations Guide to clarify issues of evaluating the methods used in calculation of percentage overhead rates, and the application of these rates. As we noted during our file review, more and more recipients are using this method, and it is a standard approach for most universities. The program s use of the Agreement Monitoring Risk Assessment form, is a good tool for a snapshot of an agreement s risk profile, but it needs to be improved by incorporating other risk factors specific to this program. For multi-year agreements, the risk profile should be determined each year as the project evolves and monitoring adjusted accordingly. None of the 13 grant files we sampled had evidence of a risk assessment. One project had exceeded the Terms and Condition s project annual funding limit of $ 2,000,000 by approximately $1,000,000. This project, after amendments, totaled $2,981,913, or almost 10% of the entire SDPP budget. While SDPP detected the problem after the fact, there is a need to improve the effectiveness of the financial commitment controls, especially in the cases of amendments. 10

21 One project was inappropriately funded as the agreement contained clauses which constituted a service contract. This not only contravenes the program Terms and Conditions but also the Financial Administration Act. The pay-out on this project, over 3 years, totaled $170,000. There is a need to reinforce oversight controls previously put into place by the MOU between HIP and SP, as HIP is ultimately accountable for all agreements under the HIP-SP MOU. A number of project files contained numerous photocopies of recipient s accounting journals which the program officers try to agree to claimed amounts. Many of these journals are produced by relatively sophisticated automated accounting systems and are organized using a series of pre-determined account and project codes. In some cases, the recipients are running many projects and various bank accounts through the same system. Without a fairly intimate knowledge of accounting structure, and the particular coding in use, most readers of these journals would tend to be more confused, rather than satisfied, as to the accuracy of claimed amounts. Interviews and discussions with the program officers indicated that they question this practice, finding little or no errors after spending considerable amounts of time. We question the cost-effectiveness of this practice, given the time involved, especially with recipients having many projects, bank accounts, applied overhead rates, etc. Also contributing to our observation is the fact that these are photocopies, and that, in many cases, the project officer is seeing only a small slice of the entire accounts. Recommendation 2: Social Development Canada should develop an expanded grant section in the Grants and Contributions Operation Guide. Recommendation 3: Social Development Canada should update the Grants and Contributions Operations Guide to clarify the application of the percentage overhead rates. Recommendation 4: SDPP management should review and improve the claims verification practices at the payment stage of the project life cycle. Audit Criterion 2.5: There are processes in place to clarify policies, resolve issues, and ensure good communications with partners and stakeholders. As mentioned in criterion 2.1, a Program Policy Improvement Unit has been set up within the Social Development Directorate with a mandate to clarify policy and provide direction. This aids management in the interaction with partners and stakeholders by ensuring a consistent message on policy issues. Our documentation review as well as interviews with management and a check of the SDPP web sites indicated various methods used to communicate with partners and stakeholders, i.e. the Call For Proposals process, mailing lists, , websites, letters to sponsors, phone calls, as well as program officer s direct contact. The program also relies on direct contact between directors and stakeholders/partners when issues arise. In terms of program information, selection criteria, and other information available to the general public, the program relies 11

22 on the directorates main web site, with SDPP as a sub category, under the category Programs for Education and Research. This will eventually link the reader to the new Terms and Conditions, listed in the SDD site. Within this section is an explanation of the program, selection criteria, application procedures etc. Although the access to the specific SDPP information is a bit awkward, the use of a search engine gives immediate results. The program could benefit from better designed and organized Internet information. Given the diversity of organizational cultures involved in delivering the program, we found that adequate processes are in place and working and, therefore meet the criterion. Audit Criterion 2.6: Recipients/sponsors meet program eligibility criteria. The program s assessment practices, in light of the program terms and conditions, including the use of assessment grids, internal and external review panels, program officer review, and manager s sign-off, all ensure that there are steps and criteria in place to evaluate the eligibility of recipients. Also, we did not find any significant issues in our file review. Overall, this criterion has been met. 3.3 Supporting Program Capacity Audit Criterion 3.1: Program staff has access to needed resources, information, skills, tools and training to ensure successful delivery. Overall, we do not feel that this criterion has been met. Our interviews showed that operational training resources are available to program staff. Learning plans, information sessions, workshops, seminars, subject-specific courses, and other activities geared towards improvement were mentioned. The Operations Guides, mentioned under criteria 1.2, are available on-line to all potential users. Policy directives, interpretations and bulletins, are also available, as are informal peer networks. The program officers are tasked with many front-line activities which require a fairly diverse skill set. Our file reviews, interviews and the results of the control self-assessment would suggest an underlying frustration on their part concerning their financial control functions. As recipients evolve into ever more sophisticated organizations, utilizing state of the art automated accounting systems, accrued accounting techniques, multiple projects with shared cost strategies etc. the program officer is feeling less and less equipped for the task. On this point, the Operations Guide states : If Program Officers are having difficulty understanding the financial information provided by sponsors, they must get help from someone with an accounting background (a co-worker or a financial officer). While this strategy may work with the majority of one-project, low-dollar, low-risk, single bank account recipients, it is not relevant to the aforementioned more complicated situations. 12

23 Recommendation 5: In order to maximize the program delivery skills of the program officers, SDPP management should consider alternatives to the current financial monitoring routines, bringing in a more formal approach to assessing recipient s financial and accounting status. This may be achieved by the utilization of a dedicated program-wide financial officer to assist and accompany the program officers on the larger, more complicated assessments, on an as-required basis, as well as an adjustment to the risk assessments and monitoring plans, as noted in criterion 2.4 above. This would dove-tail with specialization initiatives being considered by HRDC at the time of the audit. Audit Criterion 3.2: There is sufficient internal communication to ensure that program employees have consistent, accurate and current information within and across the programs. Interviews with management and staff indicate that there are both formal and informal communications systems throughout the program which adequately serve the needs of majority of staff. Formal and informal processes and practices used for internal communications include regular staff meetings, sharing of s, all-staff meetings, managers bi-weekly meetings, guidelines and policies forwarded to staff and communicated verbally, networking, points of contact, briefing sessions on issues, etc. Overall we consider this criterion met. Audit Criterion 3.3: A model exists of what a good or ideal contribution agreement and grant for the program would look like. The Model File and the Grants and Contributions Operations Guide provide guidance on the format and contents of an ideal agreement. Some directorates are customizing the formats to suit their individual needs and the needs of the recipients. Overall we consider this criterion met. 3.4 Program Monitoring Audit Criterion 4.1: Contribution agreements and grants are being effectively monitored as part of the Quality Assurance Framework to ensure funds are being spent according to the terms and conditions of the contribution agreements. To meet this criterion, there was an expectation that monitoring and quality assurance of project activity and financial administration is conducted by Program Officers (POs) and Program Operations Consultants (POCs). 13

24 HRDC s Monitoring Policy as stated in the Operations Guide requires adherence to the Financial Administration Act, the Treasury Board Policy on Transfer Payments and their principles. Agreement monitoring is intended to: a) Exercise control over the expenditure of public funds, to ensure that: Records reflect what has been claimed; Supporting documentation is available to show that expenditures took place (i.e. for items claimed, look for original invoices and cancelled cheques for each invoice examined); Expenditures have not been charged to another funding source as well (which is sometimes referred to as stacking of assistance); Project-related expenditures are reasonable (i.e. fair prices have been paid for the goods and services purchased) and claimed amounts are allowable; Revenue generated by funded activity (for example, interest earned, revenue generated by project activity and funds received from other sources) is clearly identified; Payment and advances are in keeping with the expenditures; and The closing out of the agreement completes all specified financial obligations. b) Ensure that project activities are conducted as set out in the agreement; and c) Ensure expected results are achieved as described in the agreement. Of the 27 project files examined for compliance to this criterion, 13 had risk assessments and monitoring plans that were appropriate for the risks presented by the projects. However, 11 had actual monitoring activities that were not in accordance with the developed plan. This was mainly downward deviations in the number of required monitoring visits from the plan. Other areas where we found weaknesses included no evidence of verification of per-diem salary rates to actual costs, no evidence of verification of claimed overhead rates, tracing to original invoices, journals, and other supporting documentation. See Criterion 2.4 for a further discussion of financial and administrative activities. The review of project activities was found to be inadequate, often resulting in the activities section of the monitoring form being left blank, or with minimal descriptions. The activity report, often provided by sponsors was cited by the POs as a way of monitoring activities. While this is a valid way of reporting activities, we believe that an on-site visit is required for an accurate assessment of project progress. The monitoring form, which is duly signed and dated by the program officer, is not countersigned by the recipient, nor is a copy left on-site. This good practice would ensure that all parties are in agreement with the findings, which could be of benefit should any problems arise. In our view, the current use of the Common System for Grants and Contributions is definitely a step in the right direction. Its structured, sequential approach to processing goes a long way to ensuring mandatory monitoring activities are not neglected. 14

25 Our interviews and control self-assessment revealed that only one directorate was using a Program Operation Consultant. In other directorates, program officers were filling the role as best they could. SDPP management has stated that the Quality Assurance Framework (QAF) responsibilities were located at the branch level in Strategic Integration (SI), within HIP. SI managed the QAF through a variety of functions and involvements with Directorates within the branch, including file reviews, training and a committee structure. In addition, specific individuals within the directorates were responsible for ensuring compliance for financial integrity as it was the departmental focus. The responsibilities of the Program Policy Unit for the implementation of the Six Point Action Plan provided an additional level of quality assurance. While we found some positive aspects in regards to monitoring practices, the criterion was not met because of the above weaknesses identified in the monitoring process. Recommendation 6: SDPP management should improve activity and financial monitoring by ensuring that: Actual monitoring activities are conducted in accordance with the developed monitoring plan; Verification of claimed amounts to original documentation are done with more rigour, including verification of overhead rates, verification of per-diem salary rates to actual costs; A more predominant role is given to program activity monitoring; Costs claimed are linked to project milestones achieved. 15

26 16

27 4.0 Conclusion Based on the evidence examined in support of the audit criteria, we conclude that the Social Development Partnerships Program is being adequately managed and risks are being identified and addressed. More work is needed, however, to improve certain aspects of internal control and project and program monitoring. In our professional judgment, sufficient and appropriate audit procedures have been conducted and evidence gathered to support the accuracy of the conclusions reached and contained in this report. The conclusions were based on a comparison of the situations as they existed at the time against the audit criteria. The conclusions are only applicable for the Social Development Partnerships Program. This internal audit was conducted in accordance with the Treasury Board Policy on Internal Audit and the Institute of Internal Auditors Standards for the Professional Practice of Internal Auditing. 17