Report of External Evaluation and Review. New Zealand Quality College

Transcription

1 Report of External Evaluation and Review New Zealand Quality College Date of report: 3 August 2010

2 Contents Purpose of this Report... 3 Brief description of TEO...3 Executive Summary... 4 Basis for External Evaluation and Review... 6 Findings... 7 Part 1: Answers to Key Evaluation Questions across the organisation...7 Part 2: Performance in focus areas...12 Actions Required and Recommendations MoE Number: 8170 NZQA Reference: C01565 Dates of EER visit: 2 and 3 June

3 Purpose of this Report The purpose of this external evaluation and review report is to provide a public statement about the Tertiary Education Organisation s (TEO) educational performance and capability in self-assessment. It forms part of the accountability process required by Government to inform investors, the public, students, prospective students, communities, employers, and other interested parties. It is also intended to be used by the TEO itself for quality improvement purposes. Brief description of TEO Location: Type: Size: Sites: Greenlane, Auckland Private Training Establishment Approximately 350 course attendees annually Temporary sites nationally New Zealand Quality College (NZQC) is a private training establishment owned by International Accreditation New Zealand (IANZ), a statutory accreditation body. The organisation was registered with NZQA in NZQC delivers internal auditing, external auditing, risk and compliance training, and food safety training. NZQC also offers auditor refresher courses and advice on developing and implementing quality management systems. NZQC offers general courses as well as in-house courses tailored specifically to the individual company s quality systems. The courses are suitable for people who work in Telarc-certified organisations or International Accreditation New Zealand (IANZ) accredited organisations such as laboratories and local government bodies. The tutors are registered auditors and are contracted part-time by NZQC to deliver training in their area of expertise. Courses vary from one-day seminars to a two-day internal auditor training course and up to five days for a full external auditor training course. Trainees can use their external auditor course certificate to apply to become an accredited auditor with RABQSA International. 3

4 Executive Summary Statement of confidence on educational performance NZQA is Highly Confident in the educational performance of New Zealand Quality College (NZQC). The organisation s records show that a high number of clients throughout New Zealand use NZQC for training for their employees. The average number of attendees at the Internal Auditor Training courses (known as IQA) has been between 100 and 150 per year over the last four years. This indicates that the course has maintained demand among the target training base. Course outcome records show that all attendees complete the course requirements. The relationship NZQC has with IANZ means that it is able to identify the training needs of industry and respond accordingly. NZQC responds to feedback from clients and attendees to make changes to the training programme, as demonstrated by the tailoring of all course materials to suit individual clients quality management systems. NZQC gathers feedback from attendees on their satisfaction with the training, rating it against their expectations and whether these were met. Analyses of feedback show that the majority of course attendees expectations were exceeded. In addition, NZQC seeks feedback from employers on whether the training has improved their employees work performance. Although employer feedback was limited, it was sufficient to demonstrate that the training contributed to positive outcomes in the workplace. Overall, the organisation shows a commitment to ensuring that the attendees needs are met by providing comfortable venues, relevant pre-course information, qualified tutors, and post-course support. Statement of confidence on capability in self-assessment NZQA is Highly Confident in the capability in self-assessment of New Zealand Quality College (NZQC). NZQC has a high awareness of trainee and employers training needs as indicated by its decision to develop a risk and compliance course in response to a decrease in course registrations. The decrease was a result of many accredited organisations already having put the appropriate employee/s through the initial internal auditor training. Employers had identified a further training need for employees on identifying risk and compliance requirements within their organisation. Attendees are asked to rate their expectation of the course against their views post-course. The feedback forms also ask for feedback on how well the course was run and whether any improvements could be made. NZQC has used this information to make improvements to its delivery and content, and repeat demand for the course indicates that the changes made 4

5 are appropriate. Overall, most stakeholder responses rated the training as highly satisfactory. TEO response NZQC has had an opportunity to comment on the accuracy of this report, and submissions received have been fully considered by NZQA before finalising the report. 5

6 Basis for External Evaluation and Review Self-assessment and external evaluation and review are requirements of course approval and accreditation (under sections 258 and 259 of the Education Act 1989) for all TEOs that are entitled to apply. The requirements are set through the course approval and accreditation criteria and policies established by NZQA under section 253(1)(d) and (e) of the Act. In addition, for registered private training establishments, the criteria and policies for their registration require self-assessment and external evaluation and review at an organisational level in addition to the individual courses they own or provide. These criteria and policies are set by NZQA under section 253(1)(ca) of the Act. NZQA is responsible for ensuring TEOs continue to comply with the policies and criteria after the initial granting of approval and accreditation of courses and/or registration. The Institutes of Technology and Polytechnics Quality (ITPQuality) is responsible, under delegated authority from NZQA, for compliance by the polytechnic sector, and the New Zealand Vice-Chancellors Committee (NZVCC) has statutory responsibility for compliance by universities. This report reflects the findings and conclusions of the external evaluation and review process, conducted according to the policies and criteria approved by the NZQA Board. The report identifies strengths and areas for improvement in terms of the organisation s educational performance and capability in self-assessment. External evaluation and review reports are one contributing piece of information in determining future funding decisions where the organisation is a funded TEO subject to an investment plan agreed with the Tertiary Education Commission. External evaluation and review reports are public information and are available from the NZQA website ( 6

7 Findings The conclusions in this report are derived using a standard process and are based on a targeted sample of the organisation s activities. Information relevant to the external evaluation and review process, including the publication Policy and Guidelines for the Conduct of External Evaluation and Review, is available at: Outline of scope The agreed scope of the external evaluation and review (EER) of NZQC included the following mandatory focus area: Governance, management, and strategy. The following courses were also included as focus areas: Internal Auditor Training (IQA) Risk and Compliance. The IQA course was chosen because it has the highest number of trainees throughout the year. The Risk and Compliance course was chosen as it is a new course and NZQC is interested in monitoring its value for future courses. Part 1: Answers to Key Evaluation Questions across the organisation This section provides a picture of the TEO s performance in terms of the outcomes achieved and the key contributing processes. Performance judgements are based on the answers to key evaluation questions across the focus areas sampled. This section also provides a judgement about the extent to which the organisation uses self-assessment information to understand its own performance and bring about improvements, i.e. capability in self-assessment. 1.1 How well do learners achieve? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Excellent. Explanation NZQC was established as the training arm for IANZ, which is a state-owned accrediting organisation. NZQC has around 5,500 clients who register their employees on courses with NZQC to increase their knowledge of quality systems auditing. The records show that the 7

8 trainees are achieving what they need to in order to perform their job, and repeat training is requested by the same companies. There is no formal written assessment of the knowledge and skills provided by the course. Tutors monitor trainees progress throughout the course and trainees receive a certificate of attendance if they demonstrate their knowledge of audit practices and are observed competently conducting audit techniques. NZQC analyses attendee training course evaluations, and responses are graphed against expected outcomes. The results of the analyses show that the courses are exceeding participants expectations. In addition, NZQC obtains feedback from employers. Although few employers respond, the feedback received showed that employers believed the training greatly improved employees auditing knowledge. 1.2 What is the value of the outcomes for key stakeholders, including learners? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Excellent. Explanation NZQC provides training for people in organisations that are either accredited with IANZ or certified by Telarc. The value of the training is indicated by the number of attendees on courses and feedback from attendees and employers. Although overall course attendance numbers are still around 350 per year, the organisation experienced a small drop in course attendance in The decrease has been mostly in requests for in-house courses tailored for specific company quality systems. The organisation believes this is a result of regular clients cutting back training budgets in response to the current economic recession. However, NZQC has already delivered IQA courses for 70 people in 2010, which indicates a rise in demand for training compared with The organisation identified possible reasons for this as the development of targeted resources for local government bodies that process building consents and issue building codes of compliance certificates, as well as a recent mailout advertising available courses. The feedback from attendees shows the IQA course is highly valued as it improves their knowledge of auditing their employing organisation s internal quality systems. In addition, many employers have been enrolling employees on NZQC courses over a number of years, which indicates that the IQA course is still relevant and valued by IANZ-accredited organisations. This judgement is supported by post-course employer evaluations of the training and feedback gained from employers interviewed for the external evaluation and review. 8

9 1.3 How well do programmes and activities match the needs of learners and other stakeholders? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Excellent. Explanation The IQA course is delivered over two days. The training consists of case studies and group activities which allow attendees to demonstrate their internal auditing knowledge and practices. This internal auditor course assumes people have prior knowledge of quality management systems. Trainees feedback is used to look at trends in whether tutors had a good rapport with trainees and if the resources and venue met expectations. The organisation has analysed feedback from trainees and a high number report that the course exceeded their expectations. Any suggestions for improvement are reviewed by the general manger and adjustments made. Examples of this were the inclusion of introductory training sessions on quality management systems and the use of clients own documentation for in-house training courses to suit people employed by local government bodies that process building consents and compliance notices. The increased demand for IQA courses in 2010 indicates that the changes are meeting the needs of this client group. New courses are developed according to the needs of the clients, which are identified by the general manager through visits to client organisations. The risk and compliance course is a recent example where a course has been developed to meet the needs of clients that already had internal auditing systems and needed staff training on identifying risks to meet compliance requirements. The course has been delivered once and the response was positive from both employers and attendees. NZQC maintains a list of organisation that request training and this shows that many have used NZQC to provide internal auditor training over a long period of time. NZQC also includes all the people they have trained in a list plus an analysis of how many courses they have attended to identify training gaps for employers. This assists NZQC to identify where accredited organisations current staff require upskilling to continue maintaining the relevant standards. NZQC regularly meets with the chief executive officer of IANZ to review course outcomes against the goals outlined in the organisation s statement of intent. This is to identify issues with the training and to make improvements, if required. Overall, the evidence demonstrated NZQC s ability to respond to current stakeholder needs and assist employees of IANZ-accredited organisations to gain the required knowledge for their roles. 9

10 1.4 How effective is the teaching? The rating for performance in relation to this key evaluation question is Good. The rating for capability in self-assessment for this key evaluation question is Good. Explanation The successful completion of the course is achieved by the tutors assessment of whether the attendees have the core competencies required to do an audit. The general manager reported that everyone achieves this with the assistance of the tutors, who allow for additional time for attendees to understand and demonstrate the required skills. NZQC has no formal policy on qualifications for teaching staff. However, the list of tutors provides assurance that NZQC contracts qualified auditors who have been delivering auditor training for a number of years. Two of the tutors, one of whom is the general manager, provide the IQA course. The general manager developed a handbook to assist with consistent delivery of the course content. The delivery is learner-centred and implemented according the needs of the client group. The feedback from trainees indicates that the methods used are effective in engaging attendees to achieve the required outcomes. In addition to post-course employer feedback, training effectiveness is monitored by viewing the internal audit reports of organisations that have employees attending NZQC courses. The general manager also gains informal feedback by speaking to employers after the course. The feedback indicates that attendees knowledge of quality systems management and auditing has improved greatly. The general manager checks that the training materials cover all the core competencies; tutor observations are conducted for new tutors. However, the general manager has identified a need to improve monitoring of the training delivery to ensure it is aligned to the course requirements. The general manager has also identified a need for a more formal performance measures to monitor tutors performance as well as analyse trainees feedback and compare course outcomes between tutors. This has been difficult to achieve because some tutors only deliver one course per year. In addition, it is difficult to arrange suitable time to meet as all tutors are working in New Zealand and internationally. 1.5 How well are learners guided and supported? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Excellent. Explanation Employers reported that NZQC was responsive to requests for information and that the administrative support from NZQC was helpful. NZQC determines the training needs of attendees on application to ensure they are enrolling in the appropriate course. Once the course attendance is confirmed, course information is sent to the employer and/or attendee, and both are notified of the outcomes of the course, as requested. 10

11 Information about the course is advertised on the NZQC website and regularly updated with the latest course timetables and applicable industry standards. The feedback from attendees was very positive regarding the support they received on the course to assist them to meet the course requirements. The organisation is responsive to the feedback on venues from attendees to ensure that the training is delivered in an environment conducive to attendees learning needs. 1.6 How effective are governance and management in supporting educational achievement? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Excellent. Explanation The organisation s general manager and the chief executive officer of IANZ meet weekly to review whether NZQC is achieving its goals. The organisation reviews course attendance to determine whether the courses are in demand. Management also checks the number of courses to confirm that there are sufficiently trained tutors to deliver the courses. NZQC contracts tutors to deliver most of its courses. Most training delivered in the last two years has been for the internal and external auditor training course. The organisation only delivers a few courses for laboratory and food safety compliance each year. The management team has shown considerable responsiveness to the training needs of clients and of attendees in the tailoring and development of courses to meet employer needs. NZQC adheres to the policies and procedures outlined in the IANZ quality management system. The quality management system states that, in line with business goals it shall be the policy of NZQC to offer training courses that will facilitate domestic commerce, external trade and the health and well being of New Zealanders. The organisation was able to demonstrate that its training is aligned to the current compliance standards required for industries in New Zealand to conduct business. 11

12 Part 2: Performance in focus areas This section reports significant findings in each focus area, not already covered in Part Focus area: Governance, management, and strategy The rating in this focus area for educational performance is Excellent. The rating for capability in self-assessment for this focus area is Excellent. 2.2 Focus area: Internal Auditor Training (IQA) The rating in this focus area for educational performance is Excellent. The rating for capability in self-assessment for this focus area is Excellent. 2.3 Focus area: Risk and Compliance The rating in this focus area for educational performance is Excellent. The rating for capability in self-assessment for this focus area is Excellent. 12

13 Actions Required and Recommendations Further actions The next external evaluation and review will take place in accordance with NZQA s policy and is likely to occur within four years of the date of this report. Recommendations There are no recommendations arising from this external evaluation and review. 13

14 NZQA Ph E eeradmin@nzqa.govt.nz 14