Report of External Evaluation and Review

Transcription

1 Report of External Evaluation and Review Access Homehealth Limited trading as Amida Training Confident in educational performance Not Yet Confident in capability in self-assessment Date of report: 19 September 2016

2 Contents Purpose of this Report... 3 Introduction TEO in context Scope of external evaluation and review Conduct of external evaluation and review... 7 Summary of Results... 9 Findings Focus Areas Recommendations Appendix MoE Number: 8888 NZQA Reference: C22453 Dates of EER visit: 17 and 18 May

3 Purpose of this Report The purpose of this external evaluation and review report is to provide a public statement about the Tertiary Education Organisation s (TEO) educational performance and capability in self-assessment. It forms part of the accountability process required by Government to inform investors, the public, students, prospective students, communities, employers, and other interested parties. It is also intended to be used by the TEO itself for quality improvement purposes. Introduction 1. TEO in context Name of TEO: Access Homehealth Limited Trading as Amida Training Type: Private training establishment (PTE) First registered: 1 January 1992 Location: Delivery sites: Courses currently delivered: Code of Practice signatory: Number of students: Jackson Street Saxton Football Pavillion, 103 Saxton Road, Stoke, Nelson Training for Work (T4W) programme Unit standards toward the National Certificate in Health, Disability, and Aged Support (Foundation Skills) (Level 2) No Domestic: trainees, 18% (10) Māori and 7% (4) Pasifika trainees trainees, no Māori or Pasifika trainees. International: Nil Number of staff: Scope of active accreditation: 1.5 full-time equivalents National Certificate in Health, Disability, and Aged Support (Core Competencies) (Level 3). Sub-fields at level 4 include Community Support 3

4 and Human Services. Domain accreditation from levels 1 to 4 include Assessment of Learning, community support services, English, core health, communication, financial management; reading, writing and study skills, and health and safety. Consent to assess a range of unit standards in areas of civil of defence operation, community support services, diversional therapy, English language, health education, reading, sensory support and work and study skills. Distinctive characteristics: Amida is the training entity of Access Homehealth Limited. As such, it provides learning and employment pathways to trainees seeking employment in the health and disability sector. Some Amida Training s and graduates are employed by Access Homehealth Limited, and further training may be provided by Careerforce ITO (industry training organisation). Access Homehealth Limited has a staff workforce of 4000 personnel, providing homebased health care services across New Zealand. Contracts are funded by District Health Boards (DHB s), Accident Compensation Corporation (ACC), and the Ministry of Health (MOH). Professional development and training of staff are delivered by Careerforce ITO through a training agreement. Amida Training has delivered the vocational training as part of the Training For Work programme under contract to the Ministry of Social Development (MSD) since This comprises unit standards within its accreditation toward the National Certificate in Health, Disability and Aged Support (Foundation skills) (Level 2) Since March 2016, the programme has been solely delivered in Nelson. Previously, it was also being delivered in Wellington until this ceased in November Delivery of the approved programme, the National Certificate in Health, Disability, and Aged Support (Core Competencies) (Level 3) ceased in 4

5 November Recent significant changes: In March 2015, Green Cross Health purchased Access Homehealth Limited and has been the parent body to Amida Training since then. For governance and management purposes, Access Homehealth continues to manage Amida Training. There have been changes to the governance and management positions since June 2015 and the last external evaluation and review (EER). Since the previous EER, Access Homehealth has identified that employees of Access are trainees of Careerforce, and not of Amida Training. As MSD was unable to provide suitable referrals to the programme, the contract was terminated. The Nelson contract was extended by MSD into 2016, and requested additional programmes. In early 2016, Amida Training moved the head office from Wellington central, and permanent site approval was given for the Petone site. The PTE has recently acquired site approval for the Petone head office. Previous quality assurance history: NZQA conducted an EER of Access Homehealth Limited (TA Amida Training) in NZQA was Not Yet Confident in the organisation s educational performance, and Not Yet Confident in the organisation s capability in self-assessment. A number of recommendations for improvements to systems and self-assessment practice were made. As a result, Amida Training produced a quality improvement plan for to address the following recommendations suggested by the previous EER report 1 : 1. Invest in the capability and capacity for more systematic tracking and analysis of trainee achievement, developing realistic targets, including plans that particularly focus on Māori and Pasifika trainees. 2. Implement periodic programme reviews. 1 These recommendations reflect the scope of the 2015 EER, which included programmes delivered to trainees subsequently identified as Careerforce trainees. 5

6 3. Improve the quality of feedback received from key stakeholders, particularly trainees, to better assess the effectiveness and value of their training. 4. Significantly enhance collaboration with Access to better meet the needs of workforce stakeholders and homecare clients. 5. Develop systems to more systematically review the effectiveness of teaching practice and support and guidance processes. Explore how to work more effectively with the Access cultural advisor. 6. Strengthen the reporting on educational and other key performance measures. External moderation is undertaken by NZQA and the ITO, Careerforce. Amida Training did not meet all requirements for NZQA-managed systems in 2015, but previous moderation history shows that all requirements were met. Post-assessment moderation reports in 2015 from Careerforce indicate that requirements have been met. Other: Amida is working with clients referred to them by MSD, to assist trainees to gain confidence, independence and expand their employment opportunities. The Training for Work programme is connected to the MSD requirements for enrolled trainees, and attendance is mandatory. The unit standards toward the National Certificate in Health, Disability and Aged Support (Foundation) (Level 2), are embedded in the delivery of the Training for Work programme. 6

7 2. Scope of external evaluation and review Initial contact with the TEO commenced in March, at which point discussions regarding compliance with approval, accreditation and registration rules began. Queries were met through a number of phone conversations. A face-to-face scoping meeting was held in late March 2016 with the Amida manager. The EER process, and the potential focus area was identified from this meeting. Additionally, a draft agenda was developed to assist in the undertaking of the EER visit. A selfassessment summary, and a range of other applicable documents were made available prior to the EER visit. The scope of the EER included two focus areas: Governance, management and strategy a mandatory focus area. National Certificate in Health Disability and Aged Support Amida has consent to assess, and is currently delivering unit standards toward this certificate although not an approved programme. 3. Conduct of external evaluation and review All external evaluation and reviews are conducted in accordance with NZQA s published policies and procedures. The methodology used is described fully in the web document Policy and Guidelines for the Conduct of External Evaluation and Review available at: The TEO has an opportunity to comment on the accuracy of this report, and any submissions received are fully considered by NZQA before finalising the report. The EER was conducted in May 2016, over two days. The evaluation team, consisting of two evaluators, reviewed a range of documentation and met with the following groups: Governance representatives (two), consisting of the Amida Training Manager and the Access Homehealth Director; Management and Programme leader (two), consisting of the Amida Manager and Administrator; External stakeholders (8), consisting of employers, and employment services, and an MSD Regional contract Manager by phone Tutor (one), mentor (one), trainees (seven) and graduates (four) from the programme focus area. The on-site visit was conducted with the governance and management representatives, and a mentor, at the Petone head office. Interviews with external stakeholders, trainees, graduates and the tutor were held in the Stoke Access office and the Saxton Football Pavillion, where the programme is delivered. 7

8 Following the on-site visit, two external stakeholders (employer and ITO) were conducted by phone. 8

9 Summary of Results Statement of confidence on educational performance NZQA is Confident in the educational performance of Access Homehealth Limited trading as Amida Training. Amida Training is the education delivery arm managed by Access Homehealth Ltd, and governed by Green Cross Health. Amida Training is a very small provider, that has undergone significant changes to funding and management in the last two years resulting in the closure of sites and a decrease in enrolments that limit the funding available for resources allocated to the management and delivery of the programme. Day to day operational guidance is adequately managed, but there is limited attention to compliance by the parent body, in the interest of ensuring fundamental regulatory requirements are met. Additionally, the alignment of Amida Training to the overall strategic direction of the parent body is currently unclear. These limitations have been met with the recruitment of reputable tutoring staff, with extensive local and industry networks. The engagement with the community, and the level of personal support for trainees, which is offered by the sole tutor is extensive. Employment outcomes are valued and are of quality for the trainees and stakeholders. For the Nelson cohort of the training, Amida training has high employment rates, but these were lower for the Wellington cohort as measured by the MSD indicators. Trainees gain technical and theoretical knowledge and skill that is relevant to areas of home health care. The awarding of the new contract by MSD for 2016 to 2019 indicates the capacity of Amida Training to provide required outcomes and is meeting MSD requirements for pre-employment training in Nelson. Reflections from analysis of achievement and graduate destination data could assist an understanding by the organisation of the value of the programme to the trainees and stakeholders. Trainees gain access to industry experience during study, as well as potential for ongoing employment. The health sector gains a trained workforce with introductory knowledge, as well as some specific home care skills to assist the community with complex health needs. Opportunities for further study are expanded, and individual trainee self-confidence is increased by gaining valued employment. The programme provides introductory employment skills for the industry, is practical and pitched at the right level. The entry processes are robust, ensuring appropriate selection to match suitability of trainees to the industry. Trainee needs and desired outcomes are identified and met through buddying and relevant work placements. One-to-one learning support is offered, including literacy and numeracy, and learning activities are a reasonable mix of theory and practical application. 9

10 The learning environment is inclusive, and teaching activities are appropriate for the skills required in the industry. Scheduling of assessment supports learning, and practical application is supervised by experienced practitioners. Amida Training is regularly engaged with the ITO, and development of the new qualification. However, external moderation is not effectively managed, and similarly, internal moderation practices are limited. Therefore, the extent to which management understands how assessment and teaching practice supports educational outcomes is not clear. Statement of confidence on capability in self-assessment NZQA is Not Yet Confident in the capability in self-assessment of Access Homehealth Limited trading as Amida Training. Amida Training tracks individual achievement of unit standards, and with current low numbers of enrolment, are able to monitor programme completion and trainees pathways into employment. This data is collected and reported to MSD, but it is not apparent that this is used to understand organisational capability such as strengths, challenges, or improvements. Relationships with stakeholders are maintained through formal and informal engagement activities. Destination data is collected and recorded regularly. The extent to which feedback and data is analysed, or informs organisational planning is not clear. However, there is evidence of developments to the programme in response to industry engagement, and in particular, the community needs identified. Trainee feedback is collected intermittently and analysis is not evident. In particular, a mechanism to assess the effectiveness of the tutors or the teaching is not apparent. Recruitment of staff is purposeful to ensure personnel have the adequate skills, knowledge and experience in the industry, however, performance appraisals are intermittent, and a system to regularly understand tutor and teaching effectiveness is absent. However, due to small numbers, the tutor engages and interacts regularly with trainees to understand needs. Learning materials are appropriate and trainees indicated that they supports the learning. Management operates on a high-trust model, which offers autonomy and flexibility to staff in how the programme is run, and key performance indicators are clear. However, lines of accountability are not clear between the parent body, and Access Homehealth Ltd, which limits the support afforded to Amida Training. There is limited monitoring by academic management, of staff, of the programme, and processes to understand educational outcomes, and a coherent system that reviews organisational processes is not evident. However, the organisation is currently reviewing and developing processes in line with the Amida Quality Improvement Plan

11 Findings How well do learners achieve? The rating for performance in relation to this key evaluation question is Good. The rating for capability in self-assessment for this key evaluation question is Adequate. Between March 2015 and May 2016, Amida maintained six intakes of 12 trainees in the Training for Work programme, made up of two Wellington cohorts, and four Nelson cohorts. In November 2015, the Wellington cohort ceased delivery while Nelson continued. The focus of the MSD contract is on employment and is consequently the main driver for the programme. Completion of a qualification is not an MSD performance indicator. Nevertheless, trainees are enrolled in vocationally relevant courses that target employment opportunities in the health and care sector. At the end of the funded enrolment period, Amida continues to track, support and assess all trainees until trainees have been successfully employed or have completed the unit standard courses. Trainees enrolled in the level 3 National Certificate in Health Disability and Aged Support (Core Competencies) are supported to achieve the qualification. Amida is continuing to work with 32 per cent (18) of the 56 trainees enrolled between March and December Amida use the student data management system to closely track individual progress towards the unit standards, and thereby the completion of the programme and is an improvement since the previous EER. Fig 1. Performance rates for Amida The findings in this report are derived using a standard process and are based on a targeted sample of the organisation s activities. 11

12 Completion rates in 2015, includes trainees enrolled in the Training for Work programme, who were awarded the National Certificate in Health, Disability, and Aged Support (Core Competencies) (Level 3) by Amida Training, or completed the unit standards toward the National Certificate in Health, Disability, and Aged Support (Foundation Skills) (Level 2). 3 Performance rates for 2015, shows that approximately 41 per cent of trainees who undertook the Training for Work programme, were awarded the qualification 4, or completed the unit standards; while 32 per cent are continuing to be supported to complete the qualification. For 2016, there are currently 12 people enrolled in Training for Work 5, in the Nelson cohort. Table 1. Performance rates for Amida Number of trainees Enrolment 56 Completion rate 6 23 Withdrawals 15 Continuing to work with 18 The small numbers allowed and enrolled in the programme allows staff to be connected with trainees, and maintain contact. Currently, Amida benchmarks outcomes against the MSD expected targets of 80 per cent in employment, and is tracking positively toward this with a current employment outcome of 77 per cent. MSD has indicated their satisfaction with Amida s employment outcomes. Amida has good knowledge of trainees employment pathways, with the trainees moving into community health, then rest home or disability services. 3 The National Certificate in Health, Disability, and Aged Support (Foundation Skills) (Level 2) is awarded by Careerforce ITO 4 Refers to the completion of the National Certificate in Health, Disability, and Aged Support (Core Competencies) (Level 3) which is not being delivered in Completion and award of National Certificate in Health, Disability, and Aged Support (Core Competencies) (Level 3) and unit standards that lead to the National Certificate in Health, Disability, and Aged Support (Foundation Skills) (Level 2). 12

13 Fig 2. Completion rates for Māori and Pasifika 2015 Similarly, from the four intakes of MSD trainees since March 2015, 17 (30 per cent) trainees continued to further education. Primarily for the Wellington cohort in 2015, the unit standard and qualification completion rate 7, and employment rates were greater for Māori, while more Pasifika progressed to higher study. These results for Amida are comparable to, and tracking positively against other training organisations delivering similar schemes locally. The Manager and staff have an intimate understanding of trainees progress, and data is collected, primarily for reporting purposes to the Ministry of Social Development. Tutors have an understanding of the independent and unique characteristics and needs of the Wellington and Nelson cohorts, but this is not effectively analysed or reported to inform (future) direction, delivery or progress of the programme. However, evaluators heard and sighted regular weekly and monthly reporting of performance data through the layers of Amida, and to MSD as the funder. 7 For the National Certificate in Health, Disability, and Aged Support (Core Competencies) (Level 3) that is no longer being delivered; and unit standards that lead to the National Certificate in Health, Disability, and Aged Support (Foundation Skills) (Level 2). 13

14 1.2 What is the value of the outcomes for key stakeholders, including learners? The rating for performance in relation to this key evaluation question is Good. The rating for capability in self-assessment for this key evaluation question is Good. Some trainees had limited previous education success, and while the programme offers a credential, it also offers the work placement with the potential for ongoing employment. For some trainees, gaining a qualification was a major accomplishment. Both trainees and graduates who had experienced long term unemployment expressed the satisfaction of moving into employment that is valued, productive, remunerative, and potentially ongoing. Similarly, employers, spoke about gaining a pool of qualified staff to meet the community health needs. Tutors and employers indicated that trainees gain technical and practical skills that are relevant to the sector, as well as soft skills required for communication and professionalism with clients. This includes a strong sense of work ethic demonstrated in timekeeping, and confidence. Some graduates of the programme interviewed by the evaluators indicated that they progressed to further study at higher level, or in areas such as registered nursing. Evaluators heard of trainees gaining other skills that would be transferable to allied and clinical health. There was a strong indication that the future pathways such as employment and study, which is enabled by the programme, has a strong value for current trainees and graduates. Key stakeholders acknowledge the contribution that the programme and provider makes to the community, by adding to a workforce to service the local aging community and their needs, thereby developing a stronger labour market, and meeting the high demand for qualified employees. Feedback is gathered intermittently through s, phone conversations or at networking events. The skills gained in the programme grows the health sector workforce for homecare and rest home care. The sector gains qualified personnel with specific introductory knowledge in dementia, medication administration and hoisting. Additionally, the positive reputation of the tutor, his extensive network base and effective tailored practice, and employment outcomes in the target population, has satisfied the requirements and expectations of MSD in the three year contract. The tutor works closely and carefully with each trainee as well as the community, so has an inherent understanding of the trends and needs and how to best match a trainee with an employer. However, this is not recorded, and there is no meaningful analysis to assist in planning. Similarly, the tutor also tracks destination data iteratively, on a day by day basis. There is some evidence that shows some use of destination data that has led to an emphasis on medication-related training. Overall, there are a range of valued outcomes to trainees, the community, employers, stakeholders and the industry. However there is no systematic collation 14

15 and analysis of trainee and stakeholder feedback about the value to them of the programme that would inform improvements 1.3 How well do programmes and activities match the needs of learners and other stakeholders? The rating for performance in relation to this key evaluation question is Good. The rating for capability in self-assessment for this key evaluation question is Good. Amida has a strong social conscience, and sense of social value gained as a result of trainees undertaking the programmes. The tutor has a good understanding of the employer expectations of staff, the needs of clients of services, and the knowledge required of trainees. This is appropriately applied and demonstrated by the staggered learning approach, to cater for the learning needs and timetabling of trainees, in line with employer needs. Introduction to the work environment is phased so that trainees have the appropriate theoretical and practical skills and knowledge suited to the placement and expertise expected at different intervals in the placement continuum. Similarly, the level 2 vocational programme is practical and pitched at the right level for the trainees. This provides introductory employment skills for the industry. Additions were recently made to the programme, such as manual handling, hoisting and medication literacy, to meet the contractual obligations of MSD, by growing trainee employability, and preparing them for employment. This also responds to community need to ensure that trainees are ready for work and prepared with the skills required to manage the increased number of people with high level needs. It would be useful to Amida to check with stakeholders that the intended benefits of the changes had been achieved. Trainees are purposefully matched to placement locations, based on the area of home care that a person wants to work in, and their ability to work in that area. Assessment matches the phasing into the work placement, and the pending environment and knowledge that the trainees need to know. The tutor reflects on what trainees know and need to know to be successful in work placements, and assessment is planned accordingly to fit within these. Teaching assessment materials used, are developed by Careerforce ITO. These are appropriate to the qualification, the trainee and employer needs. Amida is also engaged with Careerforce ITO around changes, moving from unit standard based to more integrated assessment. A selection process exists that vigilantly screens potential trainees suitability and motivation, through a face to face interview. At the core, the entry process ensures that people are physically, mentally and socially suitable to work with clients of home and aged care. Trainees undertake an appropriate literacy and numeracy test, and where required, ongoing one on one support is provided for trainees with literacy challenges, including verbal assessment. Amida is alert to the language 15

16 needs of all trainees, and pathways trainees into ESOL training when required. Classroom delivery include a mix of practical and theoretical activities, to make the content accessible to all trainees, which has gathered positive feedback in internal evaluations. Resources such as videos, and textbooks are useful and appropriate, accessible, and support learning. Similarly, the learning environment is inclusive, and also allows trainees to work in a simulated setting. Overall, while the programme is part of a contracted arrangement, and trainees are directed to attend, programmes and activities match the needs of trainees, employers, and stakeholders because of the understanding of the sector and community, responsiveness to change, purposeful matching into work placements, and screening processes into the health care area. However, while feedback is informal, and individual, there is limited evidence of regular feedback from trainees which is analysed and used to identify trends to inform improvements. 1.4 How effective is the teaching? The rating for performance in relation to Good. The rating for capability in self-assessment for this key evaluation question is Adequate. In recent years, Amida Training has employed highly skilled, qualified and experienced tutors. The existing sole tutor also has a positive local reputation. However, there is no performance appraisal process to manage ongoing effectiveness. Changes made to the overall Training for Work programme delivery by Amida Training, responded to the need for effective teaching practices, and the ability to meet contractual expectations. The sole tutor maintains professional registration and currency with the regulatory body EDANZ, and is offered guidance and support by management when required. This credible reputation has resulted in a number of strong local relationships with external stakeholders such as employers, and pathway organisations. Satisfaction is informally expressed by external stakeholders, and demonstrated in achieved outcomes such as vocational pathways. This anecdotal feedback has resulted in significant trust and confidence in the tutor and his expertise, by the management. Practical competency-based assessment is supervised by the tutor in the workplace and controlled settings such as the simulated environment. Trainees indicated that this led them to feel supported through the learning process and working environment. An inclusive environment is created that engages previously disengaged trainees from education, and there are opportunities to demonstrate knowledge and skills in a variety of contexts. Evaluators heard from trainees who said the learning environment encouraged and motivated them to learn and engage, because it was interesting and not judgemental. Assessment is timely and supports the learning with practical and theoretical supervision by experienced workplace practitioners, such as caregivers and nurses 16

17 from the sector. Trainees indicated that this staggered approach of learning, assessment and practical application has assisted learning and consolidation of knowledge and skills. There is a lack of robust internal and external moderation to inform management of assessment practice, which is also limited by the fact that there is only one tutor. However, Amida Training has worked extensively with Careerforce around modifications to assessments in the development of the New Zealand Certificate in Health and Wellbeing (Level 2) qualification, and in particular, the shift toward integrated assessment. They participate in moderation activities such as the cluster meetings, and maintain regular contact with Careerforce. Overall, teaching effectiveness relies on the expertise and experience of one tutor. Embedded processes to ensure, guide and support continuity of teaching practice and valuable tutor-stakeholder relationships are not evident. There is limited evidence of the collection, collation and analysis of information to inform the effectiveness of practice, and the value for trainees, stakeholders and the organisation. The self-assessment of effectiveness of teaching is limited by there being only one tutor, and no performance appraisal process. 1.5 How well are learners guided and supported? The rating for performance in relation to this key evaluation question is Good. The rating for capability in self-assessment for this key evaluation question is Adequate. The programme and support for trainees is tailored to the desired outcomes of individuals. Trainees are from a range of social, and cultural backgrounds, and the tutor models a respectful teaching and learning environment. For example, an inclusive learning environment is created by the tutor, which supports refugee migrants, and people with disabilities. Also, stakeholder engagement includes the disability sector who are represented by services that specifically assist employment pathways that suit individual needs and capabilities. The inclusive learning environment includes one-to-one attention when required, and additional lessons to help trainees understand and demonstrate required knowledge and skills. Learning support for literacy and numeracy is given on an as needs basis, identifying that Amida Training is responsive to individual needs, and the recently updated trainee handbook is a relevant and useful tool to assist and support trainees learning, as indicated in trainee evaluations. There is some evidence to illustrate a focus on Māori and Pasifika achievement, as seen in the MSD contract with the Wellington cohort. In this time, employment of Māori was at 80 per cent, above the overall employment rate; while Pasifika was at 50 per cent. Continued support is offered to trainees beyond the expected funding parameters of MSD and this is demonstrated by supporting trainees into employment in other 17

18 areas outside of health. If home health care did not suit an individual, the extensive community networks were engaged, and on two occasions assisted trainees to interviews. Feedback from stakeholders and employers is collected informally and intermittently through s and during networking events. Stakeholders, such as the organisation who refers clients with disability needs, indicated in discussions with evaluators that they are positive about the quality and value of the pastoral care that supports their own needs, community needs, and those of trainees. Weekly reports are provided to MSD, indicating the result of the referral outcome, end of training information (if relevant), and specific notes pertaining to individual s needs. Similarly, a weekly report also updates the manager on enrolment numbers and progress of the trainees and the programme. Overall, trainees have good guidance and support based on the inclusive learning environment, appropriate resources, continued support and purposeful stakeholder networks. However, there are limitations in the review, use and analysis of feedback to ensure that support systems match trainee needs, and desired outcomes. 1.6 How effective are governance and management in supporting educational achievement? The rating for performance in relation to this key evaluation question is Adequate. The rating for capability in self-assessment for this key evaluation question is Adequate. Amida training is a small three-person company, managed by Access Homehealth which provides operational guidance to the sole tutor ensuring that contractual obligations are met. This includes peer reviewing reports developed for MSD. A high trust model exists that operates between management and the tutor, and similarly between the parent body and Amida. As geographical locality separates each party from the other, some meetings are scheduled for updates of activities. At a governance and management level, performance data is not used in a purposeful way to inform improvements or planning toward achieving educational outcomes. Focus by governance on compliance expectations, implications and knowledge for regulatory bodies is not clear, such as monitoring of the currency and relevance of the quality management system (QMS) and student handbook, site approvals and the mismatch of programme delivery to the programme accreditation. However, the organisation responds promptly to NZQA regulatory requirements to ensure compliance expectations are maintained. Communication is open and regular between management and the tutor. The quality of delivery, and sustained high value is reliant on the sole tutor and his extensive networks. Similarly, practice and delivery is also reliant on the tutor s own knowledge, expertise, experience and skill. However, as noted, there is no 18

19 performance appraisal process, or a system to understand tutor and teacher effectiveness. There are no apparent processes to ensure continuity of valued practice documented and embedded in the QMS for Amida Training. This reliance presents as a potential organisational risk and there appear to be no mitigations in place, that is, processes to systematically document effective practice and delivery are absent. A system exists that tracks and reports achievement data and other key relevant information to management and governance. This includes the SMS system as a repository for data gathered, and is used in reporting to MSD, against contractual expectations. It is not clear how this information is analysed or used to inform improvements of the organisation and achievement for trainees. However, on the whole, the organisation responds effectively and timely to change, and engages with relevant key players in the sector such as the industry training organisation, and employers. Amida Training has considered and made some developments in response to recommendations 8 from the previous EER in The organisation is currently reviewing and developing processes in line with the Amida Quality Improvement Action Plan Progress against the Action Plan has demonstrated some development of strategies to increase qualification completions, and collaboration with stakeholders. Further initiatives are planned toward meeting actions and objectives of the action plan. The long term sustainability of the organisation is under consideration, and in particular, Amida s alignment to the overall strategic direction, vision and purpose of Access and Green Cross Health is unclear. Enhancing training delivery, and achieving training goals and outcomes, is included in the overall strategic focus for Access Community Health, however, attention to this is not clearly demonstrated. However, the evaluators found that the human resource allocated to Amida, the investment and processes are fit for purpose, due to the small enrolment numbers, and the second chance learner nature, of the Training for Work programme. There is currently no staff appraisal process implemented On a whole, there are limitations to the support of educational outcomes, which governance and management provide. 88 Amida Quality Improvement Plan (submitted to NZQA August 2015) 19

20 Focus Areas This section reports significant findings in each focus area, not already covered in Part Focus area: Governance, management and strategy The rating in this focus area for educational performance is Adequate. The rating for capability in self-assessment for this focus area is Adequate. 2.2 Focus area: National Certificate in Health, Disability and Aged Support The rating in this focus area for educational performance is Good. The rating for capability in self-assessment for this focus area is Good. 20

21 Recommendations NZQA recommends that Amida Training: Implement systems to effectively manage external and internal moderation practices, to inform strengths, challenges and improvements to assessment practices. Integrate monitoring of compliance expectations into regular communication processes for governance and management. Implement regular or systematic reviews of programmes. Develop a system to collect, collate and analyse trainee and stakeholder feedback, that will measure the effectiveness of the programme, the quality and the value of the tutor, and teaching. Develop a performance management system that supports staff development. 21

22 Appendix Regulatory basis for external evaluation and review External evaluation and review is conducted according to the External Evaluation and Review (EER) Rules 2013, which are made by NZQA under section 253 of the Education Act 1989 and approved by the NZQA Board and the Minister for Tertiary Education, Skills and Employment. Self-assessment and participation in external evaluation and review are requirements for maintaining accreditation to provide an approved programme for all TEOs other than universities. The requirements are set through the NZQF Programme Approval and Accreditation Rules 2013, which are also made by NZQA under section 253 of the Education Act 1989 and approved by the NZQA Board and the Minister for Tertiary Education, Skills and Employment. In addition, the Private Training Establishment Registration Rules 2013 require registered private training establishments to undertake self-assessment and participate in external evaluation and review, in accordance with the External Evaluation and Review Rules (EER) 2013, as a condition of maintaining registration. The Private Training Establishment Registration Rules 2013 are also made by NZQA under section 253 of the Education Act 1989 and approved by the NZQA Board and the Minister for Tertiary Education, Skills and Employment. NZQA is responsible for ensuring non-university TEOs continue to comply with the rules after the initial granting of approval and accreditation of programmes and/or registration. The New Zealand Vice-Chancellors Committee (NZVCC) has statutory responsibility for compliance by universities. This report reflects the findings and conclusions of the external evaluation and review process, conducted according to the External Evaluation and Review (EER) Rules The report identifies strengths and areas for improvement in terms of the organisation s educational performance and capability in self-assessment. External evaluation and review reports are one contributing piece of information in determining future funding decisions where the organisation is a funded TEO subject to an investment plan agreed with the Tertiary Education Commission. External evaluation and review reports are public information and are available from the NZQA website ( The External Evaluation and Review (EER) Rules 2013 are available at while information about the conduct and methodology for external evaluation and review can be found at NZQA Ph E qaadmin@nzqa.govt.nz 22