Report of External Evaluation and Review

Transcription

1 Report of External Evaluation and Review Social Services Industry Training Organisation Te Kaiawhina Ahumahi Inc Confident in ITO performance Confident in capability in self-assessment Date of report: 5 October 2010

2 Contents Purpose of this Report... 3 Introduction ITO in context Scope of external evaluation and review Conduct of external evaluation and review...5 Summary of Results... 6 Findings... 9 Recommendations Further actions MoE Number: 8153 NZQA Reference: C02336 Date of EER visit: 20-23, 29 July

3 Purpose of this Report The purpose of this external evaluation and review report is to provide a public statement about the industry training organisation s (ITO) performance and capability in selfassessment. It forms part of the accountability process required by Government to inform investors, the public, trainees, communities, employers, and other interested parties. It is also intended to be used by the ITO itself for quality improvement purposes. Introduction 1. ITO in context Location: Type: Wellington en. Industry Training Organisation First recognised: 2005 Number of trainees: Domestic: 900 (including 700 standard trainee measures known as STMs) International: Nil Number of staff: 20 full-time equivalents Scope of active accreditation: National Certificate in Employment Support (Level 4 National Diploma in Employment Support (Level 6) National Certificate in Iwi/Māori Social Services (Level 4) National Diploma in Iwi/Māori Social Services (Level 6) National Certificate in Social Services (Level 4) National Certificate in Social Service Work with Abuse, Neglect and Violence (Level 4) National Diploma in Social Service Work with Abuse, Neglect and Violence (Level 6) National Diploma in Social Work (Level 6) National Certificate in Tamariki Ora-Well Child Services with strands in Mainstream, Māori and 3

4 Pacific (Level 4) National Certificate in Whānau/Family and Foster Care (Level 4) National Diploma in Whānau/Family and Foster Care (Level 6) National Certificate in Youth Work (Youth Leadership) (Level 3) National Certificate in Youth Work (Level 4) National Diploma in Youth Work (Level 6) National Diploma in Community Work (Level 6) National Diploma in Counselling (Level 6) Sites: Distinctive characteristics: Recent significant changes: Previous quality assurance history: No additional sites Social Services Industry Training Organisation Te Kaiawhina Ahumahi Inc (SSITO) has undergone major changes in governance, management, and strategy since At the time of the EER it was focused on developing industry-led national degrees in social work and youth work. In 2008, the social services industry completed a review of the governance of the industry training organisation (ITO) by voting in a new constitution focused on outcomes for all learners. The new board structure will be implemented in In the interim, a two-person board and chair govern the ITO. In 2009, a second internal restructure since 2007 took place to ensure that the ITO has the people with the knowledge and skills it needs to do its work successfully. At the last external quality assurance visit by NZQA, an audit in 2005, SSITO did not meet five of 13 requirements, three related to Workplace Assessors and two to National External Moderation. A follow-up visit in September 2006 verified that all requirements had subsequently been met. National external moderation results show that SSITO is currently meeting its NZQA external moderation requirements. Other: Since 2007 the SSITO has consulted its stakeholders widely and restored confidence in the ITO by 4

5 addressing past issues, improving delivery of business as usual, and planning for the next decade to Scope of external evaluation and review The scope of the external evaluation and review of Social Services Industry Training Organisation - Te Kaiawhina Ahumahi Inc included the following areas. Governance, management, and strategy. This focus area was chosen to capture the significant changes, both structural and methodological, that have been made over the past three years. Social work training including the National Diploma in Social Service Work (Level 6) This training takes place mainly on the job. This area has the most trainees. Youth work training levels 3-6 including the National Diploma in Youth Work (Level 6) This training takes place mainly off the job. This area has the second most trainees. 3. Conduct of external evaluation and review All external evaluation and reviews are conducted in accordance with NZQA s published policies and procedures. The methodology used is described fully in the document Policy and Guidelines for the Conduct of External Evaluation and Review available at: (Refer to ITO supplement) The external evaluation and review team comprised one lead evaluator, a second lead evaluator, and an external evaluator. The team spent three days and a one-hour meeting on site in Wellington with a one-hour meeting after the visit. The team spoke to the chief executive, collaboration and engagement manager, finance manager, literacy and numeracy project manager, national moderator, two regional workforce development coordinators, assessors, employers, trainees, and the social workers registration board. Social Services Industry Training Organisation - Te Kaiawhina Ahumahi Inc has had an opportunity to comment on the accuracy of this report, and submissions received have been fully considered by NZQA before finalising the report. The NZQA evaluation team knew that SSITO had recently undergone an independent audit commissioned by TEC but it did not know the outcomes at the time of the external evaluation. 5

6 Summary of Results Statement of confidence on ITO performance NZQA is Confident in the performance of Social Services Industry Training Organisation Te Kaiawhina Ahumahi Inc. Key reasons for this include the following. SSITO has a very clear purpose and direction supported by its strategic and business plans which relate to meeting the identified needs of its industry and trainees. SSITO bases what it does on clear principles and values. SSITO has an interim board and a new governance model which are expected to support the ITO in continuing to improve outcomes for all learners and communities. The ITO has made significant progress in gaining the confidence of its sector through engagement and partnership that support the development of the capability of social services organisations. SSITO has restructured its staff so that it has the knowledge, skills, and experience to deliver initiatives which include improving qualification completion rates, engaging employers, developing literacy and numeracy, reviewing the capacity of the assessor pool, meeting the needs of all sector workers from all cultures including Māori and Pasifika, and developing the workforce. Additional new appointments were pending at the time of the external evaluation. SSITO is establishing a professional development framework focused on ensuring that national standards and qualifications are relevant to industry, are widely used by social services staff and employers, and provide educational and employment pathways, from support worker roles to advanced specialist practice and management roles. Based on widespread consultation, SSITO has identified the need to meet the growing professionalism of its industry. It is developing industry-led workplace-based national degrees in social work and youth training work and has partnered with an institute of technology (ITP) to help in this. SSITO has developed the unit standards for the degrees and submitted them to NZQA for registration. SSITO has improved its qualification completions from 9 per cent in 2007 to 23 per cent as at July This rate exceeds the Tertiary Education Commission (TEC) key performance indicator for SSITO s investment plan. This significant progress is encouraging but comes off a low baseline, and SSITO governance, management, and staff are aware that there is a need to further increase the number and proportion of completions. SSITO has developed strategies and begun implementing practices which are enhancing the quality and effectiveness of the training. These include working more closely with employers by involving them with their employees in training, supporting and recognising in-house training, and organising individual trainees and organisations into regional groups where they learn together. The recent development of more robust identification of industry, employer, and trainee needs prior to signing on any learner helps ensure that those needs are met. Employers, assessors, and trainees spoken to by the evaluation team confirmed that SSITO understands the current and future needs of its sector. Its work in this regard includes the 6

7 development of a 2020 vision for the social services workforce in consultation with the sector. New assessors and the national moderator independently confirmed that assessment and moderation support are provided to new assessors who are registered when they are sufficiently experienced to take part in the collegial peer moderation process which is both paper-based and face to face. Assessors reported that the literacy and numeracy (LN) strategies, including resources and assessments with literacy and numeracy embedded in them, effectively address LN issues and improve skills. Statement of confidence on capability in self-assessment NZQA is Confident in the capability in self-assessment of Social Services Industry Training Organisation Te Kaiawhina Ahumahi Inc. Key reasons for this include the following. SSITO has consulted widely and this has contributed to its self-assessment and enabled it to identify issues, develop strategies and practices which address them, and make improvements which have helped regain the support of the sector. The chair of the SSITO board, the chief executive, and management have been able to articulate the ITO s leadership role. SSITO has recruited staff with the appropriate knowledge, skills, and experience to carry out its statutory responsibilities and deliver a quality service consistent with its leadership role. SSITO has recognised the complex processes associated with identifying and meeting the training needs of a diverse and strategically important industry and aligning them with the needs of the different stakeholders including the government s Tertiary Education Strategy with its emphasis on the achievement of national qualifications. SSITO has simplified its processes and systems to more robustly identify the needs of all its key stakeholders, including trainees. It has moved from traditional workplace learning based on individual trainees, to a relationship which meets the employer s needs and gains the support of the employer, uses learner groups effectively, and is supervised by workforce development coordinators. The increase in completions is considered to be the result of the improved relationship between the employer, employee and the ITO and the support provided by regional workforce development coordinators and assessors; it validates the ITO s policy to focus on supporting whole enterprises rather than individual trainees in isolation from their work environment. Examples of the increased numbers of completions and improved quality and effectiveness of the training are expected to become increasingly evident. SSITO s assessment and moderation practices have been researched and improved so that they are effective in ensuring assessments for SSITO standards and qualifications are fit for purpose, valid, reliable, fair, transparent, consistent, and meet the national standard. The national assessment and moderation system also supports SSITO s in-house recognition of qualifications in conjunction with employers. SSITO is establishing a professional development framework for the sector which is based on opportunities for trainees to progress from qualifications at one level to a higher level. It has integrated pathways and maintained or postponed review of existing standards and qualifications while prioritising 7

8 such development and in order to meet a need for industry-led national degrees as perceived by the sector and the ITO. SSITO is not only focused on meeting current needs. It has consulted with the social services sector to develop a workforce development strategy for the next decade. SSITO has drafted a strategy, Skills for Wellbeing, Realising that a coordinated approach requires ongoing broad sector support, SSITO had distributed the draft for wide sector consultation at the time of the external evaluation. The recent completion and handover of a project that has comprehensively identified all SSITO s stakeholders in a very diverse sector should enable the ITO to further identify, understand and meet the needs of its sector on an ongoing basis. SSITO reports the total number of qualification completions related to the number of trainees and the number of credits achieved, which have improved from 2007 to 2010 and have met or exceeded TEC s requirements for SSITO. SSITO and the evaluation team reviewed quantitative and qualitative information, which showed the basis on which SSITO is making successful improvements. While SSITO records and reports total qualification and credit completions, it also breaks down the information so that it knows, for example, how the completions and length of time taken for completions in social work compare with youth work. This has enabled it to identify that the shorter, lower-level youth work qualifications are completed close to the expected duration, whereas the social worker, higher-level qualifications are not. SSITO has considered the reason for this difference and is making further specific as well as generic improvements referred to above. Self-assessment has led SSITO to identify the need for better recording and monitoring of trainee data and better analysis. The upgrading of SSITO s database and reporting system will enable it to capture and record information more effectively. Both formal and informal anecdotal feedback is important, valuable, and valid. The new database will capture quantitative and qualitative information from employers and trainees on the value of the outcomes of SSITO training, how well trainees are achieving, and how effective the training is. This will enable SSITO to make further ongoing improvements. ITO response Social Services Industry Training Organisation Te Kaiawhina Ahumahi has confirmed the factual accuracy of this report. 8

9 Findings How well does the ITO understand and meet the needs of industry, learners and government? The rating for performance in relation to this key evaluation question is Good. The rating for capability in self-assessment for this key evaluation question is Good. SSITO understands its sector very well. Since 2007 it has consulted the sector widely and regained and increased its support. As part of the process, in 2008 it carried out a survey, which attracted a limited number of responses but some were very detailed and drew together the conclusions of a number of peak bodies. SSITO has improved its working relationships and moved from relative isolation to collaboration on the basis that partnerships and relationships work best. SSITO has forged sector partnerships which include WelTec, the New Zealand Family and Foster Care Federation (NZFFCF), Association for Supported Employment in New Zealand (ASENZ), National Youth Workers Network Aotearoa (NYWNA), Te Kahui Atawhai O Te Motu National Collective of Iwi Māori Social Services; and relationships with Aotearoa New Zealand Association of Social Workers (ANZASW), Social Workers Registration Board (SWRB), and Aotearoa New Zealand Association of Social Work Educators (ANZASWE). While SSITO has increased its industry support, it is still developing its knowledge and understanding of the sector and is not working with all the sector yet. This is an intentional strategy related to its capacity given its still relatively low number of trainees, standard trainee measures (STMs), and completions, and in order to ensure its investment in relationships with stakeholders is carefully targeted and worthwhile. A recently completed project which has robustly identified all SSITO s stakeholders will help facilitate SSITO s increasing involvement with the whole sector it represents. From its consultation, SSITO has identified current and future industry needs which it has begun to address. It is establishing a professional development framework which the SSITO and the whole sector can use to progress from completion of one qualification to another at a higher level, and contribute to further developments. SSITO is also focused on identifying and meeting future needs. It has consulted with the social services sector, peak bodies, non-governmental organisations (NGOs), and government agencies to help determine a workforce development strategy for the next decade. From this consultation SSITO has drafted a strategy, Skills for Wellbeing, Consistent with its realisation that a coordinated approach requires broad sector support, the draft had been sent out for wide sector consultation at the time of the evaluation visit. SSITO understands the value of staircasing qualifications and is motivated by wanting to provide an incentive to those who have currently reached diploma level so that they can go 1 The findings in this report are derived using a standard process and are based on a targeted sample of the organisation s activities. 9

10 on and achieve an industry-led workplace-based national degree. It sees this as an important need to be met if the growing professionalism of the sector is to reach its potential and benefit individuals, family and whānau, and the communities of New Zealand. There are currently 13,500 social workers in New Zealand and 1500 of them have a degree. SSITO is well placed to develop unit standards and qualifications from levels 3-6. However, it has realised that it needs to have broad support and to work with higher-level tertiary organisations if it is to succeed in applying for the registration of an industry-led national degree in social work and another in youth work. It has formed a partnership with an institute of technology (ITP) and is consulting the Social Workers Registration Board and universities. The ITP and SSITO have developed and submitted high-level unit standards for social work and youth work which will be the components of the national degrees. The unit standards and degrees are subject to approval by NZQA. Their acceptance for registration purposes are subject to approval by the SWRB. The evaluation team is satisfied that SSITO is aware of the risks related to focusing on the degree development when a growing number of trainees are starting at the lower levels, especially in youth work. SSITO intends to build its progression from one qualification to another from the top down rather than from the bottom up in its bid to provide a pathway to a national degree that has fewer steps or barriers for trainees. SSITO has enhanced the delivery of the lower-level qualifications through its regional workforce development coordinators working more closely with employers and organising individual learners into groups. There are already examples of how these kinds of practices are improving completions in youth work and the quality and effectiveness of the training. In addition, youth worker training is acknowledging and developing the knowledge, skills, and experience of workers who previously received limited training and qualifications. SSITO intends that youth workers will have the same kind of learning pathways and qualifications as social services workers progressing to a national degree. SSITO reports the total number of completions related to the number of trainees and the number of credits achieved, which meet TEC s requirements for SSITO and shows improvement from 2007 to SSITO and the external evaluation team have gathered rich stories and information about how SSITO is making successful improvements. For example, the ITO identified in contracted training a mismatch between the training and assessment, stopped the programme, ensured both were in alignment, and then resumed. This led the ITO to plan a meeting of all those involved in similar training, especially trainers and assessors, to prevent any recurrence of such a situation. Both formal and informal and anecdotal feedback and data are important, valuable, and valid. SSITO records and reports total qualification and credit completions but also breaks down the information so that it knows, for example, how the completions and duration for completions in social work compares with youth work. This has enabled it to identify that the shorter, lower-level youth qualifications are completed within a timeframe very close to what is nominally expected, whereas the social services higher-level qualifications are not. SSITO has considered the reasons why and is making further specific improvements as well as the generic improvements referred to above. 10

11 SSITO has information about individual trainees current progress, but it is not able to readily access such information over all, other than manually, beyond the immediate programme or qualification the trainee is undertaking, which limits its ability to track and monitor individual trainee progress easily. It has identified that upgrading its database will enable it to capture, record, and process information more effectively, including formal evaluation data from employers and trainees on the value of the SSITO training to employers and trainees, how well trainees are achieving, and how effective their training is. This will enable SSITO to make further ongoing improvements and evaluate its own current performance more easily and further contribute to how well it understands and is meeting the current as well as future needs of its industry, trainees, and government. 1.2 What is the value of the outcomes for employers and their trainees? The rating for performance in relation to this key evaluation question is Good. The rating for capability in self-assessment for this key evaluation question is Adequate. The value of the outcomes for employers and their trainees is that the trainees develop knowledge, understanding, and skills which are directly relevant to their employment. The training not only benefits the individual trainees but also benefits their employers, the quality of the services they deliver, and their clients. SSITO has identified that its sector and responsibilities continue to include a significant number of Māori trainees. It has also identified a need to better serve Pasifika employees, who may be gaining unit standards or qualifications for the first time. Their training and achievements benefit not only their employment but also their community and culture, encouraging others to pursue education and training. TEC-funded embedded literacy and numeracy initiatives include resources, strategies, and support that benefit the individual trainees ability to learn and achieve assessment success, as well as the organisation the trainee works for. The setting up of learning groups by SSITO workforce development coordinators in regions such as South Auckland and Taranaki appears to be improving the achievement of outcomes significantly. Organisations and trainees learning in groups of five to ten in a small region helps to provide a consistent ethical voice and theory underpinning the benefits of their employment and their new learning. A workforce coordinator shared with the evaluation team how one trainee moved from hostility to the training, to seeing and understanding it, to actively developing study groups. Available data shows that social workers are further along their learning pathway and qualifications staircase than youth workers. This is partly historical, and youth workers are now gaining recognition of their existing knowledge, skills, and experience, most commonly at levels 3 and 4. Both sector groups have the opportunity to progress their learning and qualifications up to level 6 and in the near future towards the possibility of industry-led national degrees referred to in section 1.1 of this report. The provision of a pathway and staircase is important to the individual trainees and the industry because it not 11

12 only improves individual and organisational performance but also supports a career structure which can encourage and motivate people in their work. The information that SSITO has about the value of specific training is currently positive, according to feedback from employers, assessors, and trainees. The evaluation team confirmed this through interviews. SSITO s last comprehensive survey to capture such information was carried out in The robust identification of stakeholders by a contractor (whose work has just completed) and the development of a stakeholder database should help SSITO capture both generic and specific, informal and formal feedback, as part of the upgrading of the database system scheduled for the end of This should enable SSITO to better capture quality information, process and analyse it, reflect on what it means, and make any changes it identifies to improve the value of the outcomes. Currently SSITO has no destinational information about where training graduates go within their own organisation or to other organisations but it intends to track and monitor such information in the future. 1.3 How well do trainees achieve? The rating for performance in relation to this key evaluation question is Good. The rating for capability in self-assessment for this key evaluation question is Good. SSITO has improved its qualification completions from 9 per cent in 2007 to 23 per cent as at July This exceeds the Tertiary Education Commission (TEC) key performance indicator for SSITO s investment plan. This significant progress is encouraging and lifts SSITO s ranking compared with other ITOs but comes off a low baseline, and SSITO governance, management, and staff are aware that there is need to further increase the number of completions. SSITO has developed strategies, recruited appropriate staff such as workforce development coordinators, and begun implementing practices that are already achieving improved completions and enhancement of the effectiveness of the training. These practices include working more closely with employers, supporting and recognising in-house training, and organising individual learners into groups, as referred to earlier in this report. SSITO reports internally monthly and quarterly, at the same high level that it reports to TEC in terms of qualification and credit completions. SSITO quarterly meetings with contracted training providers focus on active and inactive trainees and completions. How long trainees take to complete qualifications is reported as required by TEC. However, SSITO has broken down the information so that it knows, for example, how the completions and the timeframe for completions in social work compare with youth work training. This has enabled the ITO to identify the different completion timeframes for the shorter, lower-level youth qualifications and the social services higher-level qualifications, as referred to above. SSITO is making specific improvements as well as generic ones referred to above. Workforce development coordinators and assessors out in the field monitor progress, while the collaboration and engagement manager has regular meetings with contracted providers of arranged training. Feedback from SSITO and employers and assessors interviewed by 12

13 the evaluation team included impressive examples of high trainee retentions, high qualification completions, and high credit completions, for example 33 out of 36 foster care trainees graduating. Such individual cases show that there are clearly pockets of high achievement in the SSITO training. The upgrading of SSITO s database will enable it to more readily and effectively capture and record information which it can use to determine actual trainee achievement, including the progress or value-added component, as business as usual. This will include information from large agencies, employers, and contracted providers and reflect the value of SSITO training outcomes to employers and trainees, how well trainees are achieving, and how effective their training is, as referred to in section 1.1. This will in turn enable SSITO to make further ongoing improvements. 1.4 How effective is the training arranged by the ITO? The rating for performance in relation to this key evaluation question is Good. The rating for capability in self-assessment for this key evaluation question is Good. SSITO has rolled out a new initiative, a training model for mainly on-job training in social services work which is based on involvement of the employer from the start. This helps ensure the relevance of the training of the social services worker to the particular service provider organisation they work for and helps gain the ongoing support of the employer. The needs of the employer are matched with those of the trainee, from the initial needs assessment to starting the programme. This process may take some time, a matter of a few weeks or even months, to ensure the suitability of the training for the employing organisation and the individual trainee. The two workforce development coordinators have a very important role in this process and three more such positions are expected to be filled shortly. Once the trainee is accepted for training, their progress is monitored by their employer and SSITO noting their dates of completions, through information provided by the assessor, and addressing any specific or generic issues that arise. The collaboration and engagement manager or workforce development coordinator talks to the employer and/or trainee and/or assessor on a regular basis throughout the process until the trainee completes their unit standards and gains the credits for their qualification. The trainee is then advised of the opportunities available to go on to further study, standards, and qualifications. The employer is consulted on the outcomes achieved and their value to the employing organisation. SSITO intends to monitor and track employees career progress within their own organisation or any other employer they move to referred to in section 1.2 of this report. Workforce development coordinators arrange training on behalf of the ITO and set up learning groups as referred to in more detail in section 1.2 of this report. This initiative is ensuring effective training as well as improved completions through shared learning, mutual encouragement and support, mentoring and motivation. 13

14 SSITO contracts organisations that have proven performance to deliver off-job training mainly but not exclusively for youth workers. The contracted organisation typically subcontracts a training provider to actually deliver the training. SSITO meets regularly with the contracted organisation in order to assure the quality of the arrangements and the training and the achievement of the outcomes. As mentioned in section 1.1, SSITO recently identified a serious mismatch between training and assessment and took action to remedy the defect. This led the ITO to make changes in contractual arrangements including a meeting of all those involved in of the training, especially trainers and assessors, before training starts to prevent any recurrence of such a situation. SSITO s feedback from employers, contracted organisations, assessors, and workforce development coordinators confirms that the training arranged by the ITO is effective. Increased completions also confirm this. SSITO intends to develop its evaluations of trainee satisfaction with the training and its effectiveness and quality, including the arrangements related to it in the near future when it upgrades its data system. This will enable SSITO to capture, process and analyse, reflect, and make any ongoing generic and specific improvements it identifies based on the systematic electronic capture of all data, including oral, which is currently held on a manual system. This will supplement the information from its various sources which also include moderation. 1.5 What is the quality of the assessment being undertaken towards industry standards and qualifications? The rating for performance in relation to this key evaluation question is Good. The rating for capability in self-assessment for this key evaluation question is Good. SSITO reviewed its assessment and moderation system in 2008 and now runs an effective system of moderation for accredited providers and SSITO s own registered assessors based on quality, effectiveness, and good practice. Moderation is based on meetings and postal moderation. A six-monthly moderation meeting is held for accredited providers. The quality of the assessment towards industry standards and qualifications is based on assessors completing assessor training and standards, good assessment practices, ongoing mentoring and support, assessor feedback, relevant workbooks and assessment materials which contain embedded literacy and numeracy, and robust moderation which in turn contributes to ongoing improvement of all of these aspects. SSITO had 60 contracted assessors at the time of the review. Not all assessors are used and the ITO intends to review this situation. Some assessors the evaluation team spoke to said they would appreciate more communication from the ITO about ongoing organisational changes. A new assessor and the national moderator independently confirmed that assessment and moderation support was provided to new assessors who are registered only when they are trained, qualified, sufficiently successfully experienced, and ready to engage in the national assessment and moderation process, which is both face to face and paper-based. To reach this point of approved status, the new assessors are mentored by two experienced assessors, 14

15 one of whom has subject-specific expertise. This helps ensure consistency in the assessment and moderation practices that apply to unit standards and qualifications. The new assessor also confirmed that the literacy and numeracy advice they were given by the literacy and numeracy project manager on strategies worked very successfully the first time they were used. SSITO s feedback and the evaluation team s interviews with employers, assessors, the national moderator, and the collaboration and engagement manager confirmed the quality of assessment for SSITO standards and qualifications. SSITO s assessment and moderation practices have been researched and improved and are effective in ensuring assessments for SSITO standards and qualifications are fit for purpose, valid, reliable, fair, transparent, consistent, and meet the national standard. SSITO has maintained or postponed review of some existing standards and qualifications and prioritised the development and submission of new unit standards for a new industryled work based degree in conjunction with an ITP which will meet a perceived need by the sector and the ITO. The evaluation team is confident that if and when these standards and qualifications are approved, SSITO will apply its current and effective assessment and moderation practices and systems to the new standards and national degrees. In the meantime, robust direct feedback on assessments by trainees in particular would complement the information SSITO receives directly and indirectly from its assessors, those who are responsible for and deliver the training, and the workforce development coordinators. 1.6 How well does governance and management support trainee achievement? The rating for performance in relation to this key evaluation question is Good. The rating for capability in self-assessment for this key evaluation question is Good. Three values and principles guide what SSITO does and how. These are Kōtahitanga (connecting people), Manākitanga (honouring others, honouring ourselves), and Kaitianga (upholding responsibilities). Kōtahitanga is about unity, acting in a unifying way and building and promoting unity; Manākitanga is about sustainability, acting in a way that ensures the sustainability of the organisation and the professions, industries, and communities SSITO works with; and Kaitianga is about responsibility, taking responsibility for actions and acting in a way that promotes responsible actions by the team, the organisation, the industry, and the communities they work with. SSITO has a very clear purpose and direction supported by its strategic plan and business plans which relate to meeting the identified needs of the sector and trainees. First, SSITO has made significant progress in gaining the confidence of its sector through engagement and partnership, a consultative and collaborative approach to support the capability development of social services organisations, the involvement of employers with employees in industry training, and the development of a 2020 vision for the social services workforce. Secondly, it is continuing to establish a professional development framework focused on ensuring that national standards and qualifications are relevant to industry, are widely used 15

16 by social services staff and employers, include educational and employment pathways, and lead from support worker roles through professional roles to advanced specialist practice and management roles. Thirdly, it has begun to increase completion rates through developing completion strategies which include embedded literacy and numeracy. Fourthly, it has made progress in integrating biculturalism into its services to ensure both Māori and non-māori communities are supported. Fifthly, it has ensured the quality of its organisation through basing what it does on its principles and values and developing a forward looking strategic plan for the next ten years related to its 2020 vision. The board is clear about the difference between governance and management. The chair is very supportive of the chief executive and they meet regularly between monthly reports, board meetings, and quarterly reports. The board is focused on increasing completions more quickly than currently and the responsibility and accountability the chief executive, management, and staff have for this. The board endorses and supports the changes SSITO has made and has confidence in the changes. It supports the development of the industryled national degrees and the top-down development of the related qualifications pathways they would offer without compromising the integrity of the lower-level qualifications and their quality and the trainees who gain them. While SSITO has recently successfully identified all its stakeholders, it is very aware that it does not want to overstretch its current capacity. SSITO has restructured its staff twice since 2007 in order to get the staff it needs to deliver its initiatives including improving qualification completion rates, engaging employers, developing literacy and numeracy, fostering workforce development, reviewing the capacity of the assessor pool, and meeting the needs of all sector workers from all cultures including Māori and Pasifika. SSITO has rich data which it uses to meet TEC and other agency reporting requirements. SSITO is committed to developing the way it captures the valuable qualitative information as well as quantitative data it already obtains from interactions with contracted organisations, employers, workforce development coordinators, assessors, moderators, and trainees. SSITO has made a significant investment in upgrading its electronic database by the end of 2010 which will make this information capture much easier. 16

17 Focus Areas This section reports significant findings in each focus area, not already covered in Part Focus area: Governance, management, and strategy The rating in this focus area for ITO performance is Good. The rating for capability in self-assessment for this focus area is Good. 2.2 Focus area: Social work training including the National Diploma in Social Work (Level 6) The rating in this focus area for ITO performance is Good. The rating for capability in self-assessment for this focus area is Good. 2.3 Focus area: Youth work training levels 3-6 including the National Diploma in Youth Work (Level 6) The rating in this focus area for ITO performance is Good. The rating for capability in self-assessment for this focus area is Good. 17

18 Recommendations There are no recommendations arising from the external evaluation and review other than those implied within the report. Further actions The next external evaluation and review will take place in accordance with the Tertiary Education Commission s re-recognition cycle, which is likely to occur within five years of the date of this report. 18

19 Appendix Regulatory basis for external evaluation and review In 2009 NZQA introduced the evaluative approach to quality assurance in the tertiary education sector, consisting of self-assessment and external evaluation and review. This is applied in the quality assurance of course approvals and accreditations under sections 258 and 259 of the Education Act 1989, as well as for PTE registration under Part 18 of that Act. The NZQA Board also published policies and guidelines for the conduct of external evaluation and review of ITOs on 27 May NZQA relies on ITOs to build the selfassessment and external evaluation and review model into their infrastructure. ITOs will be evaluated on their infrastructure and use of the model in the following ways: Advising Ministerial recognition of an ITO under the Industry Training Act, 1992 Accepting arrangements for monitoring and assessing of industry training under section 10 of the Industry Training Act, 1992 Registering standards on the Directory of Assessment Standards Awarding consents for ITOs (or their assessors) to assess against standards on the Directory of Assessment Standards under section 253(1)(c) of the Education Act, 1989 Applying course approval criteria where ITOs are course owners. External evaluation and review is also used by NZQA as a monitoring and evaluation tool, with the outcomes of these processes informing Tertiary Education Commission decisions relating to re-recognition of an ITO. In addition, external evaluation and review reports are one contributing piece of information in determining future funding decisions in relation to an investment plan agreed between an ITO and the Tertiary Education Commission. The understandings and expectations for the implementation of ITO quality assurance are set out in a protocol to the Memorandum of Understanding between NZQA and the Tertiary Education Commission. This report reflects the findings and conclusions of the external evaluation and review process, conducted according to the policies and criteria approved by the NZQA Board. The report identifies strengths and areas for improvement in terms of the ITO s performance and capability in self-assessment. External evaluation and review reports are public information and are available from the NZQA website ( Information relevant to the external evaluation and review process is summarised in the publication Policy and Guidelines for the Conduct of External Evaluation and Review. Two other documents explain how the process is applied to ITOs: EER Policy and Guidelines ITO Supplement and Evaluation Indicators for Industry Training Organisations. These 19

20 documents are available at: NZQA Ph E eeradmin@nzqa.govt.nz 20