AQTF Essential Conditions and Standards for Continuing Registration AUDIT REPORT

Transcription

1 AUDIT REPORT RTO DETAILS RTO Name RTO: Inner Melbourne VET Cluster NTIS Number Address Level 1, 71 Palmerston Crescent, South Melbourne Website Registration Contact Penny Vakakis Phone Number Student Numbers 385 in 2011 AUDIT TEAM Lead Auditor Chloe Dyson Auditor/s Therese Hickey Technical Advisor/s Observer/s REGISTERING BODY DETAILS Contact Person Katherine Papastathopoulos Phone Number AUDIT DETAILS Type of Audit Conditions audited Renewal Standards audited 1.1, 1.2, 1.3, 1.4, 1.5. Audit Date/s 7 8 November, 2011 Other audit notes 1, 3, 4, 6, 7, 8, 9. (Conditions 2 & 5 are not required to be audited, see page 3 below) 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7 A site visit took place on day 2 of the audit to Lynall Hall, Richmond 3.1, 3.2, 3.3, 3.4 ACCOMPANYING REPORTS Yes No VRQA Guidelines Audit Report VRQA Guidelines Re-registration Checklist Version AQTF 1 July 2010 Page 1 of 16

2 AUDIT REPORT FOCUS OF AUDIT NTIS Code QUALIFICATION/UNIT OF COMPETENCE/ACCREDITED COURSE Qualification/Unit of Competence/Accredited Course (as per NTIS) Delivery Site SIT20307 Certificate II in Hospitality (Kitchen Operations) Schools SIT20207 Certificate II in Hospitality Schools INTERVIEWEE/S: Staff name and position; employer name and position; students by program (do not list by name) Colleen Carters RTO Administrator Penny Vakakis - CEO Version AQTF 1 July 2010 Page 2 of 16

3 AUDIT REPORT AUDIT SUMMARY Conditions of Registration Compliant Noncompliant Not audited 1 Governance 1.1 CEO must ensure the RTO complies with relevant conditions, standards and guidelines 1.2 Fit & Proper Persons Tests 1.3 Input of Trainers and Assessors in senior management decision making 2 Interactions with the Registering Body 3 Compliance with Legislation 4 Insurance 5 Financial Management 6 Certification & Issuing of Qualifications & Statements of Attainment 7 Recognition of Qualifications Issued by other RTOs 8 Accuracy and Integrity of Marketing 9 Transition to Training Packages/Expiry of Accredited Courses NA Version AQTF 1 July 2010 Page 3 of 16

4 AUDIT REPORT Summary of non-compliances Conditions of Registration The RTO was non compliant in the Conditions. The RTO was compliant in Conditions 1, 4, 6, 8 and 9. Conditions 2 and 5 were not audited. Condition 3 There was not a system in place to ensure learners are informed of how legislation affects their participation in vocational education and training. Condition 7 The RTO has a policy and procedure titled Recognition of qualifications issued by other RTO s Policy and Procedure. This policy and procedure outlines the steps to be taken, Information is also available in the Student handbook under the heading Credit Transfer. The terms, Credit transfer and National Recognition are used interchangeably and this could create confusion. The policy states that the students should provide copies of their statements of attainment or qualification and these would then be verified with the provider. Suggested that originals be sighted to ensure authenticity. Recommendations Condition 3 The RTO is required to ensure that students are provided with information about legislation that affects their participation in vocational education and training. Condition 7 The RTO is required to develop and implement clear and accurate procedures for the Recognition of qualifications issued by other RTOs. Strengths Opportunities for Improvement Condition 7 The Recognition of qualifications issued by other RTO s Policy and Procedure states that the students should provide copies of their statements of attainment or qualification and these would then be verified with the provider. Suggested that originals be sighted to ensure authenticity. Version AQTF 1 July 2010 Page 4 of 16

5 AUDIT REPORT Standard 1: The RTO provides quality training and assessment across all of its operations Result Audit conclusion The RTO is non compliant with Standard 1. Element 1.1 Data collection - continuous improvement Compliant Non-compliant Not audited The RTO has a continuous improvement policy and procedure in place. This procedure provides some information on continuous improvement for example, internal audits will be carried out, users of IMVC s services have the right to provide feedback but the procedure does not provide sufficient detail to meet the requirements of VRQA Guideline 2.1. It does not state how Quality Indicator data will be collected, analysed, acted upon and reported and how staff satisfaction data will be collected, analysed and acted upon. It does not state how, how often and by whom internal audits will be conducted. The contract for the partnering arrangements does not clearly identify who is responsible for data collection and how this will be implemented. Evidence of continuous improvement The RTO provided a continuous improvement register, which indicated that a great deal of work has been completed in attempting to make the RTO compliant with the AQTF such as developing training and assessment strategies. Continuous improvement relates to improvements made once the RTO is compliant with the AQTF and the VRQA Guidelines. The activity in the register relates to the RTO becoming compliant. Quality Indicator surveys have been distributed for 2011 but not yet returned and analysed. The RTO had received Quality Indicator surveys from a sample of learners for A strong theme in the feedback was that the theory texts were poorly organised and that the majority of respondents did not enjoy the theory. The RTO has recently worked with the producers of the training materials in order to improve them ( s sighted) but although the RTO administrator stated that trainers will be asked to come to a meeting in early 2012 to discuss and try to resolve issues around presentation of theory, no planning has yet taken place to this effect. Given that students identified that the Version AQTF 1 July 2010 Page 5 of 16

6 theory training was poor in 2010 and the RTO has not yet resolved this issue it would appear that the RTO has not effectively responded to issues raised from data collected. Validation of assessment There is not a plan for assessment validation in place. It is the requirement of the VRQA Guidelines that a plan for validation is in place and this is vital for this RTO as it will need to develop a strategy so that staff from partnering schools can participate in validation, wherever possible. The RTO is required to review and strengthen its continuous improvement processes so that responsibilities for data collection and analysis from a range of sources is collected, analysed and acted upon as a basis for improvement of training and assessment. Element 1.2 The strategy is comprehensive but there are slight inaccuracies and some omissions. The strategies both state that student placements are not compulsory but the RTO CEO confirmed that they are mandatory. They are completed during student holiday periods but the role that the placements have in training and assessment is not clearly described in the strategy. Assessment methods for the unit Prepare, cook and serve food for food service were not all included the logbook was omitted. SIT20207 Certificate II in Hospitality The strategy is comprehensive but there are slight inaccuracies and some omissions. The strategies both state that student placements are not compulsory but the RTO CEO confirmed that they are mandatory. They are completed during student holiday periods but the role that the placements have in training and assessment is not clearly described in the strategy. Industry consultation The RTO provided minutes from the Industry Advisory Committee Hospitality, which indicated that the meeting highlighted some important industry concerns to be addressed in training. These minutes were dated 19/10/11 and the lessons learnt from consultation for example, the need to develop knife skills - have not had an impact on the strategies. The RTO is required to ensure that complete and accurate strategies for training and assessment are in place for the two qualifications audited and that there is clear evidenced that they were developed in consultation with industry. Element 1.3 SIT20307 Certificate II in Hospitality (Kitchen Operations) Units reviewed: SITHCCC009A Prepare vegetables, fruit, eggs and farinaceous dishes and SITHCCC027A Prepare, cook and serve food for food service Version AQTF 1 July 2010 Page 6 of 16

7 SIT20207 Certificate II in Hospitality SITCHCCC004B Clean and maintain kitchen premises and SITHFAB003A Serve food and beverages to customers There were a number of schools, which the RTO entered into agreements with when BEST Restaurant Training closed in March Four of these schools did not have sufficient training staff and/or commercial kitchens to accommodate the number of students. The CEO stated that it was unlikely that contacts would be entered into with these schools in 2012 as they did not have sufficient capability to deliver the Hospitality training. Other than the schools limiting the number of students to be accommodated in the kitchens there was not a sufficient response to ensure that students would be provided with suitable training venues. The RTO has identified issues with delivery sites but has not effectively managed these issues. The Operational Commercial kitchen checklist used by the RTO to review the appropriateness of Hospitality facilities in schools does not have a summary of action to be taken if there are insufficient resources in place. This checklist did not state how many students could be accommodated within the training kitchens. This is important because each student needs to have access to 1.5 metres of bench space, two hot plates, half of an oven and so on in order to meet the requirements of the Training Package. The checklist does not determine whether students will be given access to an industry realistic ratio of customers for the units where this is required nor does it confirm that students will have access to a sufficient variety of foods as required by the Training Package, for the relevant units. Facilities - one site visit only Lynall Hall: adequate kitchen facilities were in place although there is not a clear plan in place regarding how students will gain access to customers as required by some of the units of competency and there is not a café or restaurant as such; rather, two large tables in a recreation room. It is planned that the Hospitality program will cater to community groups in 2012 if the program goes ahead a Lynall Hall. Currently there are only four enrolments and this number of students is not sufficient to sustain a program. Refer also Element Assessment. The RTO is required to revise its processes so that it is confirmed before course commencement that Hospitality training facilities and resources are sufficient to meet the requirements of the Hospitality Training Package. Refer also Elements 1.4 Staffing and Assessment. Element 1.4 Trainer Files and the Recruitment of Staff & Contractors Policy and Procedure were reviewed. A number of qualifications were not verified, In some cases Trainer and Assessor competency was not mapped or verified Version AQTF 1 July 2010 Page 7 of 16

8 Professional development was not systematic as there was no plan in place to ensure that trainers are developed in areas relevant to the qualifications they are delivering. The Professional Development policy and procedure did not provide sufficient guidance for the planning and execution of professional development to ensure that the vocational and VET knowledge and skills of trainers is planned and implemented. CV s and professional development were not verified. The trainer matrix is used to verify the vocational competency of the trainers. It does not verify that the trainer and assessor competencies meet the NQC requirements Certificate II in Hospitality (selected units) Daniel Vitali: Daniel holds the Diploma of VET Practice, 21697VIC Awarded by Box Hill Institute, 1 July This has not been mapped to the NQC requirements for trainers and assessors Certificate II in Hospitality (Kitchen Operations) - selected units Michele Dean: matrix and CV does not demonstrate sufficient vocational experience Louise Barrett: No signed matrix in place Denise King: No position description, contract or signed CV in place (employed by the host school) Peter Crawford: Does not hold the TAE Equivalence has not been established and there is not a supervision and co-assessment process in place and evidence of implementation documented The RTO is required to change its processes so that it is ensured that trainers and assessors meet the requirements of the National Quality Council and its successors; and the requirements of the VRQA Guidelines. Element 1.5 SIT20307 Certificate II in Hospitality (Kitchen Operations) Units reviewed: SITHCCC009A Prepare vegetables, fruit, eggs and farinaceous dishes (strategy states assessment is demonstration, questioning, written test). There is a demonstration, a test and a set of questions (worksheet). The unit of competency, Context of and specific resources for assessment description states that assessment must ensure industry-realistic ratios of kitchen staff to customers. However there is no indication in the assessment conditions statement of the unit that this requirement is considered in assessment. The checklist requires students to prepare at least seven dishes but there is insufficient space on the checklist to record the range of performance Version AQTF 1 July 2010 Page 8 of 16

9 over the seven occasions. The result at the end of the checklist requires a judgement of C/NYC to be made but this is not possible at task level. There is also a test. The conditions for assessment are not provided eg open/closed book. The decision-making rules are not provided to the candidate or to the assessor: for example, the number of correct responses required. A marking guide is in place. There is no space provided for a result on the test. There is also a worksheet with model answers but no conditions for assessment or decision-making rules are provided. The assessment summary document does not have a facility to be signed by the candidate. SITHCCC027A Prepare, cook and serve food for food service (strategy states assessment is journal, questioning and third party validation): Strategy is not accurate and assessment also includes a logbook. Mapping has been done from the tasks to some but not all of the performance criteria and to the Employability Skills but not to the critical aspects for assessment and skills and knowledge. All assessment of this unit is related to the placement but according to the VET Administrator could be completed after the placement. Unit requires: collection of direct, indirect and supplementary evidence showing preparation and service of multiple items for a minimum of 12 complete food service periods; use of a range of cookery methods and production of a range of menu items. Portfolio includes responses to a range of questions and many of these questions are well constructed. However the marking guide for the written assessment about the students experience at the workplace relates to VCE requirements rather than to the unit of competency requirements. There is also a log and a third party report. There is no strategy for analysing all of the evidence collected against the requirements of the unit of competency that is, making a judgement from the evidence provided. The logbook does not set the requirements for the number of food service periods to be undertaken by the student or the requirement to use and record a range of cookery methods and food items. SIT20207 Certificate II in Hospitality SITCHCCC004B Clean and maintain kitchen premises (strategy states assessment is demonstration, questioning, written test): Strategy is accurate. Theory test: conditions for assessment are not provided e.g. open/closed book. The decision-making rules are not provided to the candidate or to the assessor: for example, the number of correct responses required. A marking guide is in place. There is no space provided for a result on the test. Observation checklist: the task to be observed is not described. The result at the end of the checklist requires a judgement of C/NYC to be made but this is not possible at task level. There are four worksheets, which have the appearance of formative assessment tasks. There are no conditions for assessment, no decision-making rules provided to the student or to the assessor. Assessment is not valid, reliable or fair given the lack of direction and the absence of an assessment task for the observation. The assessment summary document does not have a facility to be signed by the candidate. SITHFAB003A Serve food and beverages to customers (strategy states assessment is demonstration, questioning, written test). The observation checklist does not describe the task to be completed. There is another checklist, which relates to a food and beverage practical skills test, which also does not describe the task but does provide some performance indicators. The checklist is the performance criteria from the unit, which are not related back to any task. There is also a scenario. Candidates are not provided with information about the form in which they are expected to respond to this activity and there are no model responses or decision-making rules. There is no facility to record the outcome of this assessment. There is also a theory test. The conditions for assessment are not provided eg open/closed book. The decision-making rules are not provided to the candidate or to the assessor: for example, the number of correct responses required. A marking guide is in place. There is no space provided for a result on the test. The assessment summary document does not have a facility to be signed by the candidate. Version AQTF 1 July 2010 Page 9 of 16

10 RPL The RPL procedure states that students are to complete an RPL application form and submit evidence, which the assessor analyses and makes a decision about competency. The RPL application form is designed for both the candidate and the assessor but at times it is not clear who the audience for the information is (refer section 7, which states, Assessor use only but also addresses the candidate). The application form holds questions for each unit of competency but no indication of the required responses. The assessor is guided to note three pieces of evidence have been collected against each performance criteria but not to note what evidence has been considered in order to make an assessment judgement. The approach of assessing against performance criteria is not a productive one as it disaggregates the unit of competency and omits a consideration of the remainder of the unit. The overview of the RPL process includes a practical assessment, which is stated as an option in the candidate information. There are no practical assessment tasks or records in place. The RTO is required to revise assessment for the two qualifications audited to ensure that assessment meets the requirements of the Training Package and VRQA Guideline 5. Strengths Opportunities for Improvement Version AQTF 1 July 2010 Page 10 of 16

11 Standard 2: The RTO adheres to principles of access and equity and maximises outcomes for its clients Result Audit conclusion The RTO is non compliant with Standard 2. Element 2.1 Compliant Non-compliant Not audited Strategies for training and assessment clearly identify the cohorts of students who use the RTO s services. The RTO does not provide support services to meet the needs of clients. Schools take this responsibility but responsibilities are not sufficiently defined and agreed in the partnering contract. The RTO is required to review its systems for identifying and addressing client needs to ensure that responsibilities of all parties are defined and agreed. Element 2.2 Refer Element 1.1. The continuous improvement procedure does not give sufficient detail regarding data collection as the basis for continuous improvement of student services and how this data will be acted on. The contract for the partnering arrangements does not clearly identify who is responsible for data collection and how this will be implemented. The RTO is required to review and strengthen its continuous improvement processes so that responsibilities for data collection and analysis from a range of sources is collected, analysed and acted upon as a basis for improvement of training and assessment. Element 2.3 The RTO was compliant in Element 2.3. Version AQTF 1 July 2010 Page 11 of 16

12 Element 2.4 The RTO was compliant in Element 2.4. Element 2.5 The Student handbook states that trainers will provide individualised support if required and that schools should also be approached to provide support. The RTO did not have a process in place to identify the language, literacy and numeracy (LLN) support needs of students in place in The RTO administrator, Colleen Carters stated that there were some students whose schools had not identified that LLN support was needed and so support requirements were not identified in a timely manner. It is planned to ask all students to complete a LLN assessment in 2012 (assessment sighted) but the partnering contract does not include this responsibility or how schools will be required to provide support when the need is identified. The contract and its attachment/s do not state how the RTO will monitor that sufficient support is being provided. The RTO is required to ensure that the responsibilities of all relevant parties for assessing and addressing student support needs are clearly defined and agreed. Element 2.6 The RTO was compliant in Element 2.6. Element 2.7 The RTO was compliant in Element 2.7. Strengths Opportunities for Improvement Version AQTF 1 July 2010 Page 12 of 16

13 Standard 3: Management systems are responsive to the needs of clients, staff and stakeholders, and the environment in which the RTO operates Audit conclusion The RTO is non compliant with Standard 3. Element 3.1 The RTO has recently re-developed many of its systems. Audit found that some systems are still not sufficient, as follows: Staff development is not adequately planned (refer Element 1.4) Staff records are not maintained according to the VRQA Guideline requirements (refer Element 1.4) Result Compliant Non-compliant Not audited Continuous improvement processes and procedures have not been developed sufficiently to systematically improve the operations of the RTO (refer Elements 1.2, 2.2, 3.2) Management of partnering arrangements is not sufficiently rigorous (refer Element 3.3) The RTO is required to review its systems so that the RTO s management of its operations ensures clients receive the services outlined in their agreement. Element 3.2 The RTO has completed considerable development work since the RTO administrator was employed in April, The RTO is establishing systems, which once fully developed and deployed should be effective. However, the continuous improvement procedure and the partnering agreement are not sufficient to ensure that continuous improvement is applied across all of the RTO s operations, including in partnering schools. The RTO has commenced a monitoring process for schools but has not implemented it across all partnering organisations. Version AQTF 1 July 2010 Page 13 of 16

14 The RTO is required to revise its systems so that they systematically address the requirements of the AQTF and the VRQA Guidelines. Element 3.3 There were a number of schools (seven), which the RTO took up when BEST Restaurant Training closed in March It was identified through a delivery site monitoring visit that three of these schools did not have sufficient training staff and/or commercial kitchens to accommodate the number of students. The RTO has a Facilities and Resources policy and procedure but this procedure does not provide guidance on how and when it is determined that a school has an appropriate training venue and staff. It states that if facilities and equipment are insufficient the trainer creates a simulated work environment but this strategy requires facilities and equipment these cannot be simulated. The CEO stated that the RTO uses a range of strategies to offer students opportunities to gain access to resources at other schools during the holidays but this strategy was not documented for each school where shortfalls existed. The Developing partnership agreements policy and procedure states that an agreement is only entered into after premises have been deemed complaint. It does not discuss whether staffing is checked or at what stage of entering an agreement this is done. The RTO has not implemented this procedure in relation to the three school partnerships that were entered into in March It is noted that that the RTO entered into these agreements with little opportunity to prepare in order to continue students training commenced with BEST Restaurant Training. However, the shortfalls identified at these venues have not been adequately addressed. The Operational Commercial kitchen checklist used by the RTO to review the appropriateness of Hospitality facilities does not have a summary of action to be taken if there are insufficient resources in place. For example, for Princes Hill Secondary College it was identified that the venue did not have sufficient salamanders, a bain marie or a double sink but there was no action plan/strategy to address these shortfalls. This checklist did not state how many students could be accommodated within the training kitchens. This is important because each student needs to have access to 1.5 metres of bench space, two hot plates, half of an oven and so on in order to meet the requirements of the Training Package. The checklist does not determine whether students will be given access to an industry realistic ratio of customers for the units where this is required nor does it confirm that students will have access to a sufficient variety of foods as required by the Training Package, for the relevant units. Delivery site audits of equipment and monitoring visits had been completed for 13 of the 17 agreements. According to the RRTO, there has been a longstanding arrangement in place with the other four schools and evidence of a recent review of facilities and equipment is not in place. Contracts The RTO has identified that agreements with schools were inadequate and that agreements needed to be altered to meet Department of Education requirements. As VET students are now completing their qualifications 2012 agreements were reviewed at audit. Agreements comprise: a contract between the School Council and the RTO (the contractor), where the contractor is providing services to the Council and a VET responsibility matrix, which lists the AQTF elements but not the VRQA Guidelines. This contract is unsuitable for the purposes of Element 3.3 as the school is providing services to the RTO (that is, providing training and assessment, staff and facilities) not the other way around. Version AQTF 1 July 2010 Page 14 of 16

15 VET responsibility matrix: Ticks are placed next to elements under the headings and/or Home School Responsibility and/or Contractor Responsibility and Responsibility but there is no explanation of how the requirements of these elements are to be met. The contract makes no reference to the VET responsibility matrix and there is no plan/agreement for monitoring the terms of the agreement. The VET Responsibility matrix makes reference to the Lead RTO but this term is not explained in the contract or the matrix. The following partnering arrangements are in place: Strathcoma Baptist Grammar Carey Baptist Grammar McRobertson Girls School Swinburne Senior Secondary College Lynall Hall Community School Firbak Grammar School Princes Hill Secpndary College Doncaster Secondary College Thornbury High School Werribee Secondary College The Grange P 12 Sacred Heart College Firbank Grammar School Hume Secondary College Corio Bay Senior Secondary College Keilor Downs Secondary College Version AQTF 1 July 2010 Page 15 of 16

16 Wantirna College Heathmont Secondary College The RTO is required to revise the agreement with partnering organisations so that: Roles and responsibilities for implementing the AQTF and VRQA Guidelines are clearly defined and agreed Adherence to the terms of the agreements is systematically monitored and reviewed Element 3.4 The RTO was compliant in Element 3.4. Strengths Opportunities for Improvement Version AQTF 1 July 2010 Page 16 of 16