CONDUCT MODERATIONS FOR DUMMIES
TWO TYPES OF MODERATION INTERNAL (For the Training Provider) Internal moderation ensures that assessments conducted in a single learning provider, are consistent, accurate and well-designed. The three main stages to internal moderation are: Design Implementation Review Accredited providers should have individuals that manage their internal moderation systems. These internal moderators should: Establish systems to standardize assessment (process), including the plans for internal moderation Monitor consistency of assessment records Through sampling, check the design of assessment materials for appropriateness before they are used, monitor assessment processes, check candidates evidence, check the results and decisions of assessor for consistency Co-ordinate assessor meetings Liaise with external moderators Provide appropriate and necessary support, advice and guidance to (new) assessors EXTERNAL (For the SETA) External moderation is a means of ensuring that two or more providers delivering programmes to the same unit standards and qualifications, are assessing consistently to the same standard, and in a well-designed manner. External moderation systems are managed by the ETQA s. External moderation involves: Checking that the systems required to support the provision of learning programmes across the institution/learning site are appropriate and working effectively Providing advice and guidance to providers Maintaining an overview of provision across providers Checking that all the staff involved in assessment are appropriately qualified and experienced Checking the credibility of assessment methods and instruments Checking internal moderation systems Through sampling, monitoring and observing assessment processes and learners evidence to ensure consistency across providers Checking assessors decisions Individuals who will be external moderators should be experienced, know the learning area well, have undergone training for moderation, and have credibility among assessors and within their area of knowledge and expertise. A high level of personal and interpersonal skills are also required. NOTE: We will discuss both so water down what you need per type.
1. PLAN FOR MODERATION Before you can start with Moderation, ask the following questions first: -Who asked for moderation? -Why did they ask for moderation? -What type of moderation must I do? -What method of moderation must I follow? -Any special needs? -When must I do it and when is it expected back? -Where will I get the documents / resources? -Who is involved / any role players? -What Unit Standards? -What SETA is involved? -What is my fees and who is paying me?
TIPS TO INCLUDE IN YOUR PLAN Who asked for moderation? Why did they ask for moderation? How will moderation be done? Is it for your company, a different Training Provider or the SETA? Do you have an agreement with them in place? How many moderations must be done? Do you have a list of the names (summary) Do you have evidence or control what you received from them? Payment terms? Make sure there is no conflict of interest. Confidentiality issues. Internal (Providers) or External (SETA) Moderation required. Will all registered standards be moderated? Will all candidates be moderated? If not, what percentage? Will all assessments be moderated? If not, what percentage? Will all training programmes be moderated? This will also become the method of Moderation. Examples: For SETA Verification. New Programme Approval Purchases maybe Appeals Process QMS Review New Assessors New Process or Policy Implementation Measurement of process effectiveness. (Cost effectiveness of guides or Assessors) Bad SETA Site Visit Follow-up request on previous moderation report NOTE: The method will determine what type of documents I will need for the different parties and is normally one of the first questions you ask. How will moderation be done? Moderation of assessment methods, instruments and materials - before assessment? - post assessment? - both? Rolling over a five-year period with moderation of different aspects each year? On-Site or Off-Site Moderation (Will you sit in during the Assessment Process or not)
TIPS TO INCLUDE IN YOUR PLAN Any special needs? When is it expected back? When is moderation required? Where will I get the documents / resources? Who is involved / any role players? Any special needs? Any special instructions? If anyone is paying for the service, do they have any special instructions of what is expected? Any time frame given for the final report? Any time given that the final moderation reports must be handed back? Can you ask the provider for additional documents during the process and how much time will be allowed for this? Continuously? Monthly? Quarterly? Annually? Must you pick it up or will they drop it off? Do you have a control over what you received and handed back? Any confidentially or conflict of interested? Any time frames? Training Provider Accreditation Details. Assessors registration and accreditation details. Previous Moderation Reports. Copy of the Training Roll-out Plan (how was this programme delivered to the learners) Copy of the Assessment and Moderation Policy Are you allowed to contact the provider or the training administrator who normally deals with the files directly? Where must the final report go to on completion? Who requested the report and where the final report must go too. Who can you ask or contact if more documents / recourses is required?
TIPS TO INCLUDE IN YOUR PLAN What Unit Standards? What SETA? What unit standard (s) must be moderated? Are you a qualified Moderator? Are you registered with this SETA to moderate this unit standard? Did you double check that you still active. (Reminder that you must re-register every 3 years) What NQF Level will give you and indication of the level of work expected. How many credits Calculate the Notional Hours, 30% is Theory and 70% is Practical. Now you know a) how much work is required, b) the minimum classroom training that is expected for each learner and c) the amount of workplace experience that is required per learner. Registration date is the Unit Standard still active? What is the Purpose of the Unit Standard? You cannot use it for the mining industry if it s specifically for the banking sector. Learning assumed to be in place If it say had to complete another unit standard first, then you need proof that the learner completed it before he can do this one. Specific Outcomes. Will give you an indication of what is expected in the learning programme. Specific Outcomes, Assessment Criteria, Range Statement, Critical Cross-field Outcomes + Embedded Knowledge. All of this must be covered in the Assessment Process. Is there any matrix showing all the above and where it was covered in the Assessment Guide? Read the Embedded Knowledge and Range Statement for additional Assessment Requirements. Are you registered with the specific SETA to moderate this Unit Standard? Remember that one Unit Standard can fall under different SETA s, but you still need approval by each SETA to Moderate in that Sector. Did you research the specific SETA to see their Moderation requirements? Sometimes a SETA provide their own Moderation tools or have additional requirements for moderations.
ADDITIONAL: DIFFERENT TYPES OF EVIDENCE CAN BE USED NOTE: The type of evidence or resources required may also depends on the type and method of moderation select.. Types of resources required is totally depended on the a) type, b) methods and c) special requirements of the moderation. First answer these three question in order to determine what to check. Below some example of possible resources you could request / need for the different methods of moderation: Evaluation of Assessors Performance -List of assessments completed -Previous Moderator reports that include this assessor -Staff File -Previous Appeals -Personal Development Plan - Staff Appraisals Check the Assessment Policy -The Assessment Policy -QMS Review Policy -Staff procedures and Policies or Service Level Agreements with Contract Workers. -Previous Moderator Reports -SETA Site Visit Reports Full Assessment Practices for SETA Verification -Company registration -Training Provider Accreditation -Copy of the Unit Standard -Staff files and registrations as Assessors -Assessment Guide -Summary of the learners -Previous Moderator reports -SETA Site Visits
BEFORE YOU CAN CONTINUE TO THE NEXT STEP: 1. Did I get all the information I needed in my Plan from the different parties? 2. You may allow additional time during this period for the parties to produce the missing information before you can continue Remember to allocate a time period for this. 3. Do you have set cut-off dates? 4. Moderation might be on or off site. PROBLEM: Does everyone know what is required from them during your visit? What is required from everyone and what will the process be if the information is not ready? You also might include some communication letters to all parties, especially if it s a on-site moderation. (additional options)
2. CONDUCT MODERATION Take your moderator plan and start checking the product with the right attitude and strategy: Look at the product (Assessment) and try and identify what is going on, what belongs where or what is I know what is required so now I just need to find the evidence in the product (Assessment). included in the file.
TIPS ON HOW TO START WITH THIS PROCESS 1. Study the Unit Standard and make sure you know what is required. Make a brief summary of all the important things to check. 2. Make sure the Provider, the Programme and the Assessors is registered and have the scope. 3. Now start looking at the POE. Take your plan and see if you can find the required evidence. Determine if the entire processes was followed. Is there enough evidence that the assessment took place. Look for a matrix that shows how all the unit standards and assessment criteria was covered in the learning material and the assessment process. Is there enough evidence that the learner is competent, even if you were not present during the assessment process. Appeals and Special needs covered. Attendance and workplace experience. Notional hours (practical and theory to national hours) 4. Make notes as you go along that will be used in writing the final report
3. ADVICE AND SUPPORT Identify the seriousness of the problem that will determine the type of action required. (Constructive Feedback)
EXPLAINING THE APPEALS PROCESS An appeals process will be in place for applicants who feel that moderations have taken place unfairly. Appeals can be brought against: Assessments that are perceived to have been unfair, invalid or unreliable. The assessment outcome, if the assessor s judgement is considered to be biased. Inadequate expertise or experience of the assessor, if this is considered to have influenced the assessment. Unethical practices. Learners should be secure in the knowledge that they can appeal against an unfair assessment. Every candidate being assessed and or moderated has the right to appeal against the decisions, conduct or compliance of the assessor or moderator. All candidates shall be familiarized with proper appeals procedure. Any learning material should: -Make reference and accommodate for special needs. -Have a copy of the appeals process included in the material. -Have a copy of the appeals template included in the material. How can you allow for learners to appeal if you do not have the policy or template to use in place?
4. REPORT AND ADMINISTRATION Here you have two basic options and depends on your own preference and processes. 1. You can create yourself a template (table form guide) to complete. This is very helpful if you know you will do more of the same types of moderation in the future. Easy to complete. 2. You can type out an moderation report from A to Z in the form of a letter. This is very helpful if it s the first time you do a moderation may be a very specific method of moderation.
THIS REPORT SHOULD INCLUDE: INDUCTION Provider detail. Learner details. Unit Standard details. Moderation type, method and special requirements. People involved. Dates, times if any. SETA in question. Previous moderation reports or comments. BODY Detailed summary of the evidence found. Detail summary of the evidence not found. No copies required only reference to pages and evidence documents. SUMMARY Executive summary of findings. Highlight summary of SERIOUS findings. Highlight action required for SERIOUS Findings. Deadlines or possible process map to address.
REMEMBER: CONFIDENTIALITY Given the sensitive content of the feedback session and the information dealt with, it is important that the whole process of communication within the practice of moderation is conducted under the requirements of confidentiality. Keeping in mind that the implications of the content of the communication can be profound for the assessor, especially when one considers the position of the assessor as a member of a larger team, it is necessary that the information that concerns the moderation of a specific assessor s assessment practices be confidential. Failure to adhere to confidentiality requirements may, result in a situation where an assessor may feel betrayed or exposed by the moderation process. This can ultimately damage and demean the credibility of the moderation process as well as the professionalism of the assessor and the corporate aims of the organisation POSSIBLE SOLUTION Draft different reports or letters and address this in person with the relevant parties. DE-REGISTRATION OF ASSESSORS AND MODERATORS Both assessors and moderators may be suspended and/or de-registered if: They do not uphold the assessor or moderator code of conduct. They do not adhere to the policies and procedures of both the accredited provider and the SETA. They do not adhere to or uphold applicable legislation and/or regulations They do not continue to conduct assessments. They are de-registered as an assessor.
5. REVIEW 1. Did you cover all the evidence required in your plan? 2. Did you meet all the moderation requirements? 3. Did you meet the specific SETA moderation requirements or guidelines? Appropriate Fair Manageable Integration Valid Direct Authentic Sufficient Systematic Open The method of assessment is suited to the performance being assessed. The method of assessment does not represent any barriers to achievements related to the evidence. The methods used make for easily arranged, cost effective assessments. Evidence collection is integrated into the work or learning process where this is appropriate and feasible. The assessment focuses on the requirements laid down in the Standard; i.e. is fit for purpose. The activities in the assessment mirror the conditions of actual performance as closely as possible. The assessor is satisfied that the work being assessed is attributable to the person being assessed. The evidence establishes that all criteria are met and that performance can be repeated consistently. Planning and recording is sufficiently rigorous to ensure that assessment is fair. Learners can contribute to the planning and accumulation of evidence. Learners understand the assessment process and criteria that apply. Consistent The same assessor would make the same judgement again in similar circumstances. The judgement made is similar to the judgement that would be made by other assessors.