THE GREEK STATE CERTIFICATE FOR LANGUAGE PROFICIENCY GENERAL EXAMINATION DESCRIPTION 1

Transcription

1 General Information Name of examination THE GREEK STATE CERTIFICATE FOR LANGUAGE PROFICIENCY GENERAL EXAMINATION DESCRIPTION 1 KPG [Kratiko Pistopiitiko Glossomathias] (State Certicicate of Language Proficiency) Language tested Examining institution Date of this version Type of examination Purpose Target population of test-takers per year English, French, German, Italian (and as of 2008 Spanish) Ministry of National Education and Religious Affairs April 2008 International National Regional Institutional To certify FL competence/ proficiency Lower Sec Upper Sec Uni/College Students Adult What is the overall aim? Exams for different levels of proficiency aim to test the candidate s ability to make socially purposeful use of the target language at home and abroad. (Dendrinos, 2004:2, What are the more specific objectives? If available describe the needs of the intended users on which this examination is based. Specifically, the exams test the candidate s: use of language for comprehension and production of oral and written discourse his/her ability to act as mediators across languages his/her awareness of how the target language works to produce socially purposeful meanings (Dendrinos, 2004:1, What is/are principal domain(s)? Public Personal Occupational Educational Which communicative activities are tested? 01 Listening comprehension 02 Reading comprehension 03 Spoken interaction 04 Written interaction 05 Spoken production 06 Written production 07 Integrated skills (only for levels C1 & C2) 08 Spoken mediation (for levels B1, B2 and C1, C2) 09 Written mediation (for levels B1, B2 and C1, C2) 10 Language (at the level of lexis, sentence, text and discourse grammar) 11 Language Awareness THE 4 TEST PAPERS Module1: Reading Comprehension and Language Awareness Module 2: Writing Production (& Written Mediation) Module 3: Listening Comprehension Module 4: Speaking Production (& Oral Mediation) 1 Description by Prof. Bessie Dendrinos (vdendrin@enl.uoa.gr) 1

2 Give name and duration of test subtests (referred to above right): What type(s) of test tasks are used? What Information is published for candidates and teachers? 2 What is Reported? 1 Module1: 2 Module 2: 3 Module 3: 4 Module 4: Multiple choice True/False Matching Ordering Gap fill sentence Sentence completion Gapped text / cloze, selected response Open gapped text / cloze Short answer to open question(s) Extended answer (text / monologue) Interaction with examiner Interaction with peers Overall aim Principal domain(s) Test subtests Test tasks Sample test papers Past papers Video of format of oral test (for Examiners only) Global Grade Grade per subtest Duration of Subtest Different for each level (see specifications available at Sample answer papers Marking schemes Grading schemes Standardised performance samples showing pass level (for script raters only) Global Grade plus graphic profile Profile per subtest Test development What organisation decided that the examination was required? What influence do other, external organizations have on design and development? What other factors determined design and development of examination? In producing test tasks are specific features of candidates taken into account? Who writes the items or develops the test tasks? Have test writers guidance to ensure quality? s Ministry of Education Determine examination domain or content Determine exam format and type of tasks A needs analysis Internal description of examination aims Internal description of language level Profile of candidates Linguistic background (L1) Language learning background Age Social-cultural factors A team of testing and assessment specialists engaged to work at the Universities of Athens and Thessaloniki, under the supervision of professorial staff. (Ministry of National Education and Religious Affairs, 2003:7) Training 2 Note: Not everything outlined above is made available to everyone or published on the internet. (a) The video of format of the oral is made available to multipliers who hold seminars with Examiners. (b) The evaluation criteria and marking scheme of the oral is made available to everyone, but the detailed marking scheme and analytical criteria is only made available to multipliers and Examiners. (c) The detailed evaluation criteria and, karking scheme and [semi-standardized] samples showing expected excellence B2 level is made available to Evaluators of Module 2. 2

3 Guidelines Checklists Examples of appropriate tasks Calibrated to CEFR level description Is training for test writers provided? Are test tasks discussed before use? If yes, by whom? Are tests tasks pre-tested? If yes, how? Internal group discussion External examination committee Test items are pre-tested with groups of teachers and then piloted with target language users abroad. Alterantively, parallel tests are piloted with target language users in Greece. If no, why not? Is the reliability of the test estimated? If yes, how? Are different aspects of validity estimated? If yes, describe how? Marking: Subtest How are the test tasks marked? Where are the test tasks marked? What criteria are used to select markers? How is accuracy of marking promoted? Describe the specifications of the rating criteria of productive and/or integrative test tasks. Data collection and psychometric procedures Face validity Content validity Concurrent validity Predictive validity Construct validity They are a-priori and a-posteriori assessed for reliability and validity through test-item analysis of piloted materials and actual exam materials. The results are discussed by all language project managers and the rest of the members of the Central Examination Board. (cf. Ministry of National Education and Religious Affairs, 2003:7) For receptive test tasks: Optical mark reader For productive or integrated test tasks: Trained examiners /evaluators Centrally Regular checks by co-ordinator Training of markers/raters Moderating sessions to standardise judgements Using standardised examples of tasks marks for different aspects of each task rating scale for overall performance in test rating grid for aspects of test performance rating scale for each task 3

4 rating grid for aspects for each task Are productive or integrated test tasks single or double rated? If double rated, what procedures are used when differences between raters occur? Is inter-rater agreement calculated? Two simultaneous raters Double marking of scripts / recordings Average of two marks Two markers discuss and reach agreement Grading: Subtest Are pass marks and/or grades given? Results What results are reported to candidates? In what form are results reported? On what document are results reported? Is information provided to help candidates to interpret results? Give details. Do candidates have the right to see the corrected and scored examination papers? Do candidates have the right to ask for remarking? Test Analysis and Post-examination Review Is feedback gathered on the examinations? If yes, by whom? Is the feedback incorporated in revised versions of the examinations? Is data collected to do analysis on the tests? If yes, indicate how data are collected? For which features is analysis on the data gathered carried out? Are performances of candidates from different groups analysed?. If so, describe how. Describe the procedures to protect the confidentiality of data. Are relevant measurement concepts explained for test users? If so, describe how. Pass marks Grades Global grade or pass / fail Pass / fail Reported on the Internet Certificate / Diploma Internal experts Test administrators On all tests During pre-testing and piloting procedures During live examinations After live examinations Difficulty Discrimination Reliability Validity Only the members of the team of specialists has the seen the subtests of the exam and only the project manager has a final copy, taken to the Ministry of Education on the day the exam is disseminated to the Exam Centres throughout the country, through a VBI system. They are not explained but the public knows because the same procedures are used for the Panhellenic university entrance exams organized by the Ministry of National 4

5 Education. Rationale for making decisions Give the rationale for the decisions that have been made in relation to the examination or the test tasks in question. The political decision to develop a unified system, through which to measure the proficiency level in various languages, is motivated by the will to promote social multilingualism and individual pluringualism in Greece; to expedite foreign language learning which is viewed as an asset for educational and professional mobility within and outside the EU; to recognize the rich linguistic resources of Greek residents, legitimating their knowledge of languages as professional qualifications (Ministry of National Education and Religious Affairs, 2003:7) References Dendrinos, B. (2004): KPG- a new suite of national language examinations for Greece: philosophy and goals. Lecture at the ICC AGM and Annual Conference Athens, 19 March Available on line: www. icc-europe.com (Last access 11/06/2004). Ministry of National Education and Religious Affairs (2003): The State Certificate of Language Proficiency (KPG). News Bulletin. Ministry of National Education and Religious Affairs (2004a): Το Eκπαιδευτικό Σύστηµα / Πιστοποίηση Γλωσσοµάθειας / Ξένες Γλώσσες/ Γενικές Πληροφορίες - Οργανωτικά και ιοικητικά Θέµατα / Οργανωτική οµή - ιοίκηση - Επιστηµονική Υποστήριξη Available on-line: (Last access 11/06/2004). Ministry of National Education and Religious Affairs (2004b): Το Eκπαιδευτικό Σύστηµα / Πιστοποίηση Γλωσσοµάθειας / Ξένες Γλώσσες/ Ενιαίο Πλαίσιο Προγράµµατος των Εξετάσεων. Available on line: (Last access 11/06/2004). Ministry of National Education and Religious Affairs (2004c): Το Eκπαιδευτικό Σύστηµα / Πιστοποίηση Γλωσσοµάθειας / Ξένες Γλώσσες/ Κοινές Προδιαγραφές και Χαρακτηριστικά των Εξετάσεων Available on line: (Last access 11/06/2004). Ministry of National Education and Religious Affairs (2004d):Το Eκπαιδευτικό Σύστηµα / Πιστοποίηση Γλωσσοµάθειας / Ξένες Γλώσσες / είγµατα Θεµάτων και Προδιαγραφές Σύνταξης / Αγγλική Γλώσσα/Προδιαγραφές Σύνταξης Θεµάτων (Επίπ. Β2) Available on line: Last access 11/06/2004). 5