A Test Development Life Cycle Framework for Testing Program Planning
|
|
- Daniel Daniels
- 6 years ago
- Views:
Transcription
1 A Test Development Life Cycle Framework for Testing Program Planning Pamela Ing Stemmer, Ph.D. February 29, 2016 How can an organization (test sponsor) successfully execute a testing program? Successfully operating a testing program with a valid, legally defensible examination is no small task. Reviewing necessary information and making numerous decisions can be overwhelming. Effective planning and successful execution of a testing program may be achieved by using a test development life cycle framework. In this paper, we present a framework for planning and decision making, to guide successful execution of new and existing testing programs. We recommend the use of this test development life cycle framework for initial planning, and also as a structure for evaluating current testing program functioning. In this document, we introduce each test development stage. Further details, including issues to address at each stage of the test development cycle, questions to consider as a test sponsor, and questions to ask potential or existing vendors will be presented throughout the rest of this white paper series. This white paper series is focused on testing programs in the domain of certification and licensure. The purpose of a credential is to provide evidence to the public that an individual meets the minimum level of competence required for safe practice in a particular profession. Many certification and licensure programs develop one or more valid tests to assure that the knowledge and performance capabilities of their credential holders meet those standards. To achieve validity, legal defensibility, and public confidence in the credential, effort is required throughout the planning, execution, and evaluation of a testing program. Important questions need to be considered by test sponsors and information needs to be gathered from potential and/or existing vendors. Planning ahead, using a test development life cycle framework, such as the one presented in Figure 1, can ensure that important questions and decisions are not overlooked throughout the planning and execution of a testing program. Benefits of planning ahead, using a test development life cycle framework include, but are not limited to, ensuring security, quality assurance, validity evidence, legal defensibility, and preparation for accreditation application. Measurement professionals employ the 2014 Joint Standards for Educational and Psychological Testing (AERA/APA/NCME, 2014) and the National Commission for Certifying Agencies (NCCA) 2016 Standards for the Accreditation of Certification Programs (Institute for Credentialing Excellence, 2014) as psychometric criteria for validating examinations, such as licensing and certification examinations. 1
2 The test development life cycle includes the following stages: job analysis, test specifications, item development, test assembly, standard setting, equating, test administration, scoring and reporting, and technical report. 1. Job Analysis 9. Technical Report 2. Test SpecificaWons 8. Scoring and ReporWng 3. Item Development 7. Test AdministraWon 4. Test Assembly 6. EquaWng 5. Standard Se[ng Figure 1. The test development life cycle provides a useful framework for addressing important questions and decisions throughout the planning, execution, and evaluation of a testing program. 2
3 1. Job Analysis A job analysis (or practice analysis) is an empirical method used to define the content that is to be measured by an examination (Downing, 2006; Figure 1). Use of a job analysis study as a method of defining examination content is clearly endorsed by both the 2016 NCCA Standards (Standards 13 and 14) and the 2014 Joint Standards (Standards 4.12, 11.03, and 11.13). For credentialing programs, NCCA Standard 14 states that a job analysis must be conducted and documented, in order to delineate and evaluate job responsibilities and content domains associated with the credential s purpose. Numerous approaches may be taken when conducting a job analysis. Common methods and issues to consider when planning a job analysis will be discussed in the next installment of this white paper series. 2. Test Specifications Test specifications provide a detailed, comprehensive description of an examination s components and features (Figure 1). Test specifications include a test blueprint, which outlines the number or proportion of items assigned to measure each content domain (and subdomain, if applicable). Other elements of test specifications include the following: test length; item format(s); maximum testing time; candidate directions; test administration procedures; any permissible materials; and procedures for scoring and reporting (AERA/APA/NCME, 2014). According to The 2014 Joint Standards (Standards 4.01 and 4.02) and the 2016 NCCA Standards (Standard 15), comprehensive test specifications must be established and documented. Issues to consider regarding creating and updating test specifications, such as time and processes, will be presented in an upcoming installment of this white paper series. 3. Item Development Item development comprises item writing and item review (Figure 1). Important item development issues to consider include, but are not limited to, the following: subject matter expert (SME) recruitment and training; guidelines for item content and structure; security of item content; item authoring methods; item banking. The content domains outlined in the test blueprint guide the item development process. The 2014 Joint Standards (Standards 4.07 and 4.08) and the 2016 NCCA Standards (Standards 13 and 16) emphasize the importance of developing and documenting systematic item development procedures. Questions to address regarding item development will be discussed in an upcoming installment of this white paper series. 4. Test Assembly Item selection for an examination form should be guided by the test blueprint (Figure 1). In addition to alignment with test blueprint, there are many issues for test sponsors to consider regarding test assembly, including: key balance; placement of anchor items and accounting for an equating model; inclusion and coverage of pretest items; test security; and quality control. There are many approaches to test form assembly, which may be appropriate for paper and pencil administration, computer based testing (CBT), or both. Test form assembly should be guided by psychometric principles, such as classical test theory (CTT) or item response theory 3
4 (IRT). Tests may be assembled so as to be presented in a fixed format or with individualized elements, such as with computer adaptive testing (CAT) or linear on the fly testing (LOFT). Expectations for test assembly are addressed in the 2016 NCCA Standards (Standard 16) and the 2014 Joint Standards (Standard 4). Further discussion regarding test assembly issues and questions will be presented in an upcoming installment of this white paper series. 5. Standard Setting Standard setting refers to the process by which a passing score for an examination is established (Cizek, 2006; Figure 1). In certification and licensure, the goal of standard setting is to determine the level of performance (i.e., knowledge, skill and/or ability) required by an individual to demonstrate minimal competence in a particular profession. Issues for test sponsors to consider include: method selection; SME panel composition, recruitment, and training; and determining when standard setting is appropriate, versus equating a new form with an existing examination form. Standard setting is endorsed by the 2016 NCCA Standards (Standards 13 and 17) and the 2014 Joint Standards (Standards 5.22 and 11.16). Methods of standard setting and issues to consider when planning a standard setting will be presented in a future installment of this white paper series. 6. Equating According to the 2016 NCCA Standards (Standard 21), testing programs must establish that candidates will not receive any advantages or disadvantages resulting from content structure and/or difficulty across different forms of an examination (Figure 1). Issues for test sponsors to consider regarding equating include: selection of appropriate statistical procedures (i.e., equating models); establishing equivalence across examination forms that have been translated into different languages; and ensuring that examination forms are compliant with the requirements of the test blueprint and selected equating model. Questions to ask and issues to consider regarding equating will be discussed in an upcoming installment of this white paper series. 7. Test Administration Test administration is the most visible component of the test development life cycle and delivers vital evidence of test score validity, when it is executed in an organized, consistent, and effective manner (Downing, 2006; Figure 1). There are many elements of test administration for test sponsors to consider, such as: proper candidate identification; standardization of testing environments; examination security; proctor training and monitoring; quality control; and candidate accommodation. The 2014 Joint Standards (Standards ) and the 2016 NCCA Standards (Standard 18) assert the need for testing programs to establish and comply with policies and procedures designed to protect examination content and ensure standardized candidate experiences. Further discussion regarding test administration issues to consider will be presented in a future installment of this white paper series. 4
5 8. Scoring and Reporting There are many different ways to apply a scoring key to candidate responses from an examination, ranging from simple (e.g., number of correct out of total number of items) to complex (e.g., partial credit, penalties for incorrect responses; Figure 1). The range of valid scores may be changed to suit the program's needs or the candidates' understanding, through the use of raw, percentage, or scaled scores. Other important scoring issues for test sponsors to address include key validation, item analysis, and any additional statistical analysis information required for the testing program (e.g., quarterly reports, annual reports). With respect to score reporting, decisions need to be made regarding: what type of information will be provided to candidates; what information will be reported to the test sponsor (e.g., total scores, subscores, average examination performance statistics); in what format will information be provided to candidates and the sponsor organization (e.g., paper, electronic, preliminary, official); whether contextual cues such as explanatory text, visual representations, or uncertainty information will be included; and on what type of timeline will the information be provided (e.g., instant, delayed). Issues related to scoring and reporting are addressed by both the 2016 NCCA Standards (Standard 19) and the 2014 Joint Standards (Standards ). Further detail regarding scoring and reporting issues to consider will be discussed in an upcoming installment of this white paper series. 9. Technical Report Thorough documentation of the entire test development process provides validity evidence for the testing program (Figure 1). With respect to technical reports, it is important to consider the level of documentation that test sponsors should receive from contracted vendors, which is guided by the 2014 Joint Standards (Standard 7.04) and the 2016 NCCA Standards (Standards 13 21). Additional considerations regarding technical documentation will be provided in a future edition of this white paper series. Operating a successful testing program with a valid, legally defensible examination, is no small feat. Information gathering, review, and decision- making can be overwhelming. Using a test development life cycle framework may facilitate effective planning and successful execution of a testing program. For more detailed descriptions, issues, and questions to ask regarding each stage of the test development life cycle, check the Comira website at the end of each month for the next white paper publication. If you would like a free one- hour consultation on the quality of your testing program/planning, please use the following link to contact Comira s Psychometric Team. 5
6 References American Educational Research Association, American Psychological Association, National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Washington, DC: AERA. Cizek, G. J. (2006). Standard setting. In S. M. Downing & T. M. Haladyna (Eds.), Handbook of test development (pp ). Mahwah, NJ: Lawrence Erlbaum Associates, Inc. Downing, S. M. (2006). Twelve steps for effective test development. In S. M. Downing & T. M. Haladyna (Eds.), Handbook of test development (pp. 3-25). Mahwah, NJ: Lawrence Erlbaum Associates, Inc. Institute for Credentialing Excellence. (2014). National Commission for Certifying Agencies Standards for the Accreditation of Certification Programs. Washington, DC: ICE. 6
Job Analysis 101. Pamela Ing Stemmer, Ph.D. April 11, 2016
Job Analysis 101 Pamela Ing Stemmer, Ph.D. April 11, 2016 The certification program must have a job analysis that defines and analyzes domains and tasks related to the purpose of the credential, and a
More informationTest Development. and. Psychometric Services
Test Development and Psychometric Services Test Development Services Fair, valid, reliable, legally defensible: the definition of a successful high-stakes exam. Ensuring that level of excellence depends
More informationExamination Report for Testing Year Board of Certification (BOC) Certification Examination for Athletic Trainers.
Examination Report for 2017-2018 Testing Year Board of Certification (BOC) Certification Examination for Athletic Trainers April 2018 INTRODUCTION The Board of Certification, Inc., (BOC) is a non-profit
More informationPASSPOINT SETTING FOR MULTIPLE CHOICE EXAMINATIONS
WHITE WHITE PAPER PASSPOINT SETTING FOR MULTIPLE CHOICE EXAMINATIONS CPS HR Consulting 241 Lathrop Way Sacramento, CA 95815 t: 916.263.3600 f: 916.263.3520 www.cpshr.us INTRODUCTION An examination is a
More informationAudit - The process of conducting an evaluation of an entity's compliance with published standards. This is also referred to as a program audit.
Glossary 1 Accreditation - Accreditation is a voluntary process that an entity, such as a certification program, may elect to undergo. In accreditation a non-governmental agency grants recognition to the
More informationInnovative Item Types Require Innovative Analysis
Innovative Item Types Require Innovative Analysis Nathan A. Thompson Assessment Systems Corporation Shungwon Ro, Larissa Smith Prometric Jo Santos American Health Information Management Association Paper
More informationSue Orchard President & CEO Comms Multilingual Ltd. Dr. Pamela Ing Stemmer Psychometrician II Comira
Sue Orchard President & CEO Comms Multilingual Ltd Dr. Pamela Ing Stemmer Psychometrician II Comira Strategy Partnerships Implementation Localization Marketing Technology/Security Psychometrics Globalization
More informationDeveloping Certification Exam Questions: More Deliberate Than You May Think
Publications 5-2018 Developing Certification Exam Questions: More Deliberate Than You May Think Cheryl (Cheri) L. Marcham Embry-Riddle Aeronautical University, march617@erau.edu Treasa M. Turnbeaugh Board
More informationNational Council for Strength & Fitness
1 National Council for Strength & Fitness Certified Personal Trainer Examination Annual Exam Report January 1 to December 31, 2016 March 24 th 2017 Exam Statistical Report Copyright 2017 National Council
More informationThe Examination for Professional Practice in Psychology: The Enhanced EPPP Frequently Asked Questions
The Examination for Professional Practice in Psychology: The Enhanced EPPP Frequently Asked Questions What is the Enhanced EPPP? The Enhanced EPPP is the national psychology licensing examination that
More informationR&D Connections. The Facts About Subscores. What Are Subscores and Why Is There Such an Interest in Them? William Monaghan
R&D Connections July 2006 The Facts About Subscores William Monaghan Policy makers, college and university admissions officers, school district administrators, educators, and test takers all see the usefulness
More informationTest Development: Ten Steps to a Valid and Reliable Certification Exam Linda A. Althouse, Ph.D., SAS, Cary, NC
Test Development: Ten Steps to a Valid and Reliable Certification Exam Linda A. Althouse, Ph.D., SAS, Cary, NC ABSTRACT The intent of a certification program is to evaluate the knowledge and skills of
More informationMCS Experience in Selection & Recruitment (Saudi Arabia)
MCS Experience in Selection & Recruitment (Saudi Arabia) Eid Al-Harby, Ph.D Director, Assessment and Selection Program for Civil Service Personnel (ASCSP), Ministry of Civil Service, The Kingdom of Saudi
More information2. Is there a list of commonly used resources (textbooks, software, videos, case studies, etc.) for program directors for each program level?
EDUCATION RELATED FREQUENTLY ASKED QUESTIONS 1. How often are the curriculum maps revised? The maps are reviewed annually by the curricula workgroup of the Council for Excellence in Education (CEE). Minor
More informationProject Management Professional (PMP) Examination Content Outline
Project Management Professional (PMP) Examination Content Outline Project Management Institute Project Management Professional (PMP) Examination Content Outline April 2015 Published by: Project Management
More informationReliability & Validity
Request for Proposal Reliability & Validity Nathan A. Thompson Ph.D. Whitepaper-September, 2013 6053 Hudson Road, Suite 345 St. Paul, MN 55125 USA P a g e 1 To begin a discussion of reliability and validity,
More informationWhat are the Steps in the Development of an Exam Program? 1
What are the Steps in the Development of an Exam Program? 1 1. Establish the test purpose A good first step in the development of an exam program is to establish the test purpose. How will the test scores
More informationProject Management Professional (PMP)
Project Management Professional (PMP) E X A M I N AT I O N CO N T E N T O U T L I N E Project Management Institute Project Management Professional (PMP) Examination Content Outline June 2015 Published
More informationOverview. Considerations for Legal Defensibility in Standard Setting. Portland, Oregon September 16, 2016
Have you been involved in a standard setting for your industry? A. Yes B. No Test your iclicker and answer the question above. Considerations for legal defensibility in standard setting Overview Methods
More informationEnis Doğan. The future is here: The (new) Standards for Educational and Psychological Testing. Research Fellow/ Resident Expert at NCES
The future is here: The (new) Standards for Educational and Psychological Testing Enis Doğan Enis.Dogan@ed.gov Research Fellow/ Resident Expert at NCES January, 13 2016 Overview PART I: Background on the
More informationNATIONAL CREDENTIALING BOARD OF HOME HEALTH AND HOSPICE TECHNICAL REPORT: JANUARY 2013 ADMINISTRATION. Leon J. Gross, Ph.D. Psychometric Consultant
NATIONAL CREDENTIALING BOARD OF HOME HEALTH AND HOSPICE TECHNICAL REPORT: JANUARY 2013 ADMINISTRATION Leon J. Gross, Ph.D. Psychometric Consultant ---------- BCHH-C Technical Report page 1 ---------- Background
More informationTest Analysis Report
Test Analysis Report International Board Certified Lactation Consultant (IBCLC ) April 2018 Administrations June 19, 2018 AUTHORS Scott Frohn, Ph.D. Psychometrician Lawrence J. Fabrey, Ph.D. Chief Psychometrician
More informationTesting Issues and Concerns: An Introductory Presentation to the State Directors of Career and Technical Education
Testing Issues and Concerns: An Introductory Presentation to the State Directors of Career and Technical Education Kurt F. Geisinger, Ph.D. Director, Buros Center for Testing University of Nebraska-Lincoln
More informationPractice Analysis: What, Why, and How
Practice Analysis: What, Why, and How Higher level of risk to the association The market and/or regulators may use certification for highstakes decisions Technical guidelines dictate right practice (vs.
More informationSix Major Challenges for Educational and Psychological Testing Practices Ronald K. Hambleton University of Massachusetts at Amherst
Six Major Challenges for Educational and Psychological Testing Practices Ronald K. Hambleton University of Massachusetts at Amherst Annual APA Meeting, New Orleans, Aug. 11, 2006 In 1966 (I began my studies
More informationScore Reporting: More Than Just Pass/Fail. Susan Davis-Becker, Alpine Testing Solutions Sheila Mauldin, NCCPA Debbra Hecker, NAWCO
Score Reporting: More Than Just Pass/Fail Susan Davis-Becker, Alpine Testing Solutions Sheila Mauldin, NCCPA Debbra Hecker, NAWCO Overview Review professional standards Provide guidance on considerations
More informationTechnical Report: June 2018 CKE 1. Human Resources Professionals Association
Technical Report: June 2018 CKE 1 Human Resources Professionals Association 3 August 2018 Contents Executive Summary... 4 Administration... 5 Form Setting... 5 Testing Window... 6 Analysis... 7 Data Cleaning
More informationNashville, Tennessee. CLEAR 2010 Annual Conference September Third Party Accreditation. Practice Analysis ASK THE EXPERTS
ASK THE EXPERTS Presenters: Sandra Greenberg, PhD Professional Examination Service Joseph McClintock, PhD Measurement Incorporated Nancy Orr, PhD National Board of Medical Examiners Cynthia D. Woodley,
More informationDesigning item pools to optimize the functioning of a computerized adaptive test
Psychological Test and Assessment Modeling, Volume 52, 2 (2), 27-4 Designing item pools to optimize the functioning of a computerized adaptive test Mark D. Reckase Abstract Computerized adaptive testing
More informationPortfolio Management Professional (PfMP)
Portfolio Management Professional (PfMP) E X A M I N AT I O N CO N T E N T O U T L I N E Project Management Institute Portfolio Management Professional (PfMP) Examination Content Outline Published by:
More informationPgMP Exam Prep Training - 4 Days
PgMP Exam Prep Training - 4 Days Course Length: 4 Days Course ID: PgMP Credits: 21 PDUs Course Duration: 3 or 4 days (Live in person or online class) Course Level: Advanced Day workshop- PgMP Boot Camp
More informationSERVSAFE EXAM DEVELOPMENT FAQs
How is the ServSafe examination developed? The ServSafe examination development follows three basic steps. First, a Job Task Analysis (JTA) is completed. The JTA outlines the essential knowledge and skills
More informationTIPS ON BUILDING AND USING COMPETENCY ASSESSMENTS
TIPS ON BUILDING AND USING COMPETENCY ASSESSMENTS 5 SEP 2018 CHICAGO LABELMASTER DANGEROUS GOODS SYMPOSIUM SLIDE 3 OF 39 ˑ 2018 TOURISM HR CANADA ALL RIGHTS RESERVED 1 Written Examinations Types of Assessment
More informationUsing a Performance Test Development & Validation Framework
Using a Performance Test Development & Validation Framework James B. Olsen Russell W. Smith Cristina Goodwin Alpine Testing Solutions Presentation Overview Present a useful performance test development
More informationSHRM Assurance of Learning Assessment
[SHRM ASSURANCE OF LEARNING ASSESSMENT] January 1, 2015 SHRM Assurance of Learning Assessment Technical Manual 2015 Technical Manual 1 Table of Contents PREFACE...1 PURPOSE OF THIS MANUAL... 1 AUDIENCE...
More informationProgram Management Professional (PgMP)
Program Management Professional (PgMP) E X A M I N AT I O N CO N T E N T O U T L I N E Project Management Institute Program Management Professional (PgMP ) Examination Content Outline April 2011 Published
More informationTechnical Report: May 2018 CHRP ELE. Human Resources Professionals Association
Human Resources Professionals Association 5 June 2018 Contents Executive Summary... 4 Administration... 5 Form Setting... 5 Testing Window... 6 Analysis... 7 Data Cleaning and Integrity Checks... 7 Post-Examination
More informationName of Student: Supervisor's Address (you will be sent a copy of this evaluation):
Default Question Block Note that the evaluation will be sent to the student immediately so you may want to wait to submit the survey until after you have met with the student to provide verbal feedback.
More informationValidity and Reliability Issues in the Large-Scale Assessment of English Language Proficiency
Validity and Reliability Issues in the Large-Scale Assessment of English Language Proficiency The 5 th International Conference on ELT in China Beijing, China May 21, 2007 Richard J. Patz, Ph.D. CTB/McGraw-Hill
More informationPMI Scheduling Professional (PMI-SP)
PMI Scheduling Professional (PMI-SP) E X A M I N AT I O N CO N T E N T O U T L I N E Project Management Institute PMI Scheduling Professional (PMI-SP) Exam Content Outline Published by: Project Management
More informationPackage subscore. R topics documented:
Package subscore December 3, 2016 Title Computing Subscores in Classical Test Theory and Item Response Theory Version 2.0 Author Shenghai Dai [aut, cre], Xiaolin Wang [aut], Dubravka Svetina [aut] Maintainer
More informationInvestigating Common-Item Screening Procedures in Developing a Vertical Scale
Investigating Common-Item Screening Procedures in Developing a Vertical Scale Annual Meeting of the National Council of Educational Measurement New Orleans, LA Marc Johnson Qing Yi April 011 COMMON-ITEM
More informationIntercultural Development Inventory (IDI): Independent Review
Intercultural Development Inventory (IDI): Independent Review Submitted By: Andrew Wiley 917.885.0858 Awiley@acsventures.com 11035 Lavender Hill Drive, Suite 160-433 Las Vegas, NV 89135 w w w. a c s v
More informationINNOVATIONS IN COMPETENCY-BASED EDUCATION. Director of Competency- Based Education Services
INNOVATIONS IN COMPETENCY-BASED EDUCATION Dr. Larry Banks Director of Competency- Based Education Services Wonderlic, Inc Charlie Wonderlic CEO Wonderlic, Inc. COMPETENCY-BASED EDUCATION (CBE) CBE is emerging
More informationEvaluating Content Alignment in the Context of Computer-Adaptive Testing: Guidance for State Education Agencies
Evaluating Content Alignment in the Context of Computer-Adaptive Testing: Guidance for State Education Agencies Carole Gallagher June 2016 The work reported herein was supported by grant number #S283B050022A
More informationVirginia Communication and Literacy Assessment (VCLA)
Virginia Communication and Literacy Assessment (VCLA) VACTE/ATE-VA Fall Conference October 8-9, 2015 Phil Gullion Director, Educator Solutions for Licensing and Learning Copyright 2015 Pearson Education,
More informationIndependent Verification of the Psychometric Validity for the Florida Standards Assessment. Executive Summary. August 31, 2015
Independent Verification of the Psychometric Validity for the Florida Standards Assessment Executive Summary August 31, 2015 Submitted to: Vince Verges Florida Department of Education 325 W. Gaines St.
More informationALTE Quality Assurance Checklists. Unit 4. Test analysis and Post-examination Review
s Unit 4 Test analysis and Post-examination Review Name(s) of people completing this checklist: Which examination are the checklists being completed for? At which ALTE Level is the examination at? Date
More informationTHE AMERICAN HOLISTIC NURSES CREDENTIALING CORPORATION S CERTIFICATION PROGRAM DEVELOPMENT PROCESS OVERVIEW
THE AMERICAN HOLISTIC NURSES CREDENTIALING CORPORATION S CERTIFICATION PROGRAM DEVELOPMENT PROCESS OVERVIEW The Role-Delineation Process is automatically initiated every five years, unless the annual literature
More informationCheating 2.0. Case Study. How to Fight Back Against Cheaters By Dave Meissner,
Cheating 2.0 How to Fight Back Against Cheaters By Dave Meissner, Vice President, Solution Services, Prometric In today's world, it's almost impossible to find an adult who has not taken a test of some
More informationGlossary of Terms Ability Accommodation Adjusted validity/reliability coefficient Alternate forms Analysis of work Assessment Band Battery
1 1 1 0 1 0 1 0 1 Glossary of Terms Ability A defined domain of cognitive, perceptual, psychomotor, or physical functioning. Accommodation A change in the content, format, and/or administration of a selection
More informationNACE 30-HOUR COACHING CERTIFICATION PROGRAM
NACE 30-HOUR COACHING CERTIFICATION PROGRAM National Association of Colleges and Employers 62 Highland Avenue, Bethlehem, PA 18017 610.868.1421 www.naceweb.org PROGRAM OVERVIEW NACE 30-Hour Coaching Certification
More informationAGENDA. What Is Our Profession? Defining The Competencies Self Assessment Tools For Growth What About Professional Responsibility What Is Your Legacy?
Developing Professional Competence Copyright 2016 State Volunteer Mutual Insurance Company AGENDA What Is Our Profession? Defining The Competencies Self Assessment Tools For Growth What About Professional
More informationNATIONAL BOARD FOR CERTIFIED COUNSELORS AND AFFILIATES (NBCC) REQUEST FOR PROPOSALS (RFP)
NATIONAL BOARD FOR CERTIFIED COUNSELORS AND AFFILIATES (NBCC) REQUEST FOR PROPOSALS (RFP) FOR THE DEVELOPMENT, ADMINISTRATION, AND SCORING OF CERTIFICATION EXAMINATIONS RFP RESPONSE DATE: NOVEMBER 15,
More informationProposed International Standard on Auditing 315 (Revised)
Exposure Draft July 2018 Comments due: November 2, 2018 International Standard on Auditing Proposed International Standard on Auditing 315 (Revised) Identifying and Assessing the Risks of Material Misstatement
More informationBuild the Talent Pipeline: A Four Step Guide to Implementing Succession Planning. Karen N. Caruso, Ph.D.
Build the Talent Pipeline: A Four Step Guide to Implementing Succession Planning Karen N. Caruso, Ph.D. Build the Talent Pipeline: A Four Step Guide to Implementing Succession Planning Succession Planning
More informationEquating and Scaling for Examination Programs
Equating and Scaling for Examination Programs The process of scaling is used to report scores from equated examinations. When an examination is administered with multiple forms like the NBCOT OTC Examination,
More informationSeptember Promoting Regulatory Excellence. In this session we will: Getting It Right: Implementing Innovative Item Types.
Getting It Right: Implementing Innovative Item Types Moderator: Presenters: Jeffrey F. Kenney, AIA Executive Director, NCIDQ Lynn C. Webb, Ed. D. Testing Consultant Cynthia G. Parshall, Ph. D. Measurement
More informationHealth & Safety Management System (HSMS) Audit Program Standard
The following template may be used as a reference for developing your 2017 Safety Groups Advantage Program HSMS Audit Program Standard. The template incorporates the 2017 Advantage Program Employer Requirements
More informationJOB ANALYSIS GUIDE DEFENSE DEPARTMENT. Defense Civilian Personnel Advisory Service
JOB ANALYSIS GUIDE DEPARTMENT OF DEFENSE Defense Civilian Personnel Advisory Service PHASE 1 PHASE 2 PHASE 3 PHASE 4 Job Analysis is the foundation of human resources management and helps lay the foundation
More informationPromoting Stakeholder Acceptance of CBT 1. J. Patrick Jones Professional Examination Service
Promoting Stakeholder Acceptance of CBT 1 J. Patrick Jones Professional Examination Service Abstract This article describes the major elements of a communication plan for the implementation of a computer-based
More informationU.S. ARMY FUTURE FORCE SELECTION AND CLASSIFICATION RESEARCH
U.S. ARMY FUTURE FORCE SELECTION AND CLASSIFICATION RESEARCH Kimberly S. Owens, Richard R. Hoffman III U.S. Army Research Institute for the Behavioral and Social Sciences Arlington, VA, USA kimberly.owens@hqda.army.mil
More informationAmerican Council for Accredited Certification
American Council for Accredited Certification Council-certified Indoor Environmental Consultant (CIEC) and Council-certified Microbial Consultant (CMC) Summary of Job/Task Analyses Exam Blueprint Reports
More informationInnovative Item Types: How to Evaluate the Bang for the Buck. Brian Bontempo, Ph.D. Microsoft Corporation
Innovative Item Types: How to Evaluate the Bang for the Buck Brian Bontempo, Ph.D. Microsoft Corporation Continuum of Innovation Complexity of Innovation Non-Text Stimuli Polytomous Scoring Multiple Selected
More informationSummary Report. Canadian Assessment of Competence in Optometry
Canadian Assessment of Competence in Optometry Summary Report 2013-14 Our vision: be the recognized leader in competence assessments for optometry in Canada. MESSAGE FROM OUR C.E.O. ASSESSING COMPETENCY
More informationWhat to You Need to Know When Limited Production Certification is Your Hazardous Locations Compliance Strategy
What to You Need to Know When Intertek 70 Codman Hill Road Boxborough, MA 01719 www.intertek.com/hazloc 1-800-WORLDLAB icenter@intertek.com Introduction Complying with safety standards and requirements
More informationSize Matters. Growing a large item bank. Addressing Item Bank Deficits Rapidly and Effectively
Addressing Item Bank Deficits Rapidly and Effectively Dr. Phil Dickison, NCSBN Jason Schwartz, Pearson VUE Size Matters Advantages of large, robust item bank Flexibility in filling blueprint Breadth and
More informationAS9100 Series Aviation, Space and Defense. Product Guide
AS9100 Series Aviation, Space and Defense Product Guide What is AS9100 Aerospace Quality Management? The Aerospace Standards are a series of standards based on ISO 9001 and developed by the International
More informationSchool Psychology PhD
Unit Assessment Plan Guidelines Office of the Provost and Academic Affairs School Psychology PhD Learning Goals and Objectives Develop Competency in: (1) Research (2) Ethical and Legal Standards (3) Individual
More informationBeyond Academic Credentials: A 360-degree look at the Recognition of Prior Learning
Beyond Academic Credentials: A 360-degree look at the Recognition of Prior Learning Creating the New Normal : People, Tools, Process and Quality Assurance CAPLA: 2011 Fall Focus Workshop Contributors Christine
More informationA Comparison of Subject Matter Experts Perceptions and Job Analysis Surveys
A peer-reviewed electronic journal. Copyright is retained by the first or sole author, who grants right of first publication to Practical Assessment, Research & Evaluation. Permission is granted to distribute
More informationCertified Human Resources Professional (CHRP) Competency Framework
Certified Human Resources Professional (CHRP) Competency Framework 11.15 Table of Contents About the CHRP... 3 Application of the Competency Framework... 3 Path to Obtain the CHRP... 4 Maintaining the
More informationSAP EHS Regulatory Documentation OnDemand
SAP Brief SAP s for Sustainability SAP EHS Regulatory Documentation OnDemand Objectives SAP EHS Regulatory Documentation OnDemand Improve global compliance while reducing costs and risks Improve global
More informationIdentifying Training Needs and Evaluating Training. Contents are subject to change. For the latest updates visit
Identifying Training Needs and Evaluating Training Page 1 of 11 Why Attend A successful training function relies heavily on the proper identification of training needs and the evaluation of training. This
More informationGlossary of Standardized Testing Terms https://www.ets.org/understanding_testing/glossary/
Glossary of Standardized Testing Terms https://www.ets.org/understanding_testing/glossary/ a parameter In item response theory (IRT), the a parameter is a number that indicates the discrimination of a
More informationRaven's Advanced Progressive Matrices (APM)
Raven's Advanced Progressive Matrices (APM) Development 888-298-6227 TalentLens.com Copyright 2007 NCS Pearson, Inc. All rights reserved. Copyright 2007 by NCS Pearson, Inc. All rights reserved. No part
More informationEffects of Selected Multi-Stage Test Design Alternatives on Credentialing Examination Outcomes 1,2. April L. Zenisky and Ronald K.
Effects of Selected Multi-Stage Test Design Alternatives on Credentialing Examination Outcomes 1,2 April L. Zenisky and Ronald K. Hambleton University of Massachusetts Amherst March 29, 2004 1 Paper presented
More informationSTAAR-Like Quality Starts with Reliability
STAAR-Like Quality Starts with Reliability Quality Educational Research Our mission is to provide a comprehensive independent researchbased resource of easily accessible and interpretable data for policy
More informationSERVICE MANAGEMENT 03/07/2012. Professionalism, CPD (Continual Professional Development) & Mentoring
SERVICE MANAGEMENT Professionalism, CPD (Continual Professional Development) & Mentoring @MatthewKBurrows matthew.burrows@bsmimpact.com Matthew Burrows July 2012 1 Matthew Burrows FSM FRSA FISM MIC PMP
More informationCLEAR Exam Review. Volume XXVII, Number 2 Winter A Journal
CLEAR Exam Review Volume XXVII, Number 2 Winter 2017-18 A Journal CLEAR Exam Review is a journal, published twice a year, reviewing issues affecting testing and credentialing. CER is published by the Council
More informationALTE Quality Assurance Checklists. Unit 1. Test Construction
ALTE Quality Assurance Checklists Unit 1 Test Construction Name(s) of people completing this checklist: Which examination are the checklists being completed for? At which ALTE Level is the examination
More informationADMINISTRATION OF QUALITY ASSURANCE PROCESSES
ADMINISTRATION OF QUALITY ASSURANCE PROCESSES The organizational arrangements procedures outlined in this chapter have been found to be effective in higher education institutions in many parts of the world.
More informationHow Differential Item Functioning Analysis (DIF) Can Increase the Fairness and Accuracy of Your Assessments
How Differential Item Functioning Analysis (DIF) Can Increase the Fairness and Accuracy of Your Assessments Nikki Eatchel, SVP of Assessment David Grinham, SVP International Assessment Solutions Scantron
More informationAn Introduction to the Operational Best Practices for Statewide Large-Scale Assessment Programs (2013)
An Introduction to the Operational Best Practices for Statewide Large-Scale Assessment Programs (2013) The 2013 Edition of the Operational Best Practices for Statewide Large-Scale Assessment Programs (
More informationModel Curriculum Aerospace Design Testing Engineer Mechanical Systems
Model Curriculum Aerospace Design Testing Engineer Mechanical Systems SECTOR: AEROSPACE AND AVIATION SUB-SECTOR: DESIGN AND DEVELOPMENT OCCUPATION: AEROSPACE TESTING, VERIFICATION AND VALIDATION REF ID:
More informationCORROSION MANAGEMENT MATURITY MODEL
CORROSION MANAGEMENT MATURITY MODEL CMMM Model Definition AUTHOR Jeff Varney Executive Director APQC Page 1 of 35 TABLE OF CONTENTS OVERVIEW... 5 I. INTRODUCTION... 6 1.1 The Need... 6 1.2 The Corrosion
More informationPsychometrics 101: An Overview of Fundamental Psychometric Principles A S S E S S M E N T, E D U C A T I O N, A N D R E S E A R C H E X P E R T S
Psychometrics 101: An Overview of Fundamental Psychometric Principles A S S E S S M E N T, E D U C A T I O N, A N D R E S E A R C H E X P E R T S Presenters Manny Straehle Liberty Munson About Your Presenter
More informationCFS a key ingredient to your success...
CFS a key ingredient to your success... Certification is a distinction in many professions including food science. Obtaining the Certified Food Scientist (CFS) credential is a demonstration of your professional
More informationLorna Martin, PhD, CCC
Using Technology to Advance Large-Scale Competency-based Assessments A Study of the Canadian Standard Entry-to-Practice Counselling Competency Assessment Lorna Martin, PhD, CCC Canadian Standard Entry-to-Practice
More informationA Proposed Definition of Professional Accountant
A Proposed Definition of Professional Accountant A Staff Paper Prepared for Consultation January 2011 Table of Contents I. Introduction 3 II. Objectives 3 III. Methodology for the Development of this Paper
More informationGOOD PRACTICES IN FAMILY POLICY DEVELOPMENT, IMPLEMENTATION AND EVALUATION. Mihaela Robila, Ph.D., CFLE
1 United Nations Department of Economic and Social Affairs (UNDESA) Division for Social Policy and Development (DSPD) United Nations Expert Group Meeting Good Practices in Family Policy Making New York,
More information7 Statistical characteristics of the test
7 Statistical characteristics of the test Two key qualities of an exam are validity and reliability. Validity relates to the usefulness of a test for a purpose: does it enable well-founded inferences about
More informationDemonstrate HR CERTIFICATION INSTITUTE RELEVANCE COMPETENCE EXPERIENCE CREDIBILITY & DEDICATION WITH PROFESSIONAL HR CREDENTIALS
HRCI CERTIFICATIONS IN RESOURCES PHR PROFESSIONAL IN RESOURCES SPHR SENIOR PROFESSIONAL IN RESOURCES GPHR GLOBAL PROFESSIONAL IN RESOURCES HRBP RESOURCE BUSINESS PROFESSIONAL HRMP RESOURCE MANAGEMENT PROFESSIONAL
More informationConcordia University Portland College of Education Professional Education Plan (PEP) Professional Administrator Certificate
Concordia University Portland College of Education Professional Education Plan (PEP) Professional Administrator Certificate High-quality, informed district-level leadership for our public school districts
More informationENERGY PERFORMANCE PROTOCOL QUALITY ASSURANCE SPECIFICATION
ENERGY PERFORMANCE PROTOCOL QUALITY ASSURANCE SPECIFICATION Version 1.0 April 2015 Table of Contents 1.0 INVESTOR CONFIDENCE PROJECT 1.1 ENERGY EFFICIENCY PERFORMANCE QUALITY ASSURANCE SPECIFICATION 1.2
More informationEssential Guidance and Key Developments in a New Era of Testing. #TestStandards September 12, 2014 Washington, DC
Essential Guidance and Key Developments in a New Era of Testing www.aera.net/standards #TestStandards September 12, 2014 Washington, DC Standards for Educational and Psychological Testing: Historical Notes
More informationAgenda. Background: Situational Judgement Tests & Exercises. Beyond Job Knowledge: Assessing Situational Judgment for Professional Development
Beyond Job Knowledge: Assessing Situational Judgment for Professional Development and Credentialing Agenda Background o Situational Judgment Tests & Exercises (John Weiner) Applications o Licensing (Cyrus
More informationMembership. Services and Benefits. 10 Paragon Drive, Suite 1 Montvale, NJ (800) (201)
Membership Services and Benefits 10 Paragon Drive, Suite 1 Montvale, NJ 07645 (800) 638-4427 (201) 573-9000 www.imanet.org Welcome IMA (Institute of Management Accountants) is the association of accountants
More informationTechnical Report: October 2018 CKE 2. Human Resources Professionals Association
Technical Report: October 2018 CKE 2 Human Resources Professionals Association 23 November 2018 Contents Executive Summary... 5 Administration... 6 Form Setting... 6 Testing Window... 7 Analysis... 8 Data
More informationCertification Bootcamp Building Program Integrity. Andrew Falcone Applied Measurement Professionals, Inc. January 21, 2016 Denver, Colorado
Certification Bootcamp Building Program Integrity Andrew Falcone Applied Measurement Professionals, Inc. January 21, 2016 Denver, Colorado Measurement Overview Certification Examination Development Procedures
More information