Computer based Simulation Technology as part of the AC and DC

Transcription

1 Agenda Computer based Simulation Technology as part of the AC and DC a global South African perspective 1 Introduction Assessment Centers Evolution First concept development of the EAC Custom made CBST Development Methodology Research Results Value of Technology Implications for Assessment Practitioners Demo 2 About us Global managementassessment, -training and development organization providing Assessment-, Training-Technologies through e-learning and Business Simulation Solutions and Continuous Education Asia Mentorship 3 4 1

2 Geographical focus Botswana Namibia Lesotho Netherlands Switzerland Nigeria Sweden South Africa Zimbabwe Vietnam Cambodia Sri Lanka Nepal Melbourne Singapore Bangalore Johannesburg Perth Singapore 5 Computer based Business Simulation Technology in Assessment Centers 6 History of using technology in Assessment Centers 7 8 2

3 Before Technology 9 HISTORICAL TIME LINE AND DEVELOPMENT EARLY DAYS Traditional Assessment Centers PROCESS Trained Observers and Administrators Measured only Behavioral competencies Conduct 5 exercise Case Study Group Exercise In basket One on One interview Presentation Measure behavior Evaluate behavior against NORM Integrate scores through consensus Write Feedback report Give individual Feedback with Individual Development Plan Discuss performance with Management/ direct manager Type writing pool ZX Spectrum DOS Lotus Notes Win Word Win 3.1 (Word Excel, etc) Accuvison Video Vignettes AC Evolution Traditional AC Behavior Video based Behavior Multiple choice Looking Glass Technology Behavior Behavior Organizational Multiple choice and observation SJT s Behavior BIG 5 Multiple choice

4 E Learning Cyber teams Technology/ paperless environments , Cell phones, Skype, Blogging, Facebook, Mixit, GPS Google and You tube Why not shape assessment- events and - experiences to fit this exciting world of Technology? 13 USE OF BUSINESS SIMULATION TECHNOLOGY AS A COMPUTERISED INBASKET Assessment Centers with Generic Leadership Business Simulations PROCESS Developed AC to measure both Behavioral and Organizational competencies Aligned generic Behavioral competencies with client specific job competencies and used generic Organizational process competencies as per simulation Pre Assessment Prior to a 18 month development program Technology enhanced AC s 14 Issues Definition of Assessment Centers Functionally Culture Behavioral Organizational Technical Evaluation of Behavior Based on multiple inputs Specially developed simulations Multiple trained observers and techniques Judgements about the behavior is made Judgements are then reported and discussed Agreement among assessors on behavior of candidate. MOSES

5 Definition Experiential Assessment a scientifically based approach to the identification of future managerial qualities important for the future managerial success at a particular level of management. Exercises are all linked within the computerized simulation but still continue to observe behavior in the behavioral exercises Your Functionality Example 1 It s Friday night, and you have already put in 45 hours this week. The owner asks you to work tomorrow night, promising that you will be given a large dinner party which should get you a big tip. Which of the following would you most likely say? A. I d rather not, thanks. B. Sure! C. I m kind of tired, but I ll do it if you can t find someone else. D. I ll be happy to do it if you can guarantee the tip 19 Example 2 Client ;.a contract worker as a ship bridge worker He : Hi. SILENCE You : Good Morning. How can I help you? He : I am Jan I want to apply for a personal loan. What will you do? Explain the procedure : Choose the right procedure 1 Need an ID 2 If banking with XXX BANK need bank statement 3 Ask for pay slip 4 Go in to system and check if client will good for the amount 5 If Not explain to him that it will not be bale to do 6 If ratings is good explain to him the various option of a loan a b c d. 1 2,3,4,5,6, 20 5

6 Trained two groups of Observers and Administrators within client company to Administrate and deliver the various EAC Trained Observers and Administrators within ET to Administrate and deliver the EAC Selected and earmarked a few that will have to be appointed into the roles of Observer/Administrator. Ring fenced those who could not Introduction to EAC Discussion Exercise Discussion Norm Capacity Building Discussion Scale Discussion Competencies Discussion Simulations Case Study Analyze report Classify Competencies Rate behavior Feedback given Do simulation Read through feedback Facilitate discussion Reflect on outcome and behavior Group Discussion Observe discussion Classify Competencies Rate behavior Feedback given One on One Read instructions Do One on One Classify Competencies Rate behavior Feedback given Integration Sim scores and Behavioral scores Integrate scoring Feedback report Performance Matrix Discussion with Management 21 Involvement from IT Computer rooms IT infrastructure Delivery methods Technically 22 Experiential Assessment Centers AC imbedded with all or various technologies Computerized Business Simulation automated report writing capabilities Rapid development With a proper LMS Web Enabled Low on Band width Easy to build No Programming needed Customizable for any client industry specific Should still be an AC Senior Managers Experiential Center Managers Experiential Center First line mangers Experiential Center Project Managers Experiential Center Sales Managers Experiential Center 23 SIM Development Methodology Computer based Simulation Technology assessment center measures of performance Balanced Scorecard Business analysis Learning Outcomes Formative Perspective Assessment Custodians Developed assessment Summative criteriaassessments learning topics if any research and investigation on topic Compiled scenarios theory and behind story line it Custodians write feedback, score each creation of the story line scenario against competencies validation to start multimedia compilation Quality Checks test and quality check Develop Multi Media Assemble 24 6

7 Energy Sector Financial Institution Insurance Industry Manufacturing Tire Industry

8 Get SME s and behavioral team to score and agree on each answer to each scenario Toughest part Locate Most Appropriate answer ID the PRIMARY competencies per answer ID the Secondary Competencies DO that per answer per scenario Scoring : 10 point scale +5 to - 5 All answer Primary competencies Secondary Competencies Tertiary Competencies If an answer is MOST Appropriate Competencies are scored highest +4, +5 If answer is in the middle Appropriate Competencies are scored towards the middle +2, +3 or -2, -3 If answer is LEAST Appropriate Competencies are scored lowest -4, Exercises Challenges On line exercises Off Line exercises Case Study In Basket Group Discussion One on One Interview Presentation Culture Functionally Behavioral Organizational Process Technical Legal

9 Cultural Functional KNOWING YOUR CHALLENGES Complete in your own words

10 Technical Server and back up Generator Intranet vs. Intranet Bandwidth Edge GPRS 3G HD LMS or HRIS SCORM Multimedia Flash Squeeze Legal Conducting AC and DC with CBST

11 ON LINE ASSESSMENT CENTER PROCESS FLOW Conducting the EAC Your TK Manager Scheduled with candidate to candidate Reminder Experiential Center same day same day next day if needed Manager completes Case Study Manager access On-line Candidate completes In Basket Manager s case study report to ET One hour same day 3 hours scheduled day Interactive exercises Two hours Database update ET compiles report Report ed to TK When done with simulation Next business day Within two business days Exercises CBST on line: Analysis Problem Project Plan and Charter -- Business Plan or -- Sales plan Exercises Integrated Simulated In-Basket Customized Simulations could include: Group Discussion One-on-One Client or Employee Interview Presentation Technical Processes Product knowledge

12 BEHAVIORAL Competencies ORGANISATIONAL Cluster A: Achievement Motivation Innovation Energy Process Skills Managerial Clusters & Competencies Cluster B: Conceptualizing Cluster C: Entrepreneurial & Commercial Thinking Cluster D: Business Skills Cluster E: Leadership Skills Conceptual Thinking Strategic Approach Financial Management Risk Management Global Orientation Decision Making & Business Acumen Client Focus Provision of Guidance Managing Diversity Interpersonal Sensitivity Social Responsibility trade activation methodology 45 Custer F: Managing Relationship and Building Networks Interpersonal Influence Presentation Skills 46 Rating Scale Rating Scale 1 Training and Development needed 2 Development needed 2.5 Development with potential 3 Rounding-off needed 4C Adequate (Needs counseling) 4 Adequate (Norm: effective management practice) 5 More than adequate

13 on Line scenarios Feedback

14 Feedback 53 Feedback 54 Feedback Feedback to Individual Feedback to Management on results Talent Management --- assess the performance of individual in the workplace

15 Per Exercise: Simulation Cluster: Leadership Skills Talent Solution Overall results Succession Planning Assessment Centre Reports OVERALL MATCH FIT WITH HIGH POTENTIAL Rating 5 Individual Development Plans Talent Management Performance Management Results FIT MODERATE MATCH WEAK MATCH Training Assessment Results Workplace Diversity and Skills Plan NOT FIT 1 Equity Plans

16 P ot e n t ia l R a n k in g High Talent Matrix Comparing Results Integrated Approach Recommendations Talent Management Potential Ranking Development of IDP s Limited Potential Solutions focused on Potential Innovation High Potential Leadership for business results Strategy Alignment and implementation E.G Client relations vs. Client care Low PERFORMANCE High Research CORRELATION STUDIES CORRELATION STUDIES Correlation between Behavior vs. Organizational Correlation with Organizational Scorecard Performance Correlation between Technology and Behavior Comparing results with Psychometric instruments Correlation Pre and Post Assessment Correlation between Technology : Psychometric results and overall outcome

17 Methodology Assessment (two days) Inclusive of all races, gender and ages Middle Management to Senior Management Ratings on all exercises On-Line Assessment Integration Integration of Behavioural Assessments Correlation with Psychometric Instruments Actual correlations Decision-making Business Acumen Analytical thinking Energy Interpersonal skills Inter Correlation between Technology and Behavior Observed Selection in Simulation N = Actual Correlation On Line Assessment Comparing results Ranking correlation (n=403) (p.05) Behavioral (Analysis, Group, One-on-One, Presentation) : On-Line r =.85 Case Study and On-Line : Total Assessment r =.89 Compare Data with OPQ (SHL instrument) previously done Divide competencies in three categories EXTREME - HIGH -MODERATE IMPACT ON ROLE SHL to also identify EXTREME - HIGH MODERATE- impact competencies for the role separate from us Compare scores on OPQ, SIM and Behavioral Apples with Apples

18 Agreement on competencies importance Inter correlation between Technology and Behavior Extreme job match competencies High job match competencies Moderate job match competencies correlation Decision-making Business Acumen Analytical thinking Energy Observed Selection in Simulation Interpersonal skills Correlation between Technology other psychometric results and overall outcome EAC OPQ Decision-making Business Acumen Analytical thinking Client Projects Energy Interpersonal skills

19 CLIENT EXPERIENCES SADEC, NEAD and ASIA specific regions strategy to appoint quality leaders in each country confirmation on already identified talent enabler to advance on management development programs right level with right competence Assessment assessment for respectively development and fit for purpose leadership pipeline identification enhance performance The key challenges they are faced with were; Spending money correctly Saving on development costs Focused on individual development as competencies were assessed The right people cap turnover Matching personal competencies with organizational needs 73 Percentile COKE OVERALL COUNTRY RANKINGS (N=213) Sri Lanka Nepal 60.0 Vietnam 40.0 Cambodia South Africa Achievement Conceptual Business Entrepreneurial Leading and M anaging M otivation Thinking Skills Commercial Working with Relations and People Building Networks Clusters 74 COKE : Leading and working with People Cluster : N= Percentile Provision of guidance Diversity Interpersonal Senstivity Competencies in cluster Social Responsibility Sri Lanka Nepal Vietnam Cambodia South Africa Average Minimum Same competency, different methods (expected to be the highest) Different competencies, same method (would reflect method bias) Different competencies, different methods (expected to be the lowest) Maximum,

20 South Africa Coke Persentage Potential Persentage of 4 N= Limited Potential Potential High Potential Plus Potential Potensial Scale Sri Lanka Nepal Vietnam Cambodia South Africa High COKE Talent Matrix Discussion Comparing Results CLIENT EXPERIENCES quality leaders to execute their strategy required real role clarity, that is the alignment of roles with the business right level with right competence Assessment Assessment for respectively development and fit for purpose experiential learning pipeline identification The key challenges many organizations and themselves were faced with were; Spending money correctly with pipeline promotions Saving on development costs Development is now focused Focused on individual development and competencies were assessed Utilizing skills according the competencies Reduce the costs of the wrong person on a job The right people cap turnover People have a choice Do I want this role or not? Getting to know my self and my key preferences Matching personal competencies with role competencies Low PERFORMANCE High

21 Service Advisor N = 403 BEHAVIOURAL COMPETENCIES PERFORMANCE MOTIVATION Correlation of the Behavioral and Organizational Competence Innovation Energy ORGANISATIONAL COMPETENCIES FINANCIAL PERSPECTIVE Financial Management of the branch Business Acumen (Branch Managers) Analytical Thinking Branch Resource Management DECISION-MAKING SKILLS Decision-making and Business Acumen Analytical Thinking (Branch Managers) Flexibility Utilisation & Development INTERNAL PROCESSES Bank policies and procedures Passion for service excellence LEADERSHIP SKILLS Task Structuring Decisiveness Mentoring Performance Management Sensitivity Interpersonal Influence LEARNING AND GROWTH Development of people Communication COMMUNICATION SKILLS Presentation Skills Self Development Decision-making 81 ADMIN SKILLS Planning, Organising and Strategising SALES PERSPECTIVE Branch Sales and Service Planning Sales Orientation in the branch Local Marketing Orientation 82 Service Advisor N =

22 Service Advisor N = 403 Service Advisor N = Component Var(person) Var(Competency) Var(Method) Var(person * Competency) Var(person * Method) Var(Competency * Method) Var(person * Competency * Method) Var(Error) Estimate Service Advisor N = 403 CLIENT EXPERIENCES World cup 2010 Project Relationship Manager's Role in New challenges in Broker market Position changed from Admin to Business partner right level with right competence Assessment enabler to advance on management development program confirmation on already identified talent assessment for respectively development and fit for purpose leadership pipeline identification enhance performance The key challenges they are faced with were; Spending money correctly during recruitment process Saving on development costs Focused on individual development as competencies were To match the different Relationship Manager with the broker segments and business partners

23 Insurance Industry (113) Insurance Industry (113) RATING SCALE SANTAM RELATIONSHIP MANAGER PERFORMANCE MATRIX N= Behavioral CLUSTERS Organisat ional Insurance Industry (113) Insurance Industry (113) SANTAM RELATIONSHIP MANAGERS PERFORMANCE ON SCORECARD PERSPECTIVES SANTAM RELATIONSHIP MANAGERS BEHAVIORAL PERFORMANCE Rating Scale Strategic Relationship CLIENT Broker Practice PERSPECTIVE Broker Development: Broker/ Practice Cross- Upselling, Unit Growth Rentention Profit Self Development People Development Leadership and Teamwork Marketing: Business Plan, Sales Plan, Sales Maintain Brokers FINANCIAL PERSPECTIVE RESOURCES PERSPECTIVE INTERNAL PERSPECTIVE Perspectives Growing Brokers Scale INNOVATION ENERGY PROCESS SKILLS CONCEPTUAL THINKING DECISION MAKING & BUSINESS STRATEGIC RELATIONSHIP CLIENT FOCUS RISK MANAGEMENT ACHIEVEMENT MOTIVATIONCONCEPTUALISATION ENTREPRENEURIAL & COMMERCIAL LEADING & WORKING WITH PEOPLE MANAGING RELATIONS & BUILDING NETWORKS GLOBAL ORIENTATION CHANGE MANAGING DIVERSITY SOCIAL RESPONSIBILITY INTERPERSONAL SENSITIVITY INTERPERSONAL INFLUENCE PRESENTATION 91 Behavioral Clusters 92 23

24 Simulations add value in the following manner: Improve a assessee s understanding Value of CBST in AC and DC Facilitate communication and debate Allow managers to prioritize Improve understanding of an industry s dynamics Simulations add value in the following manner: Uniqueness understanding of cause-and-effect relationships make more effective decisions Allow managers to complete a what-if analysis demonstrate leadership strengths

25 The Value The most significant value can be described in three words: experience through experience Critical success factors Critical success factors Commitment and involvement Consequence management Coaching - willingness and involvement Sustainability of development Customized learning solutions that address business and individual needs Relevance of learning Strategic alignment 99 Implication for us Project Management Skills Right project and project right Stakeholders management Scope creeping Q T Budget vs. competitiveness Technology Skills E learning Managing IT people Technology changes Virtual hosting Research capabilities Validity Reliability country and group norms Process driven vs. better business results Results between organizational scorecards and AC outcomes

26 Debate Questions In conclusion Are trade activation here to stay? Trade activation Organizational- vs. Behavioral- Competence measured Reliability of 95%? Ethical issues of technology assessment centers high performance teams Process measurements vs. results measurement Potential vs. Competence? Fairness vs. predicative validity and Team competence Technology helped us to simulates real world issues and global practices and processes Aligning business workflow and process across the organization and countries Establish best practices and learning Testing of these practices and knowledge Sharing of knowledge between member countries Teaches Business Acumen Create awareness of Big picture Pre course attendance Determine the growth potential Targeted development (individually) Determine a risk profile of people Assessment as well as learning through experience Supports the new organizational culture Enable change management Competitive advantage cont.. Demo