Going the Extra Mile to Enhance Resume Reviews

Transcription

1 Going the Extra Mile to Enhance Resume Reviews Midwest Association of Colleges and Employers July 22, 2014 Presented by: Julia Panke Makela, Hongling Sun, & Gaeun So The Career Center University of Illinois at Urbana-Champaign

2 Session Goals To provide hands-on experience engaging in assessment to improve career programs and services Including invitations to Interpret data from actual assessment projects Apply a rubric to evaluate a sample resume Evaluate the strengths and weaknesses of different assessment approaches Discuss how assessment tools can be transitioned into resources for practice Introduce resources and networks to expand upon our conversations Oh, and we are using the familiar service of resume reviews as a frame for today s discussion.

3 The Frame: Resume Review The Career Center 2,940 resume reviews were conducted in FY Resume reviews are offered on a drop-in basis each afternoon in our Career Resource Center, as well as evening and weekend hours in a few satellite locations around campus Reviews are advertised to last approximately 15 minutes Reviews are conducted primarily by trained undergraduate peer paraprofessionals (92%), but also may be conducted by graduate student paraprofessionals or career counselors

4 Early Assessment Approach: The Annual Survey

5 History of Resume Review Service Assessment Time of Assessment: Every Fall Semester from 2004 to 2010 Self reported student satisfaction of resume service surveys Self-ratings learning outcomes including resume writing confidence & ability Assessment Participation from 20o4 to 2010 Year of Assessment (Fall) Users of Resume Review Service Survey Participated Students Response Rate 25% 36% 40% 30% 32% 27% 34%

6 History of Resume Review Service Assessment Comparison of Self-reported Satisfaction across 2004 to 2010 What might you learn from this data? If you received this feedback on a survey, what actions might you take?

7 History of Resume Review Service Assessment Comparison of Selected Outcomes across 2004 to 2010 What might you learn from this data? If you received this feedback on a survey, what actions might you take?

8 A Moment to Reflect & Share From Our Previous Assessment of Resume Review Service ( ).. What do you think of this type of assessment? What catches you? What would you do with those results? What challenges do you see? Two Example Challenges We Saw were. Findings rarely changed from one year to the next we were not learning new information, and needed a new strategy to expand our thinking Recognized limitation of self-report data: wanted to expand understanding of student outcomes by examining demonstrated behaviors and abilities

9 Rethinking our Assessment Approach: New Questions Emerge

10 Purpose / Goals of the Project Go beyond exploring self-reported satisfaction and outcomes Assess students demonstrated abilities to effectively communicate accomplishments within a resume format result of participating in resume services at The Career Center at Illinois Question for this Study: Is there a significant difference in resume quality for students who: a) participate in only resume review services ( Resume Only group) b) participate in resume review services and other programs and services ( Resume Plus group) c) do not participate in resume review services or other programs and services (Control group) But, how do we define and measure resume quality?

11 Methods 1. Develop Resume Review Rubric 1. Assessment Team designed a rubric for evaluating resumes based on Related Literature Professional association models Content from our own resume workshops and services 2. The rubric was test by a 2008 pilot iteration of this study 3. Systematically gathered input from professionals to apply and strengthen rubric 4. The final rubric contained eight sub-scales: 30-Second Scan Name/Contact Information Additional Sections General Format Education Positive/Professional Tone Spelling & Grammar Primary Experience

12 Methods 2. Select & Train Reviewers 1. Reviewers were career counselors & graduate assistants who had been reviewing resumes at The Career Center for at least three months 2. All reviewers were trained for scoring resumes with the resume rubric Used randomly retrieved 15 student resumes Discussed the rubric Individually rated resumes 2 resumes at a time, in an iterative process Discussed rankings until a strong group consensus were achieved - Generated familiarity with the resume review rubric & calibrated ratings across group

13 Your Turn! Let s try applying the rubric to a sample resume

14 Methods 3. Our Study Design Randomly selected 60 resumes from undergraduate students in the College of Liberal Arts and Sciences 20 per each Resume Only, Resume Plus, & Control group Trained a team of reviewers who conducted blind reviews of the 60 resumes using the resume review rubric. Each resume was reviewed by 3 raters. (91% rater agreement) Employed one-way ANOVA with Dunnett s test of multiple comparisons to explore significant differences

15 (* p <.05, *** p <.001) Key Findings: Overall Resume Quality Those students who use resume review services produce significantly higher quality resumes than those who do not.

16 Key Findings Sub-scales made different contributions to observed differences in resume quality Scale: 1=Incomplete, 2=Developing, 3=Proficient, 4=Professional Component Full Sample (60) Resume PLUS (20) Resume ONLY (20) No TCC (20) Name / Contact Information Education * Spelling / Grammar Primary Experience *** 3.2 ** 2.5 Additional Sections ** 3.2 ** 2.5 Positive / Professional Tone ** Second Scan *** 2.9 ** 2.0 General Format * 2.4 (* p <.05, ** p <.01 *** p <.001)

17 Key Findings Also gained insights on what areas of resume writing students may find particularly challenging Scale: 1=Incomplete, 2=Developing, 3=Proficient, 4=Professional Component Full Sample (60) Resume PLUS (20) Resume ONLY (20) No TCC (20) Name / Contact Information Education * Spelling / Grammar Primary Experience *** 3.2 ** 2.5 Additional Sections ** 3.2 ** 2.5 Positive / Professional Tone ** Second Scan *** 2.9 ** 2.0 General Format * 2.4 (* p <.05, ** p <.01 *** p <.001)

18 Key Findings Scale: 1=Incomplete, 2=Developing, 3=Proficient, 4=Professional What might you learn from this data? If you received this feedback from an assessment project, what actions might you take? Component Full Sample (60) Resume PLUS (20) Resume ONLY (20) No TCC (20) Name / Contact Information Education * Spelling / Grammar Primary Experience *** 3.2 ** 2.5 Additional Sections ** 3.2 ** 2.5 Positive / Professional Tone ** Second Scan *** 2.9 ** 2.0 General Format * 2.4 (* p <.05, ** p <.01 *** p <.001)

19 A Few of our Own Interpretations and Next Steps Recognize the good work that the resume review and counseling teams have done in working with students Consider areas for improvement in staff training, resume workshops, resume reviews, and (where appropriate) counseling appointments Focus on room for growth in several resume component areas, such as: general formatting, additional experience sections, positive/professional tone, 30-second scan. Training staff to make referrals to counseling appointments, particularly for students who are having difficulty developing a cohesive, logical flow to the variety of experiences on their resumes Adapt information learned in assessment project to daily practice

20 Use of Assessment Results Shared report and findings with resume reviewers and key stakeholders to: Celebrate success! Encourage continued improvements e.g., enhanced training on when and how to make referrals to additional services Adapted the assessment rubric into a training resource Used to train our own undergraduate peer paraprofessional staff Shared with academic departments to facilitate peer resume reviews in class and more!

21 For More Information Contact Us Julia Panke Makela, Associate Director for Assessment and Research, Hongling Sun, Graduate Research Assistant, Gaeun Seo, Graduate Research Assistant, Join the Conversation: Assessment & Research in Career Services (ARCS) Listserv The ARCS listserv, established in June 2014, facilitates conversations among career services professionals who are engaged in assessment and research within practice environments. We aim to share information and build collaborations to enhance career development programs and services, as well as to further the field of career development. Provide your address to join, or your request to