Metrics that Matter Measurement Program Report Catalog

Transcription

1 Metrics that Matter Measurement Program Report Catalog

2 Contents Reports for Single- and Multi-Rater Programs... 3 Measurement Program Data Download... 3 Measurement Program Respondent Comments... 3 Measurement Program Evaluation Retrieval... 4 Measurement Program Quick Question... 5 Measurement Program Crosstab... 6 Measurement Program Report Card... 6 Measurement Program Comparative Performance... 7 Measurement Program Variance... 7 Measurement Program Trend Average... 8 Measurement Program Comparison Score Card... 9 Reports for Multi-Rater Programs Measurement Program Multi-Rater Summary Leader Alignment Score Card Multi-Rater Participant Report Reports for Single-Rater Programs Employee Survey Department Report Employee Survey Overall Report... 35

3 Reports for Single- and Multi-Rater Programs Measurement Program Data Download The Measurement Program Data Download tool allows you to request a raw data file from MTM containing your Measurement Program data. Having access to raw data gives you the ability to perform offline analysis of your data that MTM does not provide. Your request will be generated from the MTM Database and packaged into a compressed (.zip) file. The.zip file will contain two files, a characterdelimited (.csv) file and an.xml file. The (.csv) file is then imported into Excel or another data analysis tool for your own analysis. The raw data collected from this will provide a record for each answer included in your data extract. Data extracts can include any of the following combinations of data types: quantitative/likert answers (numeric scale-answer questions); non-likert answers (multi-choice and multi-select answer questions); comments/text (qualitative answer questions), demographic answer data and test answer data (may be combined with demographic data), and test data. This tool also allows for converting data scales to match your user preferences or remain in the same scale the students were surveyed in. Most commonly, organizations use the data download tool for archiving data outside of MTM, accessing large sets of comments for easier searching/sorting, and for generating offline reports that are not available through MTM. Measurement Program Respondent Comments The Measurement Program Respondent Comments report allows you to view comments submitted by respondents. Having access to qualitative feedback in the measurement program you gain insight on respondent reactions. This report is easily run allowing you to see comments for recently completed measurement program(s), and can be used as a reference when trying to isolate exceptions in your reports. You will have the ability to view a list of qualitative questions and their responses, with respondent identifiers ( address or student ID) in parenthesis following his or her comments.

4 Measurement Program Evaluation Retrieval The Measurement Program Evaluation Retrieval report allows you to view individual real-time forms completed by respondents for Measurement Program(s). Having access to respondent forms provides you the ability to see individual respondent reaction. This can help you see performance at a very low level, and can be used as a reference when trying to isolate exceptions in your reports. The Measurement Program Evaluation Retrieval tool provides access to data immediately after it is entered into MTM, making it a real-time report. You can view a list of respondents by an identifier ( address or respondent ID) with their overall score, and from there click on an identifier to see the student's evaluation in electronic form. You can also view all evaluations available in a single pop-up window to access all students at once or print the evaluations with a single click. You may access submitted forms for a single Measurement Program or multiple programs.

5 Measurement Program Quick Question The Measurement Program Quick Question report displays questions and answer averages including the distribution (frequency) of the responses. The report will first show a summary section of the various question categories included in the Measurement Program (s) selected including the number of all or selected Participants and Raters who submitted at least one response for the question category, the distribution of answers for quantitative questions across the reporting scale, and the question category average. Each subsequent section will break down the question results for each question category containing data in a similar fashion to the Summary section. The question category sections will include results of multiple choice (non-likert) questions after the quantitative questions are displayed. You may also view comments for all classes included in the report at the bottom of the report page. The output of a Measurement Program Quick Question query can provide significant value as an aggregated report when generated to roll up data for multiple Measurement Programs into a single report. You have the ability to view results for individual Measurement Program results or Multiple Measurement Programs in a single report by question category.

6 Measurement Program Crosstab The Measurement Program Crosstab report compares two MTM data attributes using Likert (numeric scale) averages in a visually appealing, color-coded table. You can use this report to pivot and compare cross-sections of data, including attributes specific to 360/Multi-Rater and Employee Survey programs: rater type; program; and participant. The report also includes attributes including question categories, Likert questions, demographics, and multiple choice questions. Measurement Program Report Card Measurement Program Report Card shows overall scores by question category and question for the measurement program you provide. Measurement Program Report Cards are very powerful because they allow you to compare your organization's performance against a benchmark which can be adjusted. You can also customize the benchmark to match your custom filter selections or not, improving the reporting power of this tool.

7 Measurement Program Comparative Performance The Measurement Program Comparative Performance report provides overall measurement program results (quantitative averages) organized by question categories. These reports have the ability to roll up data for multiple measurement programs and include a benchmark indicator to make it visually simple to determine which line items in your report are performing above and below your average. Measurement Program Variance The Measurement Program Variance report shows student quantitative responses distributed among high/average/low score ranges. This provides a different perspective than Comparative Performance Reports in that it doesn't focus on the end result (average score), but rather how the end result is comprised. The Measurement Program Variance report can provide insight towards what question category scored exceptional, average, and below average levels. Using this report you can also drill down to through the measurement program type, question category, and question levels to understand what lower-level distributions are making up your higher-level frequencies.

8 Measurement Program Trend Average The Measurement Program Trend Average reports on the trend of a specific Measurement Program for a given period of time broken out in different frequencies. Running a Measurement Program Trend Average report at the most basic level will produce a trend line based on submissions for that program.

9 Measurement Program Comparison Score Card The Measurement Program Comparison Score Card report gives insight to how the performance of two different participant populations compare against one another. The report compares the scores of two independently configured groups (groups may contain anywhere from one to several measurement programs) and calculates the gap between the two. If the two groups are within a positive or negative variance agreed upon by the business, the report highlights those areas by color coding the values red to show at-a-glace where a difference outside of the set limits occurred. The legend below the table identifies what each column represents and how the values are calculated. The Appendix will identify what each group represents (measurement program(s), participant(s), rater types, and demographics). Armed with this information, management can take actions immediately to bring two groups into a more desirable alignment. The report focuses on the gap analysis between the second and the first group, so both groups are required to generate the report.

10 Reports for Multi-Rater Programs Measurement Program Multi-Rater Summary Measurement Program Multi-Rater Summary is a report used to compare data collected on Multi-Rater measurement programs. The report provides the ability to compare responses by rater types at each category level and further more at each question level. The advantage of using this report is it can provide you in a single report with the overall average for each question category as well as the overall rating for each rater type. It can also provide you with an overall average for each question within the question category. This report is used for any Multi-Rater type measurement program such as 360 competency Assessments, Performance Appraisals, or any measurement program that collects data around an individual. All raters must use the same evaluation form so the questions are comparable.

11 Leader Alignment Score Card The Leader Alignment Score Card is a report that provides insight to determine the difference in average score between the leader and a group of associates. The report compares the scores of one leader against an aggregate score of the group and calculates the gap analysis. If the leader and the group are within a positive or negative variance agreed upon by the business, the report highlights those areas by color coding the values red to show an at-a-glance view of where a difference or inconsistency occurred. Armed with this information, management can take actions to help bring the leader and group both into alignment. The report focuses on the gap analysis, thus both leader and group data is required to run the report.

12 Multi-Rater Participant Report The Measurement Program Multi-Rater Participant Report shows participants' quantitative and qualitative responses delivered in a PDF or Word document and analyzes the data broken down by standard categories. This report aims to give the participant an idea of how their scores break down, where additional help is needed, and were they perform well. The document adds value to the report by contextualizing the raw data. The Measurement Program Multi-Rater Participant Report is generated for each participant separately and it does not aggregate other participants' scores. The report can also be generated in an Aggregate form where the scores for a group of participants are combined and presented in the same format for cohort-based reports.

13

14

15

16

17

18

19 Qualitative Feedback The qualitative feedback section provides all qualitative feedback surrounding specific competencies and/or behaviors. Comments that appear 2-3 times are considered a strength or opportunity for improvement, and should be considered when developing your Individual Development Plan (IDP).

20

21

22

23

24

25

26

27

28

29 Reports for Single-Rater Programs Employee Survey Department Report The Employee Survey Department Report focuses on what matters most to managers so they can make daily decisions that affect their employee's motivation, commitment, and engagement, all of which impact company performance. This report will help answer the following questions: How does my team compare to my department? How does my team compare to the overall organization? The report's cover page displays the name of the report and the name of the person the report for generated for (A bundle of reports will be delivered to the address specified. Inside the report bundle will be a report for each participant selected from the query screen). Figure 1 - Employee Survey Department Report Cover Page

30 Department Results This report displays results for a specific department and compares the department to a selected demographic group, and then entire survey population. Figure 2 - Employee Survey Department Report Response Rates Response Rate: This table shows the number of employees that had an opportunity to complete the survey, the overall number who responded, and the response rate for each group. The first column (overall) represents the entire survey population. The second column represents the employees within the same demographic group as the manager for the selected demographic question. The third column represents all employees that report up through the manager listed.

31 Categories Based on the survey results, this table displays the average scores for each category by overall, employees with the same demographic group as the manager and all employees that report up through the manger. This table displays all category and question scores for each category by overall, employees with the same demographic group as the manager and all employees that report up through the manger. This table can be used for deeper analysis of the survey results.

32 Figure 3 - Employee Survey Department Report Category Score Summary and Question Score Summary

33 Multi-Select Question Summary When multi-select questions are used in a survey, a table would be provided for each question showing the percentage of employees in each group that selected each option. Figure 4 - Employee Survey Department Report Multi-Select Questions Summary

34 Qualitative Summary This section compiles all comments provided by the direct reports of this manager for each qualitative question on the survey. Figure 5 - Employee Survey Department Report Qualitative Responses

35 Employee Survey Overall Report The Employee Survey Overall Report allows organizations to collect vital feedback from throughout the employee lifecycle to improve the efficiency and effectiveness of talent processes. The report is designed to answer three questions when reviewing feedback gathered from an organization-wide employee survey: Where do problems/gaps exist? Is this a problem/gap companywide? Is this problem/gap isolated to a specific demographic? The Employee Survey Overall Report is a full-featured narrative report available in Word or PDF formats designed to provide comprehensive, client-ready summaries of data in tables to facilitate analysis and comparison. The report can be customized to limit the number of questions and categories that appear in the top and bottom ratings as well as limit which questions and categories appear in the top and bottom ratings section by the number of responses provided if so desired. The report can be run for one program at a time. The report's cover page displays the name of the report and who the report was prepared for (if this field had been populated on the query screen). Figure 1 - Employee Survey Overall Report Cover Page

36 Overall Results The Overall Results section displays results for the overall survey including: Response Rate, Highest/ Lowest Rated Categories, Highest/ Lowest Rated Questions and Category and Question Score Summary. Figure 2 - Employee Survey Overall Report Response Rate Summary Response Rate: The Response Rate section provides the user with an overall snapshot of: Employees - the number of participants enrolled to take the survey Responses - the number of completed surveys Response Rate - calculated by dividing the number of responses by the number of employees

37 Highest and Lowest Rated Categories: This section provides information on the highest and lowest rated categories. The highest rated categories should be considered as success stories and best practices is those areas should be used throughout the organization. Low rated categories show where there is room for improvement. Figure 3 - Employee Survey Overall Report High/Low Categories

38 Highest and Lowest Rated Questions: This section provides information on the highest and lowest rated questions. This more detailed section can be used to understand a company's strengths and where there may be need for improvement. Figure 4 - Employee Survey Overall Report High/Low Questions

39 Category and Question Score Summary: The Category and Question Score Summary shows all category and question scores accompanied by the number of responses for each. This section can be used for deeper analysis of the survey overall. Figure 5 - Employee Survey Overall Report Category and Question Score Summary

40 Demographic Results The Demographic Results section provides analysis across demographics that can help identify trends that can be addressed within specific groups within your survey population. To address the needs of a specific demographic, look for demographic groups that score lower from most questions on the survey. You can also look for questions that score lower for all groups to address more systemic needs across the organization. Figure 6 - Employee Survey Overall Report Demographic Results

41 Job Function (this title will vary based off of the demographic selected on the query screen) This section of the report displays data by the demographic item selected. This first table displays the distribution of survey respondents by demographic and the response rate based on the number respondents from whom a demographic value was supplied. Note: The ability to differentiate the count of Employees from the count of responses by demographic option is only possible when demographics are imported with the employee's enrollment record via integration. If the enrollments are completed by hand, the Employee and Responses columns will be the same because only the employees responding to the survey are answering the demographic question. This table displays the average score per category and question for each demographic response option. Highlighting is automatically applied when scores vary by more than 15% from the average. This section can be used for deeper analysis of the survey overall.

42 Figure 7 - Employee Survey Overall Report Category and Question Score Summary by Demographic