CTSA COMMON METRICS SCORECARD GUIDANCE This document provides instructions on entering data values and narrative text into the CTSA Common Metrics

Transcription

1 CTSA COMMON METRICS SCORECARD GUIDANCE This document provides instructions on entering data values and narrative text into the CTSA Common Metrics Scorecard. It also includes steps for creating additional underlying strategies and corresponding performance measures.

2 TABLE OF CONTENTS Key terms and concepts: Common Metrics Initiative... 2 Key terms and concepts: CTSA Common Metrics Scorecard... 3 The Common Metrics Scorecard... 4 Entering Data Values For Common Metrics Performance Measures... 5 Entering the Turn the Curve Narratives For a Common Metric s Performance Measure... 6 Operational Guidelines and Data Collection Notes... 6 Document Data Collection Issues... 6 Download the Operational Guidelines... 6 Entering and Tracking Strategies... 7 Set up a New Scorecard... 7 Adding Strategies and Performance Measures... 7 CTSA Common Metrics Scorecard Guidance 1

3 KEY TERMS AND CONCEPTS: COMMON METRICS INITIATIVE 1. Results-Based Accountability (RBA) A strategic management framework that utilizes data and rigorous analysis to maximize the impact of programs and services for their customers and to improve conditions of well-being, such as health, for communities Performance Measures Data that can be collected and reported over time. In the Results-Based Accountability framework, there are three kinds of performance measures: a. How much did we do? b. How well did we do it? c. Is anyone better off? 3. CTSA Program Common Metric A performance measure or set of performance measures designed to provide insight on the impact of the CTSA Program within an area of activity of a hub or the network 4. Common Metric Operational Guidelines Technical specifications for the data for the performance measure or set of performance measures that constitute a Common Metric 5. Turn-the-Curve Thinking and Turn-the-Curve Plans A step-by-step decision making process focused on improving the trend for a performance measure 2 and designed to garner, with rigor and collaboration, the best thinking of stakeholders 6. Strategies Activities (which can range in scope from the creation of a new entity e.g., a department, core, or educational degree to more modest initiatives but will involve enough sustained effort to merit/require their own performance measures) that will be implemented to improve the trend to turn the curve for a performance measure 7. Actions Discrete activities (that do not merit/require their own performance measures) that will be implemented to improve the trend to turn the curve for a performance measure 1 RBA was developed by Mark Friedman, author of Trying Hard is Not Good Enough: How to Produce Measurable Improvements for Customers and Communities (Trafford 2005) 2 In RBA, in addition to management with performance measures, turn-the-curve thinking is applied to the deliberations of collaborations organized to improve conditions of well being for whole populations in neighborhoods, cities, states, nations and international communities. CTSA Common Metrics Scorecard Guidance 2

4 KEY TERMS AND CONCEPTS: CTSA COMMON METRICS SCORECARD Vers: 8/18/16 1. Scorecard A set of performance measures and/or strategies associated with a hub, one or more programs or areas of activity within a hub, or a strategy or set of strategies being implemented by a hub to turn the curve of a performance measure. 2. Common Metrics Scorecard This scorecard lists each of the Common Metrics and the performance measure or set of performance measures associated with that Common Metric. 3. Strategies Scorecard A strategies scorecard lists the strategies associated with a performance measure (e.g., a performance measure for a Common Metric) and the performance measures for each of those strategies 4. Program This term is defined broadly to include strategies (as defined above in the Key Terms and Concepts for the Common Metrics Initiative), the entire hub, and the combination of activities/functions represented by a Common Metric. 5. Performance Measure This term is defined above in the Key Terms and Concepts for the Common Metrics Initiative. 6. Note A note is simply narrative a location in the software where narrative can be easily entered and edited. Notes can also include links to files or other information, including Webpages and other resources accessed via the Internet. A set of notes will typically be a set of headings by which information has been organized and or a set of steps in a process. 7. Turn-the-Curve Page A graph with the baselines (historic and forecasted) for a performance measure and notes with the underlying analysis and corresponding set of proposed strategies and actions to turn the curve of the performance measure 8. Common Metric Page A set of notes with pertinent information about the Common Metric (e.g., the Operational Guidelines, Data Collection Notes) along with the performance measure or set of measures for the Common Metric 9. Strategy Page A set of notes with pertinent information (e.g., a description of the strategy) about a strategy along with the performance measures for the strategy. CTSA Common Metrics Scorecard Guidance 3

5 10. Actions This element in the software provides fields for describing and managing each of the actions associated with a turn-the-curve plan. The fields include assignment of responsibility for the action, planning and tracking implementation and completion of the action, assigning to do tasks associated with the action, and attaching budgets and other related documents. 11. Gantt Chart This element of the software provides the functionalities of a Gantt Chart for tracking and managing multiple actions. 12. Strategy Map This element of the software provides a variety of diagramming and editing functions for composing visuals displays that can be useful in providing overviews of the relationships among measures and strategies. THE COMMON METRICS SCORECARD The following is guidance for accessing the Common Metrics Scorecard. To navigate to the scorecard follow these instructions: 1. Login to your hub s CTSA Common Metrics Scorecard site at a. If you experience any issues logging into Scorecard, please contact the Tufts team at TuftsMCCMIHelpdesk@tuftsmedicalcenter.org or Select Scorecard Scorecards from the left side menu a. Alternatively, select the dropdown arrow on the left side the black bar at the top of the screen 3. Select the Common Metrics scorecard The Common Metrics scorecard includes the following common metrics and their associated Performance Measures. 1. Median IRB review a. Median # of days to IRB approval 2. Pilot Funding Publications and Subsequent Funding a. # of pilot research projects that have expended pilot funds since January 1, 2012 b. % of pilot research projects that have at least one research publication c. % of pilot research projects that have at least one subsequent research award 3. Careers in Clinical and Translational Research a. # of KL2 graduates who are currently engaged in clinical and translational research b. % of KL2 graduates who are currently engaged in clinical and translational research c. Of the KL2 graduates who are currently engaged in clinical and translational research, the # of underrepresented persons. d. Of the KL2 graduates who are currently engaged in clinical and translational research, the % of underrepresented persons. e. Of the KL2 graduates who are currently engaged in clinical and translational research, the # of women. CTSA Common Metrics Scorecard Guidance 4

6 f. Of the KL2 graduates who are currently engaged in clinical and translational research, the % of women. g. # of TL1 graduates who are currently engaged in clinical and translational research. h. % of TL1 graduates who are currently engaged in clinical and translational research. i. Of the TL1 graduates who are currently engaged in clinical and translational research, the # of underrepresented persons. j. Of the TL1 graduates who are currently engaged in clinical and translational research, the % of underrepresented persons. k. Of the TL1 graduates who are currently engaged in clinical and translational research, the # of women. l. Of the TL1 graduates who are currently engaged in clinical and translational research, the % percent of women. ENTERING DATA VALUES FOR COMMON METRICS PERFORMANCE MEASURES To enter data values for a Common Metric s Performance Measure navigate to the Common Metrics scorecard using the steps above then follow these steps. 1. Select the orange PM icon to the left of the Common Metric Performance Measure 2. Select Add Data Values 3. Select the green Plus icons to display time periods 4. Enter the data value for the Performance Measure in the appropriate time period s Actual Value box 5. When you are done entering data values select the Save button in the top right corner CTSA Common Metrics Scorecard Guidance 5

7 ENTERING THE TURN THE CURVE NARRATIVES FOR A COMMON METRIC S PERFORMANCE MEASURE To complete the Turn the Curve analysis for a Common Metric Performance Measure navigate to the Common Metrics scorecard using the steps above and then follow these steps: 1. Click on the name of the Performance Measure you would like to enter the narrative for 2. Select the Edit (Pencil) icon next to the note you would like to enter narrative for 3. Enter your text/narrative into the note field 4. Select the Save (Disk) icon in the top right corner of the note 5. Repeat steps 2-4 for each note area Note for the Strategies note, please see the section on Entering and Tracking Strategies below in this document OPERATIONAL GUIDELINES AND DATA COLLECTION NOTES If you need to review the operational guidelines or want to record notes aboutyour data collection process for a common metric, navigate to the Common Metrics scorecard following the steps above and then follow these steps: DOCUMENT DATA COLLECTION NOTES 1. Click on the name of the Common Metric 2. Select the Edit (Pencil) icon on the Data Collection Notes field 3. Enter your text/narrative into the note field 4. Select the Save (Disk) icon in the top right corner of the note DOWNLOAD THE OPERATIONAL GUIDELINES 5. Click on the name of the Common Metric 6. Click on the link to download the operational guidelines CTSA Common Metrics Scorecard Guidance 6

8 ENTERING AND TRACKING STRATEGIES If you decide that a strategy will be large and involved enough to have its own performance measures that have their own turn the curve analysis done you will want to create them as objects in the software. Use these steps for each Common Metric Performance Measure that will have its own additional strategies created. SET UP A NEW SCORECARD 1. Select Scorecard Scorecards from the left side menu 2. Select the green Add New button 3. Enter the Name of the Performance Measure (not the common metric name) in the name field with a - Strategies after the name a. For example: % of pilot research projects that have at least one research publication Strategies b. This scorecard will be used to hold all the strategies that relate to this Performance Measure 4. Select the Common Metrics scorecard for the Parent Scorecard 5. Click Save 6. Once the scorecard has been created you can move to the next section to create the Strategies and their Performance Measures in the software ADDING STRATEGIES AND PERFORMANCE MEASURES To create new strategies and track performance measures for those strategies follow the steps below. Edit the scorecard of the Performance Measure you would like to add a Strategy and Performance Measure for 1. Select Scorecard Scorecards from the left side menu 2. Select Edit on the Common Metrics Scorecard (look towards the far right) CTSA Common Metrics Scorecard Guidance 7

9 3. From the edit screen, scroll down to the bottom and select New Program Vers: 8/18/16 4. Enter the name of the Strategy in the name field 5. Enter the tag for the Strategy based on what common metric it is related to (this is used for reporting purposes) a. For Median IRB Review Duration related strategies enter the tag IRB-S b. For Pilot Funding Publications and Subsequent Funding, enter the tag PFP-S c. For Careers in Clinical and Translational Research enter the tag CCTR-S d. These tags all have a -S to denote that they are labeling the underlying strategies. If the tag does not exist you will be prompted to create it, enter the name of the common metric in the description for the new tag. 6. Select New Performance Measure under the newly created Program 7. Enter the name of the Performance Measure 8. Enter the same tag for the Performance Measure as you did for the Strategy a. For Median IRB Review Duration related strategies enter the tag IRB-S b. For Pilot Funding Publications and Subsequent Funding, enter the tag PFP-S c. For Careers in Clinical and Translational Research enter the tag CCTR-S 9. Select the other Performance Measure properties as necessary 10. When you are done adding all the Strategies and Performance Measures you want, select the Save & Exit button in the top right corner CONTACT INFORMATION If you have any questions about the scorecard system or technical issues, please contact the Results Scorecard support center at support@resultsscorecard.com. If you have questions about what types of information to enter into scorecard for the Common Metrics project, please contact the Tufts team at TuftsMCCMIHelpdesk@tuftsmedicalcenter.org or CTSA Common Metrics Scorecard Guidance 8