METRICS THAT MATTER: ASSESSING WORKFLOW AND STAFFING

Transcription

1 METRICS THAT MATTER: ASSESSING WORKFLOW AND STAFFING Michele Vaughan, BS, CCRP Research Strategy and Compliance Specialist Nationwide Children s Hospital, Columbus, OH Susan Sedwick, PhD, CRA, CSM Senior Consultant Attain LLC

2 Learning Objectives Develop a rating matrix for proposals, awards, and clinical studies applicable to their institution Understand how to use workflow matrix to justify additional positions and increase salaries to retain experienced staff

3 Metrics that Matter The ability to quantify associated workloads can have a significant impact on the efficiency and success of a research organization.

4 Why do we need to assess workload? Balance workloads among staff Staff Satisfaction Efficiency in workloads Justification in grant/clinical trial budgets Strategic planning for budgeting Help anticipate growth in staffing Staff retention Staff competencies for workflows

5 Scoring Workloads Workload Complexity

6

7 Which Tool?? a. Validated and Published Tool b. Fits the needs of your practice c. Buy In from your staff d. Leadership team values the data e. Reproduce the data f. Compare your results to other departments g. Metrics that you can make decisions from

8 Tools are Available OPAL: Ontario Protocol Assessment Level developed by Ontario Institute for Cancer Research Measuring Clinical Trail-Associated Workload in a Community Clinical Oncology Program Advantage Clinical-Protocol Scoring and Determining Study Load per Clinical Research Coordinator American Society of Clinical Oncology NCI Trial Complexity and Elements Scoring Model University of Michigan Research Effort Tracking Application (RETA) Ontario Protocol Assessment Tool (OPAL) Wichita Community Clinical Onc Program Protocol Tool (WPAT)

9 What Metrics Matter? Treatment vs Non Treatment Multiple Contact Events # Central Processing #Special Procedures ACUITY # Active Patients Frequency of Visits Phase of Study Monitoring Source Documentation

10 OPAL- Scoring System Workloads Metrics Complexity of Trial Number of patient visits Number of patients Type of Trial Status of Patient on trial What Matters Multiple Treatments, complex data forms, special procedures, central processes Frequency of staff to visit patients How many patients are enrolled Early Phase to registry trials On study, off study, on follow up

11 How to Get Started 1 2 3

12 OPAL-Protocol Scoring (Base Score) Treatment Phase II, III, IV Single SP/ CP Treatment Phase II, III, IV Multiple SP or CP-more than 2 occurrences Treatment Phase II, III, IV Single SP + Multiple CP or Single CP and Multiple SP Treatment Phase II,III,IV 2 or more SP 2 or more CP Any Phase 1 trial SP-Special Procedure Phase II, III, IV Non Drug Interventional Multi Contact Non Treatment Single Contact Non Treatment CP- Central Processing

13 OPAL-Protocol Scoring Determine Base OPAL Score Each Protocol Calculate OPAL Total Score Each Protocol Protocol Base Score Inpatient (max.5) Monitoring or 100% Source # or Duration TX FUP Visits Industry Sponsor CRO IIT Multiple Surveys TOTAL Score NCH NCH Base Score + Optional Elements = Total Score for Protocol

14 OPAL-Protocol Scoring Optional Elements Determine the Total Score for Each Protocol determine If there are Optional Elements to the base score: Scoring Add Value Inpatient needed Maximum value.5 On site Monitoring or 100% Source Documents Submission # or Duration of treatment/follow Up Visits Industry sponsor/cro/iit Multiple Survey s (3+). Questionnaires Maximum value.5 Maximum value.5 Maximum value.5 Maximum value.5 TOTAL OPAL SCORE Each Protocol

15 OPAL-Protocol Scoring Determine Base OPAL Score Each Protocol Calculate Total OPAL Score Each Protocol Protocol NCH 001 NCH 002 Base Score Inpatient (max.5) Monitoring or 100% Source # or Duration TX FUP Visits Industry Sponsor CRO IIT Multiple Surveys TOTAL Score Base Score + Optional Elements = Total OPAL Score for Protocol

16 OPAL- Protocol Scoring Metric Higher Protocol Score More Complexity Involved # Trials Open Acuity Score Per Trial # Trials Enrollments

17 OPAL Scores Per Protocol Acuity Scores Per Protocol Open in Each Service Line BMT ONC HEM 5 0

18 OPAL Scoring for Staffing Workload Complexity OPAL Protocol Scores

19 OPAL Scoring for Staff

20 OPAL- Add Staff Volume Total OPAL Staff Workload Calculation: (# Active Patients X Total Score) + (# FUP X Total Score/2)+ Total Score Susan Workload Michele Workload Protocol OPAL Score # Active Patients # FUP Patient Total Score Protocol OPAL Score # Active Patients # FUP Patient Total Score NCH NCH010 6 NCH NCH020 3 NCH NCH030 2 NCH NCH040 3 TOTALS TOTALS NCH002= (10X3) + (3 X (10/2) ) + 10= 55

21 OPAL- Add Staff Volume Total Staff Workload Calculation: (# Active Patients X Total Score) + (# FUP X Total Score/2)+ Total Score Susan Workload Protocol OPAL Score # Active Patients # FUP Patient Total Score NCH NCH NCH NCH TOTALS Michele Workload Protocol OPAL Score # Active Patients # FUP Patient Total Score NCH NCH NCH NCH TOTALS Total Active Patients=14 Acuity Total=165 Total Active Patients=39 Acuity Total=131

22 Staffing Metrics Determine workloads per staff member: Employee #Active # FUP Total Acuity Score # Different Protocols Sue Amy Mike Sally Aaron Kelly Heidi Veronica Melinda Joe

23 Staffing Metrics Determine Workload per Service Line Teams 2017 OPAL SCORES by Division Jan-17 Feb-17 Mar-17 Apr-17 May-17 Jun-17 Jul-17 Aug-17 Sep-17 Oct-17 Nov-17 Dec Avg/mth Hem Onc Neuro Onc BMT Host Def Add Acuity scores all employees / # of employees=avg Acuity Per Month

24 Average Staff Acuity Per Month Jan-17 Feb-17 Mar-17 Apr-17 May-17 Jun-17 Jul-17 Aug-17 Sep-17 Oct-17 Nov-17 Dec Avg/mth Average Acuity Score Per Month Host Def BMT Neuro Onc Onc Hem

25 Metrics that Matter Learning Objectives Protocol and Staffing: Choose a Rating Matrix for your team Apply a scoring system to assess workload Justify staffing needs with your matrix data Workloads assessment includes acuity Workloads per team Acuity per protocol

26 OPAL Scoring for Feasibility OPAL Scoring to determines Staff Time and Effort for Research Budgets

27 How to Get Started OPAL & Feasibility 1 2 3

28 OPAL Scoring for Feasibility Protocol Score % of Effort Per Month RN % of Effort Per Month CRC Hours Per Month RN Hours Per Month CRC <2 10% 0% % 0% % 5% % 10% % 15% % 25% % 40% Monthly hours assumes 2080 hours per 12months ( per month)

29 OPAL Scoring for Feasibility Protocol Score % of Effort Per Month RN % Effort Per Month CRC Hours Per Month RN Hours Per Month CRC % 10% Determine Protocol Score Grant Application Time and Effort Needed for Research Staffing Clinical Trial Budget Use Hours Multiply by # Months

30 OPAL Scoring for Feasibility Protocol Score % of Effort Per Month RN % Effort Per Month CRC Hours Per Month RN Hours Per Month CRC % 10% Protocol Score $ Per Month RN $ Per Month CRC 5-6 $2,256-$2,704 $676 RN hourly rate=$52 CRC hourly rate=$39

31 OPAL Scoring for Feasibility Budget Request: Sponsor: XXX Opal Score: 5-6 Estimated Number of Months: 6 Phase of Study: 3 Protocol Score $ Per Month RN 5-6 $2,256- $2,704 $ Per Month CRC $676 Research Time $ Per Month # of Month On Study TOTAL $$ % Effort RN $2,256 6 $13,536 25% CRC $676 6 $4,056 10% TOTAL Per Patient $2,932 6 months $17,592 35% Estimated cost per patient for 6 months

32 Metrics that Matter Learning Objectives Feasibility: Choose a Rating Matrix can use for feasibility Collect team s data for time and efforts per protocol Update $$ rates yearly with employee raises Decrease staff needs by streamline workflow Budget estimates Buy In from Staff and PI s Costs were Reproducible across Protocol Scores

33 COMPLEXITY SCORING

34 Daily Download

35 Tracking Progress

36 Criticality

37 Where is it?

38 Who is it with? Initial Review Awaiting C&G Action Legal Review Sponsor Review Coordinator A B C D E F G H I J K L M N N O P Daily Total

39 Numbers Don t Equal Workload Type of Agreement Sponsor Scope Budget Intellectual Property Subawards Risk

40 Numbers Don t Equal Workload Type of Agreement Contract Grant or Cooperative Agreement Material Transfer Agreement Use Agreement Non-disclosure Agreement Clinical Trials Agreement Incoming Subcontract Purchase Order/Service Teaming Agreement Memorandum of Understanding Other Agreements

41 Numbers Don t Equal Workload Sponsor Federal State Industry Foundation Foreign New Sponsor Problem Sponsor

42 Numbers Don t Equal Workload Scope Vague or Specific Place of Performance Criticality of Performance Subawards Compliance IRB IACUC IBC Export

43 Numbers Don t Equal Workload Budget Cost Share Significant Dollars US Dollars Contingency Funds

44 Numbers Don t Equal Workload Intellectual Property Ownership BIP Licensed Technology Involved Principal Investigator

45 Numbers Don t Equal Workload Subawards Multiple FDP or Institutional Template FAR Flow Down Past experience with subawardee Dependence High risk Large dollar amount

46 Numbers Don t Equal Workload Risk Place of performance Principal Investigator Warranty on deliverables Export controlled or NIST requirements Foregin sponsor

47 or Negotiator A Complexity Scoring TYPE SPONSOR BASE SCOPE BUDGET IP SUBS RISK C-SCORE Grant NSF Coop NSF Contract DoD P/T Other State Other State SBIR DoD TOTAL 31

48 NUMBER ROLE Sheer Numbers 36 Sr. Coordinator 28 Sr. Coordinator 29 Sr. Coordinator 28 Coordinator 21 Sr. Coordinator 18 Sr. Coordinator 17 Coordinator 14 Coordinator 11 Coordinator 5 Coordinator 5 Asst. Director 0 Proposal 0 Proposal 0 Proposal 0 Proposal 0 Proposal 0 Proposal 212

49 WORKLOAD ROLE WORKLOAD 72 Sr. Coordinator 56 Sr. Coordinator 44 Sr. Coordinator 28 Coordinator 21 Sr. Coordinator 18 Sr. Coordinator 51 Coordinator 21 Coordinator 18 Coordinator 5 Coordinator 25 Asst. Director 0 Proposal 0 Proposal 0 Proposal 0 Proposal 0 Proposal 0 Proposal 269

50 WORKLOAD ROLE WORKLOAD (Hint: It is not just numbers or weighted workload). 72 Sr. Coordinator 56 Sr. Coordinator 44 Sr. Coordinator 28 Coordinator 21 Sr. Coordinator 18 Sr. Coordinator 51 Coordinator 21 Coordinator 18 Coordinator 5 Coordinator 25 Asst. Director 0 Proposal 0 Proposal 0 Proposal 0 Proposal 0 Proposal 0 Proposal 269

51 Over time this will more accurately predict Capacity Trends Staffing Compensation Promotion Taking into consideration FTEs Events (ARRA, natural disasters)

52 Proposal Productivity Proposals per Year SPS/25 SPS/8 SPS/5 PS/3 PS/2 PS/1 PS/<1

53 Proposal Productivity Proposals per Year Workload per Year SPS/25 SPS/8 SPS/5 PS/3 PS/2 PS/1 PS/<1

54 Multiple Applications Proposals Invention Disclosures Post-award (Billing)

55 INPUT AND QUESTIONS Michele Vaughan, BS, CCRP Research Strategy and Compliance Specialist Nationwide Children s Hospital Michele.Vaughan@nationwidechildrens.org Susan Sedwick, PhD, CRA, CSM Senior Consultant Attain LLC ssedwick@attain.com