National Institutes of Health

Transcription

1 National Institutes of Health Much of the biomedical research in the United States is supported by the Federal Government, primarily the National Institutes of Health (NIH) 1

2 NIH Extramural Program Grant Patron (assistance, encouragement) Cooperative Partner Agreement (assistance but substantial program involvement) Contract Purchaser (acquisition) 2

3 Grants & Acquisition at the NIH Grants Acq. Approximate obligations in billions of dollars in

4 NIH FY 2002 ACQUISITIONS $3,417.9 M DELPRO 11.1% PURCHASES 12.8% PURCHASE CARDS 6.6% BPAS 3.6% NON R&D CONTRACTS 24.3% R&D CONTRACTS 41.6% Includes all acquisition FY 2002 funds obligated by NIH organizations. 4

5 NIH FY 2002 ACQUISITIONS $3,417.9 M By Awarding Office PURCHASE CARD 6.6% BPAS 3.6% NCI 19.4% OLAO 16.4% OTHER: NIDA 2.4% NICHD 1.9% NIMH 1.8% NIDDK 1.5% OLAO R&D 1.5% NIAMS 1.1% NINDS 0.8% NIAAA 0.5% CC 2.6% ORS CONSTR 4.3% OTHER * 11.5% NLM 3.4% NIEHS 4.2% NIAID 8.9% NHLBI 8.0% DELPRO 11.1% Includes all acquisition FY 2002 funds obligated by NIH organizations. 5

6 6

7 NATIONAL INSTITUTE OF CHILD HEALTH AND HUMAN DEVELOPMENT CONTRACTS: $65.9 M JAMES QUINN NATIONAL CANCER INSTITUTE CONTRACTS: $599.9 M PURCHASES: $63.5 M TOTAL: $663.4 M JOHN CAMPBELL CLINICAL CENTER Contracts: $24.3 M Purchases: $66.1 M Total: $90.4 M SYDNEY JONES NATIONAL INSTITUTE ON ALCOHOL & ALCOHOLISM CONTRACTS: $17.4 M ROBERTA WILHELM NIH ACQUISITION ORGANIZATION FY 2002 OFFICE OF LOGISTICS AND ACQUISTION OPERATION DAVID RAMOS OFFICE OF PROCURMENT MGMT. CONTRACTS: $383.6 M PURCHASES: $178.0 M TOTAL: $561.6 M DIVISION OF RESEARCH ACQUISITION CONTRACTS: $52.7 M HENRY DOVE NATIONAL INSTITUTES OF HEALTH OFFICE OF ACQUISITION MANAGEMENT AND POLICY DIANE J. FRASIER (HEAD CONTRACTING ACTIVITY, NIH) TOTAL: $3,417.9 R&D: $1,421.2 M NON R&D: $1,268.5 M PURCHASE CARDS: $225.6 M DELPRO: $379.1 M BPAS: $123.5 M NATIONAL INSTITUTE OF ALLERGY AND INFECTIOUS DISEASES CONTRACTS: $243.7 M BRENDA VELEZ CONTRACTS: $39.4 M PURCHASES: $21.4 M TOTAL: $60.8 REBECCA GUENTHNER TOTAL: $304.5 M NATIONAL HEART, LUNG AND BLOOD INSTITUTE CONTRACTS: $255.2 M PURCHASES: $ 18.2 M TOTAL: $273.9 M ROBERT BEST NATIONAL INSTITUTE OF NEUROLOGICAL DISORDERS AND STROKE CONTRACTS: $26.1 M KIRK DAVIS NATIONAL INSTITUTE ON DRUG ABUSE CONTRACTS: $82.7 M JOHN HAMILL NATIONAL INSTITUTE OF ARTHRITIS AND MUSCULOSKELETAL DISEASES CONTRACTS: $33.5 M EILEEN WEBSTER CISSEL NATIONAL INSTITUTE ON MENTAL HEALTH CONTRACTS: $61.1 M DAVID ESKENAZI NATIONAL INSTITUTEOF DIABETES & DIGESTIVE & KIDNEY DISEASES CONTRACTS: $39.2 M PURCHASES: $.10.9 M TOTAL: $50.1 M PATRICK SULLIVAN * Non R&D total includes contracts and purchases. NLM figures include NTIS funds. NATIONAL LIBRARY OF MEDICINE CONTRACTS: $55.5 M PURCHASES: $60.8M TOTAL: $116.3 M KAREN RIGGS (ACTING) NATIONAL INSTITUTE OF ENVIRONMENTAL HEALTH SCIENCES CONTRACTS: $89.2 M ALLAN BENTON CONTRACTS: $36.9 M PURCHASES: $17.5 M TOTAL: $ 54.4 M ANTOINETTE BRIDGES TOTAL: $143.6 M OFFICE OF RESEARCH SERVICES CONSTRUCTION CONTRACTS: $146.5 M MELISSA RICHARDSON 7 5/29/03

8 Balanced Scorecard Background: Procurement Executive Council (PEC) Office of Federal Procurement Policy (OFPP) Government Performance and Results Act (GPRA) a comprehensive approach to performance management and improvement 8

9 Balanced Scorecard A group of Departments: Energy, Transportation, HHS and others looked at various organizations to conduct these Measurements A contractor was engaged for the effort 9

10 Balanced Scorecard Seven Steps: Develop purpose, objectives, and goals Identify population and select participants Select the survey medium Design the survey instrument Obtain survey clearance from OMB Pretest and revise the survey instrument Plan for data entry and selected analytical tools 10

11 Purpose, objectives & goals: Balancing Performance Three key elements support acquisition excellence: Acquisition Excellence Delighted Customers and Dedicated Vendors Efficient Business Processes Innovative Leaders and Empowered Employees It is important to maintain the balance among all three elements 11

12 Balanced Scorecard (paper) 1998 National Institute of Allergy and Infectious Diseases (NIAID) National Institute of Neurological Disorders and Stroke (NINDS) 12

13 Balanced Scorecard (paper) 1999 National Institute of Child Health and Human Development (NICHD) National Cancer Institute (NCI) 13

14 Balanced Scorecard (web-based) 2000 Division of Research Contracts (DRC) National Heart, Lung and Blood Institute (NHLBI) National Library of Medicine (NLM) NIH Information Technology Acquisition and Assessment Center (NITAAC) Office of Procurement Management 14

15 Balanced Scorecard (web-based) 2001 National Institute of Environmental Safety and Health (NIEHS) 2004 Going back to ALL GROUPS 15

16 Acquisition Performance Summary: NIEHS Research Contracts Branch December 6, 2001 The NIEHS had the following profile: 9 procurement employees 39 customers 37 contractors 16

17 NIEHS Major Conclusions Overall customer ratings good; some quality and timeliness issues Overall vendor satisfaction excellent for both contracting and program offices Vendors more satisfied with cooperation of the contracting office than that of the program office Overall employee satisfaction not nearly as good several executive leadership issues 17

18 NIEHS Statistical Considerations RCB survey response rates high Customer 79% Vendor 76% Employee 89% Manager 100% Such response rates minimize non-response bias (i.e., most opinions are reflected) Margins of error are well within plus/minus 3 percentage points (at the 95 percent confidence level) 18

19 Analytical Concepts: Identifying Performance Gaps Performance gaps identified graphically for each office: Category Measured Level of Performance Performance Gap Maximum Possible Level of Performance (Excellence) 0 20% 40% 60% 80% 100% Percent Who Agree or Strongly Agree 19

20 Customer Survey NIEHS: Highest Performing Areas Quality Has adequate know ledge of the products/services being procured 66% Timeliness Works w ith me to develop contracting strategies early in the planning process 67% Meets projected contract aw ard dates 63% Obtains products/services w hen I need them 63% Service/Partnership Deals w ith me in a courteous and business-like manner 84% Respects my technical expertise 83% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Percent Who Agree or Strongly Agree 20

21 Quality Customer Survey NIEHS Major Performance Gaps Selects contractors with best value (quality and price) 47% Obtains high-quality products/services 46% Timeliness Works with me on contract monitoring strategy soon after contract award 47% Prevents problems that may lead to delivery delays 47% Service/Partnership Helps me to solve problems quickly and effectively 62% Is responsive 60% Works closely with me to monitor contractor performance 60% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Percent Who Agree or Strongly Agree 21

22 Selection of Improvement Areas We identified potential improvement areas with the largest performance gaps for RCB: Customer-derived Potential Improvement Areas/Factors Area Factor 1. Quality Obtains high-quality products/services Selects contractors with best value 2. Timeliness Prevents problems that may lead to delivery delays Works on contract monitoring strategy soon after contract award 3. Service/Partnership Is responsive Works closely to monitor contractor performance Helps solve problems quickly and effectively 22

23 Selection of Improvement Areas (Continued) Consider other factors besides low scores in selecting improvement efforts: High importance to management Ease of improvement -- low-hanging fruit Contractor posited believes that NIEHS can better balance its acquisition performance by improving: Customer areas Vendor areas Employee areas 23

24 Improvement Efforts Contractor recommended that RCB: Select two improvement areas (e.g., employee executive leadership, customer timeliness, etc.) Establish a project improvement team for each selected area Arrange for team support, as needed (e.g., obtain the service of a facilitator) Develop a team reward and recognition system for the project teams 24

25 Improvement Efforts (Continued) Based on the survey results, we recommend that NIEHS take the following steps in the next twelve months: Share results with the entire staff Review performance gaps Select appropriate and realistic improvement targets Establish project teams to help make improvements Obtain facilitator to guide project teams, as needed Determine team reward and recognition system to be used for project teams Implement performance improvements 25

26 Balanced Scorecard Related Activities Quarterly Balanced Scorecard Users Group Meeting Efficiency and Effectiveness Analysis Benchmarking Forum 26

27 Balanced Scorecard User s Group Quarterly meeting of the Balanced Scorecard User s Group Review new templates Discuss Improvement Efforts Prepare for Forum Discuss new areas: Grants, Simplified Acquisiton 27

28 Efficiency & Effectiveness Analysis OGAM tasked contractor to assess acquisition efficiency and effectiveness of HHS OPDIVs, and to propose HHS benchmarks Contractor used two proven tools to make this assessment Statistical cost-workload functions (cost regression functions) Linear programming frontier analysis (Data Envelopment Analysis, DEA) 28

29 Efficiency Definition Statistical cost-workload functions help isolate fixed costs from variable costs For the purposes of this presentation, good operating efficiency is achieved with low average fixed costs Low average fixed costs result from: Heavy Workloads Lean Processes Appropriate mix of personnel 29

30 Efficiency & Effectiveness (cont) We used the following data sources Workload outputs Obtained from Departmental Contracts Information System (DCIS) Data included: new awards, funded modifications, administrative modifications, simplified acquisitions, FY01 closeouts only Data covered FY97 FY01 Qualitative outputs Obtained from acquisition BSC customer survey Timeliness, service partnership, quality, overall evaluation 30

31 Efficiency & Effectiveness (cont) Data sources (cont) Labor resources Obtained from HHS personnel database Full-time employees in acquisition occupations: GS1102, GS1105, GS1106 FTEs and salary cost in FY01 dollars (adjusted for inflation) Data covered FY97 FY01 31

32 Efficiency & Effectiveness: Implications for Improvement OPDIVs can use individual results to make improvements Reduce average cost of operations Increase customer satisfaction scores Also consider employee and vendor survey results Costs can be reduced by Process improvements Fixed cost reductions Workforce planning 32

33 Implications for Improvement: Survey Scores (cont) Factors to consider in selecting improvement areas Low scores High importance to management Ease of improvement-- low-hanging fruit Once improvement areas are selected Establish project improvement teams Arrange for team support (e.g., facilitator) Develop team reward and recognition system for project teams 33

34 Implications for Improvement: Cost Example B A Average cost 1 Avg Var Cost Average cost 2 W B W A Workload Level 34

35 Regression Analysis: Average Fixed Costs Average fixed cost is a good measure of efficiency OPDIV Average Fixed Cost High Efficiency FY97 FY98 FY99 FY00 FY01 A D C B E 1,669 2,322 1,597 1,565 1,831 1,238 2,126 1,532 1,452 1,440 1,327 1,266 1,434 1,364 1,468 1,052 1,139 1,338 1,598 1, ,246 1,254 1,597 1,569 Average Efficiency H G 3,048 2,483 3,805 2,582 2,954 1,995 2,251 2,424 1,938 2,524 Low Efficiency I J F 2,682 1,510 4,214 3,348 2,019 3,662 2,889 2,554 3,690 3,398 2,788 3,781 2,633 3,324 3,806 35

36 Data Envelopment Analysis: Graphical Example Graphical example for assessing efficiency/ effectiveness (if constant returns to scale) 40 B Efficiency Frontier: combination of maximum possible number of total actions and customer service scores Total Actions 10 C A Customer Service 50 36

37 Efficiency/Effectiveness Definition For the purposes of this presentation relative efficiency/effectiveness is calculated using DEA Efficiency/effectiveness measured as ratio of outputs to inputs (e.g. workload index/staff costs) DEA gives relative efficiencies (relative to data considered) not absolute efficiencies 37

38 Conclusions Taking into account both regression and DEA analyses, OPDIVs A and B are benchmarks OPDIV Efficiency (regression) Effectiveness (DEA) Overall Assessment A B High High High High Excellent C High Average Good D E F G H High High Low Average Average Low Low High Average Average Average I J Low Low Low Low Marginal 38

39 Benchmarking Forum 2003 Presentation from benchmarking organizations Who are you? Successful Process what strategies/initiatives have helped? What improvements contributed? What obstacles did you overcome? 39

40 Questions? Greg Pryor Procurement Analyst Office of Acquisition Management and Policy National Institutes of Health Phone: Fax: