CIBMTR Audits. Deb Christianson, Senior Manager Monitoring and Auditing Matt Petcoff, Senior CRA

Transcription

1 CIBMTR Audits Deb Christianson, Senior Manager Monitoring and Auditing Matt Petcoff, Senior CRA

2 Conflict of Interest Disclosure We attest that we have no relevant financial, professional, or personal relationship with a commercial interest producing health care goods/services related to this educational activity. We will not discuss off-label use of commercial products. 2

3 Objectives Compare center specific error rates to all network centers. Identify the reporting areas with the highest error rates. Learn to correctly report data in these areas. 3

4 AUDIT RESULTS SUMMARY Presented by: Deb Christianson 4

5 Transplant Center Audit Results Audit results are shown in averages per cycle Audit cycles are four year periods: Cycle 1: Cycle 2: Cycle 3: Cycle 4:

6 Data Included in Audit Results Related and unrelated allogeneic, and autologous transplant data since December 3, 2007 Unrelated transplant data prior to December 3, 2007 Data NOT Included in Audit Results CIBMTR Milwaukee Legacy audit results are NOT included: Related allogeneic and autologous transplants prior to December 3,

7 Average Error Rates per Cycle Cycle 1 Cycle 2 Cycle 3 Cycle 4 Overall Error Rate Critical Field Error Rate Random Field Error Rate 3.1% 1.8% 1.5% 2.3% 5.3% 3.1% 2.5% 2.3% 2.6% 1.5% 1.1% 2.2% 7

8 Transplant Center Audit Results Error Rate Averages for All Transplant Centers 6% 5% passing score critical field error rate % 5.3% 4% 3% passing score critical field error rate starting 2011 Cycle 1 3% 3.1% 3.1% Cycle 2 Cycle 3 2% 2.5% 2.3% 2.3% 2.6% 2.2% Cycle 4 1% 1.8% 1.5% 1.5% 1.1% 0% Overall Critical Random Error Rate 8

9 Cycle 1 ( ) Critical Field Error Rates 77 Centers Represented Cycle One Critical Field Error Rates 12% 11% 10% 9% 8% 7% 6% 5% 4% 3% 2% 1% 0% Transplant Centers 9

10 Cycle 2 ( ) Critical Field Error Rates 89 Centers Represented Cycle Two Critical Field Error Rates 8% 7% 6% 5% 4% 3% 2% 1% 0% Transplant Centers 10

11 Cycle 3 ( ) Critical Field Error Rates 112 Centers Represented Cycle Three Critical Field Error Rates 12% 11% 10% 9% 8% 7% 6% 5% 4% 3% 2% 1% 0% Transplant Centers 11

12 Cycle 4 ( ) Critical Field Error Rates 107 Centers Represented Cycle Four Critical Field Error Rates ( ) 8% 7% 6% 5% passing score critical field error rate % 4% 3% 3% passing score critical field error rate starting % 1% 0% Transplant Centers 12

13 Center-Specific Summaries AUDIT SUMMARY CCN / TC Center Name Year Overall Critical Random Reporting Areas of Concern CAP Required Cycle % 2.1% 0.2%Karnofsky/Lansky No Cycle % 1.6% 1.6%Karnofsky/Lansky, GVHD Yes Cycle % 2.8% 1.5%Product Analysis, GVHD, Karnofsky/Lansky No Karnofsky/Lansky, HSCT infusion/product, method of latest disease assessment Cycle % 0.9% 0.2% Yes 6.0% Error Rate Summary ##### 5.0% 4.0% 3.0% 2.0% 1.0% 0.0% 2.4% 3.5% 1.7% 1.0% 2.8% 2.1% 1.6% 1.6% 1.5% 0.9% 0.2% 0.2% Overall Critical Random Error Rate

14 Center-Specific Summaries (cont.) Audit Packets Include: Audit Packet Information Sheet Center Specific Statistics Page (if applicable) Error Rate Averages Graph Critical Field Error Rate Line Graphs (all cycles) Additional information on resources and sample worksheets If there is not a packet for your center here today you can contact Edna Eich, Audit Coordinator by phone or by eeich@nmdp.org to have one sent to you.

15 REPORTING AREAS WITH HIGH CRITICAL FIELD ERROR RATES Presented by: Matt Petcoff 15

16 2011 Audits >300,000 total fields audited >8,400 total errors 52% discrepancy errors 32% omission errors 16% missing documentation errors (reports sent out as of 1/5/12) 16

17 Reporting Area A reporting area generally accounts for 10% of all errors at a center Reporting Areas Percentage of 2011 Centers Disease Status Pre- and Post- HSCT data fields Method of Latest Disease Assessment data fields Acute and/or Chronic GVHD data fields 92% 72% 53% HSCT Product/Infusion data fields 45% Karnofsky/Lansky 38% The average number of reporting areas per center:

18 Reporting Area: Disease Status Pre-HSCT Disease Status Post-HSCT Best Response Post-HSCT Complete Remission Achievement Post-HSCT Relapse or Progression Post-HSCT Current Disease Status/Method of Latest Disease Assessment 18

19 Reporting Area: Disease Status The following slides provide highlights of reporting issues seen during audits Not every reporting issue is covered Refers to a single question (or set of questions) on a form Handouts provided show the question as it appears on form 19

20 Reporting Area: Disease Status - Pre- and Post-HSCT Many errors seen in disease status pre-hsct Resources for reviewing disease status parameters: Pre-TED manual Disease Specific Manuals (Pre- and Post-HSCT AML) Pre- and Post-HSCT Disease Specific forms 20

21 Reporting Area: Disease Status Best Response to HSCT On disease-specific Post-HSCT forms (e.g., Form 2114 MDS) Q1. Compared to the disease status prior to the preparative regimen, what was the best response to HSCT since the date of the last report? Q2. Date best response first began: Refer to Handout 21

22 Post-HSCT MDS Disease Data F

23 Reporting Area: Disease Status Best Response to HSCT (cont.) Report overall best response to transplant, not best response within each reporting interval If best response achieved in previous reporting interval, confirm best response and check date previously reported 23

24 Reporting Area: Disease Status Best Response to HSCT (cont.) If new line of therapy administered after HSCT for relapse or progression, best response to HSCT can only be assessed up to date treatment began Therefore, best response will not change following treatment for relapse or progression Assess for Best Resp. Best Resp. Already Occurred HSCT Relapse/ Progression New Line Of Therapy 24

25 Reporting Area: Disease Status CR Achievement Post-HSCT On disease-specific Post-TED forms (Questions 79-81, version 2) Was a CR ever achieved in response to HSCT? CR criteria for each disease located in disease-specific tables at end of Pre-TED Manual Confirm assessment with physician particularly borderline cases Refer to Handout 25

26 Post-TED F2450, version 2 26

27 Reporting Area: Disease Status CR Achievement Post-HSCT (cont.) Option 1: Recipient already in CR at the start of the preparative regimen Select this option if recipient s disease status is CR or CRU for disease-specific status at transplant questions on Pre-TED form 27

28 Reporting Area: Disease Status CR Achievement Post-HSCT (cont.) Option 2: Yes, post-hsct CR achieved Select this option if recipient achieves CR post transplant without any change in therapy Report the first date in reporting period in which CR was achieved 28

29 Reporting Area: Disease Status CR Achievement Post-HSCT (cont.) Option 3: No, never in CR Select this option if recipient has not achieved CR in response to transplant and planned therapy Select this option if tests were completed, but CR status cannot be fully evaluated according to established criteria Date should reflect most recent and definitive evaluation in the reporting interval 29

30 Reporting Area: Disease Status CR Achievement Post-HSCT (cont.) Option 4: Not evaluated This option rarely used Indicates NO tests (e.g., radiology, laboratory, clinical assessments) were performed to evaluate CR during reporting interval 30

31 Reporting Area: Disease Status Post-HSCT Relapse or Progression Post-TED Form: First relapse or progression after HSCT (in this period, any type, not persistent disease) Report only the first relapse or progression Report only the first instance of each method of detection (Molecular, Cytogenetic/FISH, and/or clinical/hematologic) The detection methods may fall into different reporting periods if not all methods are tested at first sign of relapse. Be aware of when to use previously reported and not evaluated Refer to Handout 31

32 Post-TED F2450 Version 2 New Slide 32

33 Reporting Area: Current Disease Status & Method of Latest Disease Assessment Post-HSCT Current Disease Assessment On disease-specific Post-HSCT forms (e.g., Questions on Form 2110 AML) Method of Latest Disease Assessment On TED forms (e.g., Questions version 2) Refer to Handout 33

34 Post-TED F2450, version 2 34

35 Reporting Area: Current Disease Status & Method of Latest Disease Assessment (cont.) Report most recent assessment for each category in reporting interval: Molecular Cytogenetic, FISH, Flow Cytometry Clinical/ Hematologic 35

36 Reporting Area: Current Disease Status & Method of Latest Disease Assessment (cont.) Molecular Assessment: Involves determining whether a molecular marker for disease exists in blood or bone marrow Examples: PCR (e.g. FLT-3 mutations) or RFLP with PCR amplification (e.g. BCR/ABL) Report if assessing disease, do not report if assessing engraftment only 36

37 Reporting Area: Current Disease Status & Method of Latest Disease Assessment (cont.) Cytogenetic, FISH, and Flow Cytometry: Cytogenetic Assessment Involves testing blood or bone marrow for presence of a known cytogenetic abnormality which reflects recipient s disease FISH (Fluorescence in situ hybridization) Sensitive technique that uses probes which recognize and bind to fragments of DNA commonly found in disease identifies disease cells Flow Cytometry Technique performed on blood, bone marrow or tissue preparations where cell surface markers can be quantified on cellular material Report if assessing disease, do not report if assessing engraftment only 37

38 Reporting Area: Current Disease Status & Method of Latest Disease Assessment (cont.) Clinical/Hematological Assessments: Pathology (e.g. bone marrow) Radiology (e.g. X-ray, MRI, CT, PET) Blood Serum (e.g. CBC, peripheral blood smear) Physician Assessment 38

39 Reporting Area: Current Disease Status & Method of Latest Disease Assessment (cont.) Clinical/Hematological Assessments (cont.): Report most recent, yet most definitive assessment Often several tests are done within reporting interval There is almost always a clinical/hematologic assessment during reporting interval A physician visit is a clinical assessment! 39

40 Reporting Area: Current Disease Status & Method of Latest Disease Assessment (cont.) Clinical/Hematologic Assessment Example: A recipient with lymphoma has a positive PET/CT scan 5 weeks prior to the date of contact. On the date of contact, the recipient has a physical exam that shows signs of lymphadenopathy. Which to report? 40

41 Reporting Area: Current Disease Status & Method of Latest Disease Assessment (cont.) Disease Assessment Second Example: Yes AML/ALL Post-HSCT CRF: Was the disease status assessed since the date of the last report? Yes, is the same assessment as [AML: 31-47; ALL 21-41], as no treatment was given No Report any molecular, cytogenetic/fish/flow cytometry, clinical/hematologic assessments (including CBC) Report ONLY if the recipient relapsed, the assessments were reported in the relapse section, and no additional treatment was given. Check again for a CBC or physician evaluation! Refer to Handout 41

42 Post-HSCT AML Disease Data F

43 Reporting Area: Acute and /or Chronic GVHD Acute vs. Chronic Diagnosis, staging, grading Biopsies If not clear in notes Clinical judgment Clinical judgment Do not use to stage/grade Consider creating tool with physician(s) to better document GVHD 43

44 Reporting Area: HSCT Product / Infusion At Infusion time point is required Additional time points are great! Product analysis values are absolute values (do not report them in x10 6 /Kg) Thaw start and stop times are critical data points that are audited for all cryopreserved products (frozen PBSC, CBUs) 44

45 Reporting Area: Karnofsky / Lansky KPS scores are critical data fields and are captured at the following time points: TED TRACK CRF TRACK Pre-TED Baseline Within 30 Days prior to HSCT Pre-TED Within 30 days prior to HSCT 100 Day Post- HSCT Six Month Post- HSCT Within 14 days prior to contact date Annual Post- HSCT Within 1 Month prior to contact date 45

46 General Observations Review previously submitted forms for longitudinal consistency Review forms before submission to verify all required fields are completed to avoid omission errors If your center has data in an EMR, provide auditor access to EMR to avoid missing documentation errors 46

47 General Observations (cont.) Have all inpatient and outpatient progress notes available for auditors Create tools to help physicians record information specific to reporting requirements Helpful for data that is difficult to find Pre-HSCT KPS score Myeloablative versus NMA/RIC GVHD staging/grading 47

48 Contact Information Deb Christianson, Sr. Manager Auditing and Monitoring Matt Petcoff, Sr. Clinical Research Associate