Medicare and Medicaid Audits Using Statistical Sampling and Extrapolation: Challenging Methods and Results

Transcription

1 Presenting a live 90-minute webinar with interactive Q&A Medicare and Medicaid Audits Using Statistical Sampling and Extrapolation: Challenging Methods and Results THURSDAY, JUNE 14, pm Eastern 12pm Central 11am Mountain 10am Pacific Today s faculty features: Anna M. Grizzle, Member, Bass Berry & Sims, Nashville, Tenn. Dr. Patricia L. Maykuth, Ph.D, President, Research Design Associates, Decatur, Ga. The audio portion of the conference may be accessed via the telephone or by using your computer's speakers. Please refer to the instructions ed to registrants for additional information. If you have any questions, please contact Customer Service at ext. 1.

2 Tips for Optimal Quality FOR LIVE EVENT ONLY Sound Quality If you are listening via your computer speakers, please note that the quality of your sound will vary depending on the speed and quality of your internet connection. If the sound quality is not satisfactory, you may listen via the phone: dial and enter your PIN when prompted. Otherwise, please send us a chat or sound@straffordpub.com immediately so we can address the problem. If you dialed in and have any difficulties during the call, press *0 for assistance. Viewing Quality To maximize your screen, press the F11 key on your keyboard. To exit full screen, press the F11 key again.

3 Continuing Education Credits FOR LIVE EVENT ONLY In order for us to process your continuing education credit, you must confirm your participation in this webinar by completing and submitting the Attendance Affirmation/Evaluation after the webinar. A link to the Attendance Affirmation/Evaluation will be in the thank you that you will receive immediately following the program. For additional information about continuing education, call us at ext. 2.

4 Program Materials FOR LIVE EVENT ONLY If you have not printed the conference materials for this program, please complete the following steps: Click on the ^ symbol next to Conference Materials in the middle of the lefthand column on your screen. Click on the tab labeled Handouts that appears, and there you will see a PDF of the slides for today's program. Double click on the PDF and a separate page will open. Print the slides by clicking on the printer icon.

5 Medicare and Medicaid Audits Using Statistical Sampling and Extrapolation Anna Grizzle - Bass, Berry & Sims, PLC Pat Maykuth, Ph.D. - Research Design Associates, Inc.

6 Use of Statistical Sampling for Overpayment Estimation Medicare or Medicaid audit MAC audits following TPE reviews ZPIC or UPIC audits OIG audits Medicaid agency audits MIC audits OIG self-disclosure protocol Internal compliance audit Calculation of damages in FCA case? 6

7 Legal Basis for Statistical Sampling for Overpayment Estimation The use of statistical sampling to project an overpayment... does not deny a provider or supplier due process. Neither the statute nor regulations require that a case-by-case review be conducted in order to determine that a provider or supplier has been overpaid and to determine the amount of overpayment. HCFA Ruling

8 Legal Basis for Statistical Sampling for Overpayment Estimation Statistical sampling does not violate due process so long as extrapolation is made from a representative sample and is statistically significant. Chaves County Home Health Service, Inc. v. Sullivan, 931 F.2d 914 (D.C. Cir. 1991), cert. denied, 402 U.S (1992). 8

9 Legal Basis for Medicare Statistical Sampling and Extrapolation A Medicare contractor may not use extrapolation to determine overpayment amounts... unless... There is a sustained or high level of payment error; or Documented educational intervention has failed to correct the payment error 42 U.S.C. 1395ddd(f)(3) 9

10 Legal Basis for Medicare Statistical Sampling and Extrapolation The PIM provides basic concepts rather than a checklist to complete a valid statistical sampling. When applied correctly, the PIM s concepts can lead to a proper methodology to use as a basis for extrapolation. The PIM s concepts often are not applied correctly in developing the statistical sampling methodology used as the basis of extrapolation in Medicare audits. Source: Chapter 8 Benefit Integrity; Medicare Program Integrity Manual; available at: 10

11 Legal Basis for Medicaid Statistical Sampling and Extrapolation Dictated by state law If no explicit authority, look to due process requirements 11

12 Performance of Statistical Sampling and Extrapolated Overpayment Major Steps Selecting the provider or supplier Prior history of overpayment error TPE Review by MACs Selecting the period to be reviewed Defining the universe, the sampling unit, and the sampling frame Define the provider, issue to be reviewed, time period, and methodology for measuring overpayment Source: Chapter 8 Benefit Integrity; Medicare Program Integrity Manual; available at: 12

13 Performance of Statistical Sampling and Extrapolated Overpayment Major Steps (cont.) Designing the sampling plan and selecting the sample Reviewing each of the sampling units and determining if there was an overpayment or under payment Estimating the overpayment Source: Chapter 8 Benefit Integrity; Medicare Program Integrity Manual; available at: 13

14 Documentation Requirements The following items must be documented: Universe Sampling frame (sorted) Random sampling process (seed, program, inputs and printout) Sample size determination calculations Extrapolation formulae, inputs and printouts Extrapolation recalculation where appropriate 14

15 Valid Methodologies MPIM specifically lists: Simple random Stratified random Cluster MPIM specifically requires proper execution of chosen methodology 15

16 Valid Statistics Statistically Valid Random Sample (SVRS) Probability Sample Use correct formulae Follow statistical requirements of chosen statistics Point Estimates and Confidence Intervals are valid statistics if properly executed Minimum Sum Method and Penny sampling not been validated in the statistical literature 16

17 Probability Statistics To know the number of samples of the chosen size that can be created from the frame Known likelihood of selection of each sampling unit Proper randomization Proper execution of sample methodology Use correct formulae Accurate measurement of the variable of interest (overpayment) Source: Chapter Benefit Integrity; Medicare Program Integrity Manual; available at: 17

18 Frame Sample (1, 2, 3, 4, 5) (2, 3, 4, 5, 6) (1, 11, 21, 31, 41) (1, 12, 23, 34, 45) (8, 23, 46, 73, 90) Outside of frame x (1, 2, 3, 4, X) 18

19 Probability Sample 1. To be able to identify the number of samples of a given size that can possibly be selected from a frame of a frame of a given size claims. How many samples of 5 can be selected from a frame of 100? 2. In a simple sample each claim must have a known and equal probability of selection. Strata take simple samples from strata frames. 19

20 Probability Yardstick Single sample chosen for the audit is only one sample (of the chosen size) out of a large number of possible samples Sampling distribution of all possible means provides mathematical model of what is likely to occur if all possible samples analyzed If a large number of samples were drawn from the frame: A mean can be calculated for each sample Each sample mean would not be exactly the same value as others Means would be different from one another but would cluster around the frame s central value (or mean) Differences in the means of different samples are basis of the error that occurs inferring from a sample to the frame rather than measuring all of the claims in the frame If repeated random samples (moving toward infinity) were made, means would be expected to fall into a normal distribution 20

21 Possible Samples That Can Be Randomly Drawn From the Frame 21

22 What s the Big Deal About the Normal Distribution? In probability theory, the normal (or bell-shaped) distribution is continuous probability distribution (a function that tells the probability of a number falling between any two real numbers). The normal distribution is symmetric around the mean. The mean, median and mode are the same number. The normal distribution is immensely useful because of the Central Limit Theorem which states that the mean of many random variables independently drawn from the same distribution is distributed approximately normally, irrespective of the form of the original distribution. That is, the overpayment means will be randomly distributed if the sample is large... moving toward infinity. 22

23 Normal Distribution 23

24 Confidence Levels on Normal Distribution Point Estimate Lower Confidence Limit Upper Confidence Limit 24

25 25

26 Illustration not based on actual data Mode Median Where is the Confidence Level? One-sided or Two Mean , ,

27 Distribution of Frame Paid Dollars 27

28 Non Normal Mean and Standard Deviation Sample mean +/- 1 standard deviation Mean = sd = = =

29 Confidence Levels Non Normal Data Point Estimate +/- ½ Confidence Interval Confidence Interval = 16, ,778 = 33,556 Lower 27, ,778 = $43, 925 Upper 27,147-16,778 = $ 10,368 Lower confidence Level??? Unknowable 29

30 Properly Executed Random Sample Prior known error rate or probe Appropriate for use with the audit data (dependent or independent) Of sufficient size to support the statistic used Use a replicatable random process Tested for randomness Is representative (without mathematical bias) and tested 30

31 31

32 Sample Size Determination Based on Chosen Precision and Confidence 32

33 RAT-STATS Results Confidence Level 80% 90% 95% 99% Precision Level 1% % % 29* % 10* 15* 20* 29* 15% 5* 8* 10* 16* * Caution sample sizes less than 30 33

34 Representative Samples A small yet complete and accurate picture of the data in the frame. A subset of a statistical universe that accurately reflects the numerical membership of the entire universe and its distribution. A representative sample is an unbiased indication of what the frame is like. Representativeness is tested mathematically. When a sample is not representative, the result is known as a sampling error. 34

35 Representative Sample Frame sample 35

36 Sobering How Poor a Sample Can Be 36

37 RAT-STATS Overpayment Estimation Formulae: Confidence Level: 37

38 Precision vs Accuracy Precision measures how often the measurement tool produces a similar result every time it is used. Accuracy measures how close the sample value comes to the true value. How close is the sample overpayment mean to the true mean of overpayments in the frame. Sample results can be accurate but not precise or precise but not accurate Extrapolations require that the results meet a predetermined level of both. 38

39 Precision Calculation Point Estimate +/- Precision amount = Confidence Level Precision amount/point estimate = Precision % It is absurd to say the lower confidence level benefits the provider. The accuracy of the estimate is only the top and bottom of the confidence level around the point estimate. The lower confidence level is as close to accurate overpayment that can be obtained. 39

40 Sample of 5 If: A random sample of 5 selects the top most expensive claims (5540, 5550, 5555, 5560, 5570) mean = 5555 Frame of 100 ranges from 500 to 5570 has a mean = 2827 The precision will very small and the confidence levels will be very tight but the point estimate would be nearly twice as large as the frame mean The precision would be good but the accuracy would not Point estimates can exceed the amount actually paid It is a bad estimate. 40

41 41

42 Sample of 5 If: A random sample of 5 selects the top 2 claims and bottom 3 (500, , 5560, 5570) mean = 2539 Frame of 100 ranges from 500 to 5570 has a mean = 2827 The precision will very large and the confidence levels will be very large but the point estimate would be close to the frame mean The precision would be unacceptable but the accuracy would be OK The lower confidence level can be a negative number indicating an underpayment It is a bad estimate 42

43 Poor Audit Design & Execution Produce Only Invalid Results Statistics in the hands of an inept auditor are like a lamppost to a drunk: they are used more for support than illumination. 43

44 Defending Against Extrapolation Results Medicare Appeals Process Redetermination Reconsideration Administrative Law Judge Hearing Medicare Appeals Council Federal District Court Medicaid Appeals Process Appeal rights under state law 44

45 Defending Against Extrapolation Results No administrative or judicial review of determination of high level of payment error BUT determination must be made Failure to follow one or more requirements in Benefit Integrity Manual does not necessarily affect validity Not sufficient to argue better or more precise methods are available 45

46 Defending Against Extrapolation Results Can challenge validity of sampling methodology based on the actual statistical validity of the sample as drawn and conducted Test: Was the sample statistically valid? Contractor has burden of establishing sample was in fact random and statistically valid 46

47 Defending Against Extrapolation Results Procedural Challenges Did the contractor provide work papers allowing for review and replication of sampling results for every stage of the process? Did the contractor follow the guidelines? Medicare: MPIM Medicaid: State requirements Were allowed claims included in overpayment sample calculation? Were calculations performed correctly in the audit and at each level of appeal? 47

48 Defending Against Extrapolation Results Substantive Challenges Need a statistician to make your case. Where can you find one? One size does NOT fit all. It is not your job to explain how it should be done. 48

49 Defending Against Extrapolation Results Examples of Substantive Challenges Is the sample representative? Is the sample statistically significant? Is the sample size reliable? Is the sample within the required precision and confidence levels? 49

50 Hold Auditors Accountable Did they know what should be done statistically? Requires consultation with statistical expert Did they do what is necessary to create and audit a probability sample (not just say they did)? Have to test the data to be sure Did they accurately follow chosen methodology key assumptions (accurately execute the methodology, use proper randomization, use the correct formulae and accurately measure variables of interest)? All of these criteria must be met for extrapolation Did they select a random sample of sufficient size made up of independent observations that are normally distributed, randomly selected and representative? 50

51 Common Substantive Issues Sample size not associated with established precision or confidence levels Incorrect use of formulas Use of wrong formulas - choose wrong method Use of inapplicable methodology simple, stratified, cluster, multi-stage Non-representative sample Fail to meet key assumptions of statistics math basis of statistics Exclusion of zero paid claims Accuracy outside of recommended range too little precision Reporting incorrect precision and/or confidence levels 51

52 Defending Against Extrapolation Results Obtain all documentation related to sampling calculations Consider provider s prior audit history Know appeal timelines and requirements for each level No new evidence after reconsideration absent good cause Request for ALJ Hearing 42 C.F.R (a)(3) Include information on each sample claim to be appealed File request within 60 calendar days after the party receives the last reconsideration for the sample claims Assert the reasons for disagreement with how the statistical sample and/or extrapolation was conducted in the request for hearing 52

53 Defending Against Extrapolation Results Understand reasons for denial at each level Present reasons in written protest or position paper Prepare for oral testimony at ALJ hearing 53

54 Questions? 54

55 Faculty Anna M. Grizzle, Esq. Member Bass, Berry & Sims PLC 150 Third Avenue South Suite 2800 Nashville, TN (615) Pat Maykuth, Ph.D. President Research Design Associates, Inc. 721 E Ponce de Leon Decatur, GA pm@researchdesignassociates.com (404)