Sample size Re estimation and Bias

Transcription

1 Sample size Re estimation and Bias Medical University of Vienna, January 4, 03 Frank Miller, Stockholm University, Sweden Partly joint work with Tim Friede, Dept. of Medical Statistics, University Medical Center Göttingen, Germany

2 Content Sample size re-estimation, bias and correction Continuous variance monitoring Blinded continuous variance monitoring More insight into the bias and it s consequences

3 Clinical study and sample size calculation Randomized study comparing two treatments (e.g. new treatment versus control) A variable is observed which is assumed to be independently normally distributed with common unknown variance and treatment difference Significance test with level and power for, a H (new treatment equal to control) versus 0 : 0 H : 0 (new treatment better than control) a Sample size per treatment n v, v z z a Since is unknown, an initial guess required

4 Example: MacDonald a et al. (008) Objective: Assessment Lumiracoxib s effect on blood pressure in Osteoarthritis patients with hypertension Treatments: Lumiracoxib or Ibuprofen Primary endpoint: change from baseline at week 4 in average 4 h systolic blood pressure Significance level α=0.05 (-sided), power 80% for mmhg a Standard deviation σ=??? mmhg White et al (00): 9 mmhg observed (but different population) Sowers et al (005): assumed , observed mmhg(6week follow-up) Other studies in non-oa population: up to 4 mmhg

5 Uncertainty in the pa planning phase Total sa ample size Std Dev N Standard deviation

6 Uncertainty about variance a Problem: Often considerable uncertainty regarding the variance Solutions include: One interim look: estimate from accruing data in an interim sample size review and use this estimate to adjust sample size Several interim looks with update of variance estimate and sample size Continuous monitoring of the variance

7 Sample size based on one interim analysis Observe n patients ( Stage ) & estimate variance: Final sample size n vˆ v Observe additional n =n-n patients ( Stage ) ˆ New Control n ˆ n ˆ n The variance estimator ˆ at the end of the trial is biased. It underestimates the true variance!

8 Why is there a bias? ˆ lower than true hard to correct in second stage, since n low New Control ˆ n n larger than true easier to correct in second stage, since n large New Control ˆ n n underestimates the variance

9 Consequence of the bias in the final variance estimator The t-test Reject H t t with 0 n, ˆ t ˆ / n does not control the alpha level

10 How large is the bas bias? Bias of the ( naïve ) variance estimator can be computed Bounds for bias (Miller, 005) recall: n Eˆ v n n v ˆ 0, v z z Additive correction of final variance estimate: a ˆ ac ˆ n n 0,, v if stopped ddirectly after it interimi otherwise

11 Correction of bias Bias of variance estimator Type I error of t-test test and t- test with additive correction

12 Continuous variance a monitoring o Continuous variance monitoring procedure: Monitor variance after each (pair of) patients: starting after patients ( ) n n Stop study as soon as sample size sufficient according to this estimate ( nv ˆn ) This is a stochastic process with stop-timetime N min n n, n,... ˆ n n / Discussed in the context t of clinical i l studies e.g. by Mehta & Tsiatis (00) and Jennison & Turnbull (007) Investigated by Friede & Miller (0) v ˆn

13 Continuous variance monitoring as stochastic process Example study with true variance=, v= v z z v.0 for 0.05, 0., a a First time under linear boundary stops study E( ˆ N ) (negative bias of variance estimator) t

14 Bias of variance estimator and test Bias of variance estimator Bias of test size ˆ N Variance estimator at stop-time N negatively biased Therefore, the t-test with t ˆ Reject H ˆ / 0 t tn, N does not control the alpha level N

15 Blinding in clinical ca trials as Randomized clinical trials for drug development are usually blinded Database of ongoing study has no treatment information Separate file with treatment info kept secretly Database Patient Treatment Result Result... * * * * * Treatments Patient Treatment Placebo Active 3 Active 4 Placebo 5 Active

16 Blinding and sample size red g a d sa pe s e e estimation To perform the sample size re-estimation shown before, the treatment of all patients needs to be known for the computation of the variance known for the computation of the variance (unblinding necessary) Regulatory authorities prefer methods not requiring Regulatory authorities prefer methods not requiring unblinding n n j j j j n X X X X n ) ( ) ( ˆ n j n j n, X X X X n blind ) ( ) ( ˆ j j n

17 Blinded continuous monitoring o The different sample size re-estimation procedures can be performed blinded Here we show the blinded version for continuous monitoring Blinded d continuous monitoring i procedure: Monitor overall (one sample) variance ˆ n,blind after each (pair of) patients, ignoring the different treatments Stop study as soon as sample size sufficient according to this blinded estimate ( n ˆ n, blind v ) Estimate final variance unblinded

18 Bias of variance estimator and test blinded procedure Bias of variance estimator Bias of test size Still bias for variance estimation but essentially no bias for test.

19 Blinded procedure: Why is the test size (almost) unbiased? t ˆ ˆ N N / N F N ˆN ˆ N If the (two-sided) test is written as F-test, the numerator and the denominator have the same bias after blinded continuous monitoring

20 Power of test after continuous monitoring Unblinded continuous monitoring Blinded continuous monitoring Desired power approximately maintained; better so for the blinded procedure.

21 Blinded procedures: es Continuous monitoring versus sample size reestimation with one interim look 0.05, -ββ 0. 90,δ δ a One look after n =0, 50, 00, 00, or 400 patients per treatment to estimate variance Compare with continuous monitoring (CM) If n chosen too large: risk to overshoot necessary sample size If n chosen too small: SD for sample size can be considerably higher than for CM

22 Continuous monitoring o is it logistically feasible? Increased use of electronic data capture techniques in clinical studies E.g. in chronic pain studies Patients are provided with electronic diaries for their daily pain recording SMS reporting has successfully been applied (Axén, Bodin, Bergström et al, 0) Pain intensity ratings can immediately be transferred to central database of sponsor Continuous monitoring feasible in some (but not all) clinical study situations

23 Back to the bias for sample size re- estimation with one look Bias of variance estimator Type I error of t-test test and t- test with additive correction

24 Bias for unblinded case with IA Bias of variance estimator n n Eˆ v 0 Exact bias: where F is the chi-square distribution function and

25 Bias for unblinded case with IA Absolute bias of variance estimator Relative bias of variance estimator n n Eˆ v 0 n n v E ˆ 0

26 Bias for unblinded case with IA Absolute bias of variance estimator n n Eˆ v 0 Assumptions for this graph Sample size in first stage n =0 per group Minimum sample size in second stage: n min =0 α=0.05, 0 05 power=90%, δ a =. a v z z Sample size formula for interim N max v ˆ, n n min How does the bias depend on n min, n, δ a?

27 Bias for unblinded case with IA Absolute bias of variance estimator Probability for stopping the study with minimal sample size n +n min E ˆ P(N n n min )

28 Bias for unblinded case with IA Bias of variance estimator, n min =0 Bias of variance estimator, n min =0

29 Bias for unblinded case with IA Bias of variance estimator, n =0 Bias of variance estimator, n =40

30 Bias for unblinded case with IA Bias of variance estimator, n =0 Bias of variance estimator, n =5

31 Bias for unblinded case with IA Bias of variance estimator, v=4.34 (α=0.05, β=0., δ a =.) Bias of variance estimator, v=.34 (α=0.05, β=0., δ a =3) Maximal relative bias for both cases ~.5%

32 Dependence of bias on parameters The absolute bias for large variances depends almost not on n and not on the minimum number of patients in Stage The relative bias is largest if there is no minimum number of patients for Stage and n is small; however, decreasing n to very small values (e.g. from 0 to 5) does not increase the relative bias much The relative bias in the considered scenarios was at most 4%

33 Summary Sample size re-estimation can ensure appropriate power (neither over- or under-powered) even under uncertainty of nuisance parameters The effects on estimates and tests are usually small and often, they might be totally acceptable In specific situations (small studies) it might be worth to investigate the bias further

34 References eee Axén, Bodin, Bergström et al (0). The use of weekly text messaging over 6 months was a feasible method for monitoring the clinical course of low back pain in patients seeking chiropractic care. J. Clin. Epidem. 65: Friede & Kieser (003). Blinded sample size reassessment in noninferiority and equivalence trials. Statist. Med.,, Friede & Miller (0). Blinded continuous monitoring of the nuisance parameter in clinical trials. J. Royal Stat. Soc. Series C, 6, Jennison & Turnbull (007). Adaptive seamless designs with control of the type I error rate. J. Biopharm. Statist. 7:35-6. MacDonald et al (008). Effect on blood pressure of lumiracoxoib versus ibuprofen in patients with osteoarthritis and controlled hypertension: a randomized trial. Journal of Hypertension 6: Mehta & Tsiatis (00). Flexible sample size considerations using information-based monitoring. Drug Inf. J. 35:095-. Miller (005). Variance estimation in clinical studies with interim sample size re-estimation. Biometrics, 6, Siegmund (985). Sequential analysis. Springer.