GSP revisited: Increasing Value, Reducing Waste. Daniel Strech, MD, PhD

Transcription

1 GSP revisited: Increasing Value, Reducing Waste Daniel Strech, MD, PhD

2 Standard GSP Recommendations Plan carefully! Get ethics review! Do not manipulate, lie, or plagiate! Document and store appropriately! Be fair with authorships! Etc.

3 Good scientific practice - revisited What does good in biomedial science mean? When is research of value? When is research waste? What is the status quo?

4 Objective: Translation of basic biological discoveries into clinical applications that improve human health Series of complex steps Discovery of basic information about pathogenesis Assessment of potential to lead to a clinical advance Development of candidate interventions Optimization of the candidates in preclinical settings Testing in human clinical trials Application for approval for widespread clinical use Assessment of approved interventions in real-world

5 Overall success?

6 Drug Development: Success Rates ,451 drugs from 835 companies and 7,372 independent clinical development paths in 417 unique indications (Hay et al. 2014) LOA = Likelihood of approval

7 Reasons?

8 An unspoken rule among early-stage venture capital firms is that at least 50% of published studies, even those in top-tier academic journals, can t be repeated with the same conclusions by an industrial lab. Amgen: Could only replicate 6 (11%) of 53 landmark cancer studies

9 Objective: Translation of basic biological discoveries into clinical applications that improve human health Good scientific practice Series of complex steps Discovery of basic information about pathogenesis assessment of potential to lead to a clinical advance; development of candidate interventions optimization of the candidates in preclinical settings Regulatory assessment of potential for human use testing in human clinical trials application for approval for widespread clinical use assessment of approved interventions in real-world

10 Publication bias in preclinical research?

11 Researchers (n=454) thought that about 50% of animal experiments are published Employees (n=21) of for-profit organizations estimated that 10% are published

12 Reproducibility crisis?

13

14

15 Robust experimental design?

16 Internal validity Selection bias creating groups with different confounder solved by randomization Performance bias and detection bias investigators assess subjects on the treatment arm more positively controlled by blinding interventions and outcome assessments Attrition bias dropouts of subjects with a negative outcome not included in the final result Modified from Ulrich Dirnagl

17 Random sample from PubMed From Emily Sena

18 Good quality journals From Emily Sena

19

20 Effects of attrition in experimental biomedical research From Ulrich Dirnagl PLoS Biol. 2016;14:e

21

22 Bias against null hypothesis?

23 Bias against the NULL hypothesis p < 0.05 Move on to next experiment, write paper p > 0.05 Repeat experiment Add animals or repeat statistics with different test (e.g. contrast) (i.e. p-hack) Remove outliers (to nudge effect size in proper direction) Try different strategy (antibody, assay, claim that the previous one 'did not work'), etc. Once mission accomplished (p<0.05): don't talk about how you got there. From Ulrich Dirnagl

24 Solutions? Strategies?

25 Top down and Bottom-up

26

27

28 JAMA 2014;312(5):483-4

29 Bottom up strategies

30 Solution I: Publication of Null Results Prevents: Publication Bias

31 Solution II: Pre-Registration Prevents: Outcome switching, Cherry picking of results

32 Solution III: Data Sharing OPEN SCIENCE POLICY: Find, Access, Interoperate, Reuse Data (FAIR)

33 Solution IV: Experimental Design Assistant

34 Solution V: Reporting Guidelines

35 Social Sciences, Psychology, Physics? Most probably the same problem! Increasing evidence! Most probably the same solution strategies!

36 Conclusion Standard GSP guidance is very important but needs to be combined with attempts to increase value and reduce waste in research! Internal validity: Randomization, blinding, sample sizes Pre-registration, non-selective publication Reporting guidelines, data sharing Stakeholder: Institutions, funders, regulatory bodies and you (next generation of scientists)

37 First mover: Advantage or Disadvantage?

38 Thank you very much for your attention! Daniel Strech, MD, PhD