Introduction to The Forensic Statistical Tool (FST) Richard Torres aid.org

Transcription

1 Introduction to The Forensic Statistical Tool (FST) Richard Torres aid.org

2 Evidentiary DNA profiles can be messy How many people are in this mix? Peaks below OCME s 75 RFU threshold Potential Drop In, contamination, stutter, etc. Drop Out

3 OCME Created a DNA Interpretation Software to Try to Solve the Complex Mixture Problem Type of program is commonly called probabilistic genotyping FST is an example of a semicontinous model (does not incorporate peak height information). Reports a likelihood Compares probability of observing the mixture given two different hypotheses

4 What does FST (try to) Say? The suspect is a contributor to a mixture vs The suspect is not a contributor, and an unknown, unrelated person is the contributor The DNA mixture found on the Dunkin Donuts Cup is 12.7 million times more probable if the sample originated from Richard Torres and one unknown, unrelated person than if it originated from two unknown, unrelated persons.

5 FST Results Are Always Reported In Suspect File (FBS15 ######)

6 FST Report From a Case File If the third line is used then that means they are using Low Copy Number aka High Sensitivity Testing

7 Admissibility Rulings on FST Two Frye Hearings People v. Rodriguez, NY. Co. Ind. No. 5471/2009 (Sup.Ct., N.Y. Co. October 24, 2013)(Carruthers, J.) People v. Collins, 49 Misc.3d 595 (Sup. Ct., Kings Co., 2015)(Dwyer, J.) Lots of decisions without having an actual hearing allowing FST: People v. Carter, 50 Misc.3d 1210(A)(Sup. Ct., Queens Co., 2016)(Schwartz, J.) People v. Debraux, 50 Misc.3d 247 (Sup. Ct., N.Y. Co., 2015)(Kahn, J.) People v. Carlton Graham, Ind. No. 2784/12)(Sup. Ct., Bronx Co., Feb. 20, 2015)(Benitez, J.) People v. Pelt, Ind. 2607/13 (Sup.Ct. Queens County 2015)(Aloise, J.) People v. Julio Lebron, Ind. 791/14 and 1733/14 (Sup.Ct. Queens County 2015)(Lewis, J.) People v. Talik Lyons, Ind. No. 1433/13 (Sup.Ct. Queens Co., 2015)(B.Schwartz, J.); People v. Belle, 47 Misc.3d 1218(A)(Sup. Ct., Bronx Co., 2015)(Fabrizio, J.) People v. Styles, 40 Misc. 3d 1205(A)(Sup. Ct., Kings Co., 2013)(Donnelly, J.) People v. Whetstone, Ind. No. 5728/12 (Sup. Ct., NY Co., June 17, 2013)(White, J.) People v. Foster Bey, Ind. No. 9239/10 (Sup. Ct., Kings Co., May 22, 2013)(Goldberg, J.) People v. Bowens, Ind. No. 336/11 (Sup. Ct., Richmond Co., April 29, 2013)(Rooney, J.) People v. Caballero, Ind. No /11 (Sup. Ct., Queens Co., April 15, 2013)(Knopf, J.) People v. Brissett, Ind. No. 847/10 (Sup. Ct, Bronx Co., February 25, 2013)(Webber, J.) People v. Garcia, 39 Misc. 3d 482, (Sup. Ct., Bronx Co. 2013)(Iacovetta, J.) People v. Wortham, Ind. No. 3148/2011 (Sup. Ct., NY Co., September 6, 2012)(Ward, J.)

8 Rodriguez Decision focuses more on whether likelihood ratios are accepted than whether the models used in FST are. Great weight was placed on: Approval from the DNA Subcommittee Approval by National Forensic Science Technology Center s Audit of OCME and Approval of FST Two published articles by FST developers Hubris court tries to be a scientist and gets things wrong With FST the assessment is based upon the amount of DNA shown to be actually present during the quantitation stage of analysis. Drs. Rudin and Lohmueller, on the other hand, testified that the peak heights appearing in the electropherogram alone provide an accurate reading of the amount of DNA present. Even before the advent of FST and similar software programs, forensic scientists knew the steps necessary to analyze complex DNA mixtures.

9 Collins a Nut Shell Who is the scientific community? Are Judges the community? Judges should be counting scientists' votes, and not verifying the soundness of a scientific conclusion. p. 603 DNA Subcommittee or Commission on Forensic Science? no state subcommittee can be equated with the general membership of the relevant scientific community. Will we next consider the matter closed, because the members of a committee in Idaho or Florida approve of a procedure? p

10 Collins in a Nut Shell Who is the scientific community? Validation Studies? [A]a laboratory's satisfaction with its validation results does not show general acceptance of techniques and procedures, if the validation studies fail to create such general acceptance. p. 614 Audits? As to audits, they appear to test whether procedures are being implemented in accordance with protocols, not whether the principles underlying the procedures are valid. p. 614

11 Collins in a Nut Shell Cited critiques from the field Black Box FST is,..., truly a black box a program that cannot be used by defense experts with theories of the case different from the prosecution's. p. 618 drop in and drop out rates number of contributors relatedness fixed thresholds Quant No one else is using Quant to estimate Drop Out and Drop In. They are using peak heights. p

12 Collins in a Nut Shell Cited critiques from the field Sufficient Research Error rate was calculated for the Asian population ( from Hokkaido to Ceylon and from Java to Mongolia ) using only 3 profiles. p. 620 Analysts analyzing the unanalyzable A program like this should not rely on analyst s estimate the number of contributors. And a mistaken estimate comes with serious consequences. Mixtures created by three or four contributors result in four times as many false positive assessments as do mixtures created by two contributors. p. 620 False Positives FST is missing variables that effect false positives. There is no way for a defense expert to plug them in. p. 620

13 Collins in a Nut Shell Other Points by Dwyer Not a decision against Likelihood Ratios or other software. P. 623 Who cares that the defense is using FST in other cases where it is helpful to them. P. 624

14 10 Problems With FST 1. Drop Out model is a fundamental aspect of FST. The Underestimating the Drop Out Rate is Conservative study went unrecorded. Q: So where is the data from that test? A: I don't have it to give you. Q: I mean physically where is it? A: I've done runs and looked at the results, I don't that I've saved everything. Q: And you didn't publish on those results? A: No, this is not published Q: Has anybody published a paper on that hypothesis? A: I can't think of one. (Adele Mitchel)

15 10 Problems With FST 2. No one measures drop out by measuring quant for a reason: The Quant Value is Directly Related to Drop out study went unrecorded. Q: You did a whole study about changing the quantity of D.N.A. input to see how that could affect the likelihood ratio and didn t put that in the validation study? A: I didn t do a formal study like that. I ve altered the input value to see how it changes, but I haven t put together a formal binder of results. No. (Adele Mitchell) 3. OCME had its own definition of conservative. A: Not that a lower drop out rate will always give a lower LR, that s not what I mean by conservative. Q: It depends on the sample, right? A: It depends on We want a low drop out rate for non contributors so by choosing a lower we want a low LR for non contributors so by choosing a lower drop out rate we are erring on the side of lower LR for a non contributor. (Adele Mitchell)

16 10 Problems With FST 4. FST Validation High False Positives In the FST false positive study, the OCME used 439 different mixtures, and ran each mixture against a database of 1,300 profiles at total of 480 times (or 480 bulk runs ). In roughly 10 percent of those bulk runs, the mixture would have at least one false positive above a known contributor. Most of those runs were mock touch samples 5. FST Validation Needle in the haystack. There are instances in the validation studies, 3f and 2c, where the same input was put into FST and different outputs came out of the software. OCME acknowledged this but blamed it on an error with the bulk calculator rather than what is being used in casework. This means what is being used in casework is not identical to what was validated.

17 10 Problems With FST 6. Never Road Tested Pristine lab samples do not simulate real world drop out. DNA can be damaged and inhibited by a variety of factors: microorganisms, UV from sun or artificial lights, moisture, cross contamination, clothing dies, etc. The items used for the validation did not adequately represent the variety of ways drop out happens. Estimating DNA drop out on lab created samples by measuring quantity of DNA does not translate to it works with real world DNA samples. 7. Bad Assumption: Unrelated Individuals Make Up the Mixture Limitation of FST is that all contributors are assumed to be unrelated Other approaches, including LikeLTD and LRmix, take interrelatedness into account 8. Code Unchecked Blackbox Not examined outside of OCME, yet Will not release the source code

18 10 Problems With FST 9. No Performance Testing Performance testing = false positive test Important because it verifies what the strength of the evidence is in relation to a database of noncontributors Should be done on the evidence mixture, not on the model Gill: It is necessary in casework 10. No Test Driving OCME will refuse to allow you test out other hypotheses (morgue DNA profiles, lab workers profiles, viewing as a three person,. ) In the words of one of the other Probabilistic Genotyping developers, Charles Brenner: Prosecution & defense may hypothesize differently. Surprisingly, the number of contributors most favorable the defense to assume is often different from that for the prosecution. Insisting on the same number for both as other programs do is therefore wrong mathematically and prejudicial.

19 FST Reverse engineered FST LAS reverse engineered FST. Other contributors and other hypotheses can be tested out. engineered_quant Based_Tool/tree/master/reQBT%20Express Sorry, no comprehensive validation studies were done on our soy FST. It can be used to get a feel for what FST will say in a case. Use it gauge whether you want to ask OCME to run FST on a different number of contributors in the hypotheses. Use it to see what effect using non called alleles may have. Have your expert check it out and see if it is helpful. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

20 Materials That You Must Have for Trial Free Stuff Good for Learning DNA for The Defense Bar Forensic Bioinformatics Videos Free Stuff Good for Learning and for Cross Exam DNA Technology in Forensic Science (NRC 1); The Evaluation of Forensic DNA Evidence (NRC 2) Quality Assurance Standards for Forensic DNA Testing Laboratories OCME s manuals SWGDAM s manuals Phone call to LAS DNA Unit

21 Materials That You Must Have for Trial Books you must buy to learn this stuff and to cross examine: John Butler s Three Books: Fundamentals of Forensic DNA Typing Advanced Topics in Forensic DNA Typing: Methodology Advanced Topics in Forensic DNA Typing: Interpretation Forensic DNA Interpretation, John Buckleton, Jo Ann Bright and Duncan Taylor