22 Sep Evidence Doesn’t Lie … Or Does It?

Forensic science is great for TV ratings, but not much else. A report released by the White House this week reveals the results of a study conducted by the President’s Council of Advisors on Science and Technology (PCAST). This study on the scientific validity of forensic evidence techniques stunningly debunks many foundational beliefs about DNA, fingerprint, bitemark, firearm, footwear, and hair analyses. The report “concludes that virtually all of these methods are flawed, some irredeemably so,” and strongly recommends the swift implementation of new standards for validating methods, training examiners, and guaranteeing the objectivity of forensic labs by making them independent of police and prosecutors.

DNA analysis received the best grades in the study, which found it to be a foundationally reliable method, particularly with single-source samples or simple mixtures of two individuals (as with rape kits). However, numerous opportunities for human error certainly exist, from mixing up or contaminating samples to incorrect reporting and interpretation. These opportunities increase proportionally to the number of samples being tested. For example, a complex mixture of blood stains can contain samples from many unknown individuals in unknown proportions. The more complex a mixture is, the more likely that subjectivity enters into analysis – the expert may choose to include only certain samples, the expert may already be focused on a particular suspect’s DNA, and so on. When deciding the validity of any DNA analysis, the Council stresses, the importance of the following disclosures cannot be underestimated –  an expert’s performance on proficiency tests, his or her awareness of any influencing facts of the case, and information about his or her lab’s past quality testing.

Unlike DNA analysis, bitemark analysis fell far short of meeting the scientific standards for foundational validity. Not only are bitemarks difficult to uniquely distinguish, human skin as a medium for bitemark impressions is altogether unreliable, distorting and stretching any potential bite evidence. Among the very few empirical studies undertaken on bitemark analysis, false positives in identifying the source of a bitemark were so high as to invalidate the method altogether. Much of the existing scientific data also suggest that experts are even unable to agree with any consistency on whether a mark was caused by a human being at all.

Perhaps the most shocking of all of the PCAST revelations are those dealing with latent fingerprint analysis. In evidentiary employ for over one hundred years, fingerprinting has been historically lauded as an infallible science. That this reputation has developed in spite of the scarcity of actual empirical proof is, at best, surprising, and at worst, unconscionable. Astoundingly, the first scientific black-box study of fingerprint analysis was not published until 2011 by the FBI Laboratory, and the only other similar study was performed in 2014 but has yet to be published. The PCAST report states it best when it says, “Overall, it would be appropriate to inform jurors that (1) only two properly designed studies of the accuracy of latent fingerprint analysis have been conducted and (2) these studies found false positive rates that could be as high as 1 in 306 in one study and 1 in 18 in the other study. This would appropriately inform jurors that errors occur at detectable frequencies, allowing them to weigh the probative value of the evidence.”

Though theoretically reliable, the application validity of fingerprint analysis is highly impacted by circular logic and confirmation bias produced when examiners alter the features marked in a latent print to match them with a suspect’s print or a print from a database. Contextual bias is also common, arising from judgments based on irrelevant facts from the case. Brandon Mayfield of Portland, Oregon, whose fingerprint an FBI examiner erroneously matched with “100% certainty” to one recovered from the 2004 Madrid train bombing, ultimately got a $2 million settlement from the U.S. Government. Others sitting in prison today were not so fortunate.

As with fingerprint analysis, firearms analysis has only recently had its reliability subjected to legitimate empirical testing. The firearms analysis field is based upon the idea that toolmarks made on fired cartridges by different firearms vary substantially. In the past fifteen years, a number of studies have shown that forensic examiners can identify the source weapon of particular fired ammunition, however many of these studies used staged scenarios unlike those encountered in actual casework. Hence, PCAST reported that this forensic discipline falls short of the criteria for foundational validity, primarily due to inadequate research. It is currently left to the courts to decide whether firearms analysis should be admitted. When it is allowed in court, the jury should be made aware of the error rates seen in the one appropriate black-box study – estimated at 1 in 66, with a 95 percent confidence limit of 1 in 46.

PCAST’s conclusions on footwear analysis will read familiar. “There are no appropriate empirical studies to support the foundational validity of footwear analysis to associate shoeprints with particular shoes based on specific identifying marks (sometimes called ‘randomly acquired characteristics’). Such conclusions are unsupported by any meaningful evidence or estimates of their accuracy and thus are not scientifically valid.” In other words, don’t bet the farm on those “ugly-ass” Bruno Maglis.

Undoubtedly, this report will have a profound preventative impact on the incarceration and execution of innocent people. The hard truth is that “faulty forensic evidence has been found in roughly half of all cases in which post-conviction DNA testing has led to exoneration.” Whether you are going to criminal trial or heading to family court, the conclusions of the President’s Council will have an immediate impact. Lawyers across the land will challenge prosecution experts with this data, and we will find ourselves operating in a more just system. We here at Helmers+Associates are filing motions right this minute supported by information gleaned from this ground-breaking report.