Building More Reliable Forensic Sciences (Part Two)

SPEAKER 1: Welcome to "Justice Today," the official podcast of the Department of Justice's Office of Justice Programs, where we shine a light on cutting-edge research and practices and offer an in-depth look at what we're doing to meet the biggest public safety challenges of our time. Join us as we explore how funded science and technology help us achieve strong communities.

JAMES DAWSON: Hello. I'm Jim Dawson, the Senior Science Writer with the National Institute of Justice, and this is the second half of my conversation with David Stoney and Greg Dutton about the long transition to a more reliable forensic science. David holds a Ph.D. in Forensic Science and is the Chief Scientist and Head of Stoney Forensic in Chantilly, Virginia, and Greg is the Program Manager and Physical Scientist with NIJ's Office of Investigative and Forensic Sciences. 

In part one, we discussed the 2009 National Research Council report that was highly critical of forensic science and the impact that report had on law enforcement crime labs, courtrooms, and the broader scientific community. We also discussed how forensic science is improving with more research being done in the field and with more input from the broader scientific community. Lastly, we discussed resistance to change within the forensic community. Part two picks up with issues surrounding specific types of forensic evidence, such as fingerprints. 

Let me--let me try to get specific for what we're talking about with evidence here for just a minute. Most people that are not forensic specialists or not scientists, we've been raised with the notion that--let me look at fingerprinting just for a moment, because I understand there are some issues that have gone on, at least discussions about that. You know, I grew up with the idea that there are no two people on the planet that have the same fingerprint, and if I leave a good fingerprint at a crime scene, then that that's a good way to identify me absolutely, and I understand like the Defense Crime Agency a number of years ago said you cannot say a single fingerprint belongs to an individual. You have to start giving probabilities. You have to make it more scientific. You just can't assert that it's true. And using that example, maybe of fingerprints, because most people are familiar with how that's done, what--what kind of controversies do you face as a scientist looking at a specific thing like that? What are the issues?

DAVID STONEY: Well, I think it's useful to contrast. Fingerprints are an excellent example. We could contrast the approach to fingerprints that was the way everyone was doing it before the turn of the century to what's happened since that time. The prior approach to fingerprints was based on expert examination, individual expert examination of prints. Experts were trained by apprenticeship. The examination methods were visual, so this would be a side-by-side comparison, typically without making any notes and without retaining any other records. This is a method that was found very effective. The results that were formed is conclusions. You have an identification, like you alluded to, it's his print, or an exclusion, it's not his print. And otherwise, there'd be no opinion that would be of no value for identification. There wasn't any middle ground. 

There's different ways to look at that. One way to look at it that's important is the profession was operating to shield results such as "Well, it might be his print. I can't really tell for sure, but it sure looks like him." That was unacceptable in the profession. And as a whole, the body of fingerprint examiners refused to say anything like, "it might be his print." And within the profession, that was viewed very, very unambiguously as a shield against abuse of fingerprint evidence. Well, the results were then were checked by another expert colleague. But then they were reported, like you say, as absolute truth. I am 100% certain this fingerprint came from you and no other person in the world made this print. 

Now, from the standpoint of a defense attorney or, you know, a scientist looking from outside, they'd say, "Well, how do you know?" And the response that was accepted and given was, "Well, I see it when I see it. If you want to test that, get your own expert to look at it." If you say, "Well, where's your work product?" You know, show your work as in all science, and say, "Here's my work," and they'd show up a picture of a fingerprint. "No other expert will disagree with me." "Well, how do we know it's reliable?" "Well, I've been doing this for this many years and I've never made a mistake and fingerprints have been used for this many years and no two prints have ever been found that are alike."

Now, there were errors under that system, but they were treated this way: the individual expert made a mistake. No good expert would make a mistake. And the expert under that system was excommunicated. Fingerprints are still a hundred percent certain, just sometimes there are bad eggs that don't do their job. So, in that way, it created this situation where the system worked in a sense, but when scientists started looking at that practice, it was very, very abrupt. “Wait a second. This isn't good. Why isn't it good? Well, scientific approach, first of all, is open. You have defined procedures. You make observations. When you make observations, you don't just keep them in your head. You collect data and you keep records so it's reviewable by another scientist. Then after you have your data, you interpret the data. And from that, you get your conclusions. Now when you do it that way, you can test it, you can see: do these conclusions--with this data, do these conclusions end up being true? So, scientists will test that. So, when an outside scientist looks at what we were doing in fingerprints and says, "Wait a second. There's no well-defined method. You're not keeping a record of your observations. You're not make any measurements." Most observations, any time a scientist can, they'll make a measurement. And you're not testing the conclusions. And these are basic failures from any scientific approach. They aren't failures of fingerprints, they're failures with the way we were doing fingerprints. 

And if we look today, right, so what's changed? We are working—and this is exactly what OSAC's doing, exactly what the focus of the profession is now: defining our procedures, keeping very good records, and testing whether when we apply this well-defined procedure, the results that we report are accurate. And we're testing whether or not we're doing a good job. So throughout all that, right, what you grew up with, fingerprints are absolute, it's him, and their reliable, fingerprints haven't changed. When you look back from it and say, "What kind of evidence can they provide?" It's going to be the same kind of evidence. It's going to be phrased differently, and it's also going to be more reliable. OK, the reliability, which is what we're talking about here, is increasing reliability of forensic sciences. Reliability and openness of the methodology has changed.

JAMES DAWSON: So, an expert now testifying in court would say what now that would be different from what he said 10 or 20 years ago? A fingerprint expert. If he's on the stand, how would he present it in the new way?

DAVID STONEY: Well, there are different ways that folks will do it right now. And we are in a--in a period of transition. But the aspects of it that were not present before that would be included is, "Well, can I see your notes? Can I see your data? What were your observations?" And that's open, which is a scientific practice, and others can review it. Then another aspect would be, "Well, what procedure did you use?" "I followed this procedure as recommended by colleagues and doing this." "What safeguards are there in that procedure?" You can fully discuss that. Then you could say, "Well, has your procedure ever been tested?" And we can say, "Well, yes, it has. There have been specific research studies where people have applied this well-defined procedure under situations where it could be shown to be an error. And it has been tested. It has been found to perform well." OK, then that's all of questions that could be asked about what did you do, and what procedure did you follow. Then they get to the crux of the matter. “Well, what’s your opinion? Is it this guy’s print or not?” And people will answer that differently. How would I answer it? I would answer it, “Well, I can tell you what my opinion is. I can tell you that if I was making a decision based on whether or not this evidence shows that this print came from that person, I would be entirely confident making the decision that this print came from that individual. Does that mean no other individual in the world could possibly make a print that could be confused with them? I’d say no that would be possible, but, in this case, the print is of such quality and such an extent, it's big enough and it's clear enough, that I think it would be unusual to the point of being untruthful to you to suggest that it could have come from someone else."

JAMES DAWSON: And I want to mention here now, and correct me if I'm wrong, but the fingerprints that are often involved in cases are not the kind you see rolled out in the police department, where they take your fingerprints. They're latent prints, which are partial prints, smeared prints, that kind of thing. So, the reality in law enforcement is most of the time, you're not dealing with perfect fingerprints to start with, right? So that that adds a layer of complication to this.

DAVID STONEY: That's true. And recall, what I said earlier, what used to be done is unless it was pretty much a perfect print, you never saw it in the courtroom. At least that's the way it was intended. The problem was, and it was a problem, is that that was dependent on an individual's judgment, or one or two people's judgment, whether that was a super good print or whether it was just an OK print.

JAMES DAWSON: How are courtrooms reacting to this, judges specifically? Or is there a movement among some judges to push for a more scientific basis? I've seen that in some cases for evidence being presented. Are they allowing this new approach or this--the expert challenging experts to go forward? Have you seen that in courtrooms? How is this changing courts?

DAVID STONEY: The standards pay a very--play, very clear role in this. I can give you one experience of a case that I was in, and it was a federal case, a very well-informed judge, and there was an issue of a laboratory that was doing things primarily the older way that I described. And meanwhile, all these changes were going on. And so, one could say, well, look, like I went through, “there isn't a well-defined method, there's no recording of observations, there's no measurements.” And the very well-informed federal judge said, "Well, I understand it. That's the way Dr. Stoney likes to do it. And this other crime laboratory does it a different way. Show me a standard that says it should be done Dr. Stoney's way. And in the absence of that, it's been done this way. It's been accepted before. There's no problem." You know, I can see that as a reasonable judicial response. And what they're saying is, if you guys as a profession really think it should be done the other way, show me the standard.

JAMES DAWSON: Ok. And you mentioned in one of our discussions on this that--I think you were noting that because it can be a jury trial, that kind of thing, there are other people looking at this and it isn't--in the end, the courtroom is not a scientific arena. In other--there are other things going on there. So it may be, you know, the color tie the expert's wearing, your attitude, that kind of thing influences it. So how do scientists approach courtrooms and deal with that kind of issue when they're going in there with what they believe to be facts? Do they need to do any mental adjustment before they go in and start dealing with the law, dealing with courtrooms and criminal procedures?

DAVID STONEY: I think the main thing they need to be as humble. Science is invited into the courtroom when it's needed. The second thing they need to do is to not be intimidated by it. And--mean, it is our job. And, certainly, the way I was specifically taught is it's my responsibility to make sure that a false impression about the evidence is not planted in anyone's mind. So that's--I'm there cause I was invited and I'm there cause they have questions about science. And for that last part, to not plant a false impression in the minds of the jury or the judge, you have to communicate. And the communication is something that is very, you know, in any arena where you're trying to talk to somebody, you have to allow some give-and-take with the language. You have to be able to look at the jury and say, "Are they gettin it?" You have to be able to say, "Let me phrase that a couple of different ways here because I want to make sure I'm not--you're not hearing me say this, you are hearing me say that." So, with your fingerprint example, if I'm saying, "Well, no, I can't say I'm 100% certain it's his print," that's something I'm going to explain when I'm in court, so that I'm not giving them the false impression. It doesn't mean fingerprints are bad. It doesn't--what it does mean, I haven't compared it with everybody in the world. And here's the strength for saying it's this person. Here's the argument against it. Now, you know, that may be a lot more than they want to hear, but that's the--that's the task in the courtroom is to be a responsible guest and to communicate.

JAMES DAWSON: Ok, let me turn to Greg for a minute cause, Greg, you're overseeing a lot of the grants and deciding on which research gets supported in looking at this problem. And I know it covers a number of agencies, but it also covers a whole lot of fields. The trace evidence, I know you're an expert on that, arson, ballistics, everything else. How do you see the issue? And how do you decide what gets supported with limited funds that the federal government has for this kind of thing?

GREGORY DUTTON: Yeah, so it's not an easy task, because as you say, we—you know, we have limited resources. We only have so much money, and we’ve got a lot of disciplines to support. I will say that it’s certainly important to us to build on the strengths of other federal agencies. So, we've already talked about NIST and the NIST OSAC, which is tremendous. NIST has really been increasing their own forensic science research that they do internally. So, you know, I think it's a matter of assessing what's being done, where, how we can best put our resources to fund R&D, that needs to be done. NIJ recently, just earlier this year, we put out a Forensic Science Strategic Research Plan to put out for the next five years the priorities that we have that we've seen for research and development of forensic science. So, we're depending on the plan. And so, I think we, you know, we want to support an ongoing program in forensic science R&D, and we do that as best we can with our researchers and with our federal partners. So, it's an ongoing effort.

JAMES DAWSON: Right. And so, I guess for both of you, you've got a 5-year plan. At the end of 5 years, how is this going to have--is it going to be gradual improvement in different fields? Are we going to have broad acceptance in crime labs? Is the world going to have changed a bit over time for this?

GREGORY DUTTON: Well, I’ll hope that we have made progress in that time. I'm sure we will. But, as David said at the outset, you know, it’s an ongoing process. There is no end point. So, you see where the needs are as you go, and you try to address those problems and--and move on to the next.

JAMES DAWSON: David?

DAVID STONEY: I could add this, that it's not--it's not a box you can unopen. There was a situation last century, the latter half of the century, where the scrutiny being applied by the external scientific community on whatever forensic scientist was doing was absent, essentially absent. And now it is present, and it--that's not going to stop. It is a legitimate academic pursuit to study forensic sciences and to look at the procedures and critique them. That's perfect. We needed that. And it's not--not gonna go away.

JAMES DAWSON: All right, gentlemen. I appreciate both of you being here. Thank you very much for doing this. And, David, I--will continue to follow your work. And, Greg, I'll continue to follow your grants. So, I appreciate it. And I want everybody to remember to tune in and watch for the next NIJ podcast. Thank you all very much.

SPEAKER 2: To learn more about today's topic or about NIJ, visit the links in the episode description and join us for new episodes every month.