Enhancing fairness in DNA jury trials

Foreword | Controversies over how the law should regulate the presentation of expert testimony on DNA forensic science were explored in an experimental study comparing traditional verbal with audiovisual modes of delivery. Pre-trial DNA knowledge, as assessed in 3,611 jury-eligible Australians, was limited. From this group, 470 citizens watched a simulated homicide trial containing a cognitively-sequenced generic tutorial on DNA profiling evidence. The expert tutorial significantly improved DNA knowledge, irrespective of the mode of presentation. Given clear and well-sequenced complex information, lay jurors dealt competently with it. Only jurors with little knowledge of DNA who were exposed to verbally presented evidence deferred to the expert’s views. The multimedia tutorial effectively improved the decision-making of people whose comprehension of the scientific evidence was lowest, bringing their verdicts in line with those whose understanding of the evidence was most accurate. Enhanced DNA knowledge increased scepticism about experts and reduced convictions. These results will aid courts and policymakers in adopting procedures to enhance justice in criminal cases in which DNA is introduced.

Adam Tomison

Director

Use of DNA evidence in Australian courts has increased exponentially since 1989 (Easteal & Easteal 1990; Walsh et al. 2004). After 20 years, DNA technology is well-tested and is no longer the subject of defence challenges (Haesler 2008).

Increasingly, a single forensic expert guides the jury through the DNA evidence; in five out of six DNA cases studied in New South Wales, this was the model (Findlay 2008). The presence of DNA evidence predicts convictions. Archival research revealed that juries were 23 times more likely to vote guilty in homicide cases and 33 times more likely to vote guilty in sexual assault cases when DNA evidence was admitted (Briody 2004). Concern has arisen that the safety of these verdicts may be compromised by widespread misconceptions about the infallibility of DNA evidence (Gans & Urbas 2002) by jurors who are 'overawed by the scientific garb in which the evidence is presented and attach greater weight to it than it is capable of bearing' (R v Duke 1979 22 SASSR 46 per King CJ: 48). Field studies, interviews with actual jurors and jury simulations confirm that individual jurors struggle to understand and apply the statistical information conveyed by forensic experts about the likelihood that the DNA match occurred randomly, by chance alone (Findlay & Grix 2003; Lieberman et al. 2008; Schklar & Diamond 1999; Wheate 2006).

The problem with misunderstood evidence is that it may compromise justice. Post-trial interviews of jurors who served on six criminal trials in New South Wales disclosed that jurors who admitted difficulty understanding DNA expert evidence nevertheless proceeded to convict (Findlay 2008). To minimise biases, investigation of methods to facilitate juror understanding of the probative value of DNA evidence was identified as a crucial area for empirical research (National Research Council 1996). Attention first focused on a traditional legal safeguard—jury directions. In Australia, the judiciary were advised to 'develop a model jury direction for use where DNA evidence has been admitted in criminal proceedings' to aid judges and juries in evaluating DNA evidence (ALRC 2003: Rec 44–2). However, one controlled experiment testing the Australian Law Reform Commission's model instruction showed it was ineffective (Dartnall & Goodman-Delahunty 2006). Jury deliberation has also proved inadequate (Findlay 2008; Wheate 2006). Jurors' educational levels were more substantial contributors to comprehension than deliberation (Dann, Hans & Kaye 2007). Recently, attention has shifted to features of the expert evidence (Bornstein 2004), to reduce misplaced reliance by jurors on pre-existing beliefs about DNA profiling and to enhance jury comprehension (Edmond & Mercer 1999). In a qualitative study of six DNA trials, high pre-trial familiarity with DNA appeared to reduce juror doubt about the strength of the prosecution case (Findlay 2008). However, no direct tests of the impact of knowledge on verdicts were feasible and the measures of comprehension of a DNA match and confidence were unverified subjective self-reports.

Use of visual aids to improve jury understanding of expert evidence was recommended following a comprehensive review of DNA uses in court (Wood 2003). The introduction of visual aids can raise 'questions about the reliability and persuasiveness of pictures in the digital age' (Feigenson & Speisel 2009: 103). Research on visual aids in court has produced mixed findings; some improved jury understanding (Hewson & Goodman-Delahunty 2008), some made no difference (Dunn, Salovey & Feigenson 2006) and others misled mock jurors to follow visual cues unsupported by the oral evidence (Kassin & Dunn 1997). Results of a NSW case study suggested that submission of visual aids, including a joint exhibit outlining the steps in DNA sampling, enhanced jury performance (Findlay 2008), but no firm conclusions were possible. Obtaining more specific information about jurors' pre-trial knowledge of DNA profiling is the first step towards systematically adapting the content and delivery of forensic expert evidence to assist jurors. The current study used multiple-choice questions to compare jurors' pre-trial and post-trial knowledge about DNA and the meaning of a random match. It also examined the relationship between DNA knowledge and verdict. The influence on jury comprehension and use of DNA evidence through a cognitively-sequenced tutorial, presented by a single expert, was examined. To test improved understanding, the expert testimony was presented in three ways:

• verbally only;
• with partial multimedia (on DNA only); and
• with full multimedia (on DNA and the random match probability).

Aim

The aim of this study was to identify factors that improve jury understanding and use of inculpatory DNA evidence; that is, evidence that links a suspect to a crime. A quasi-randomised, between-subjects factorial design compared five experimental groups of virtual jurors. The first factor, presence of an expert, compared outcomes in a case with no expert versus a single expert. The second factor, mode of presentation of expert evidence, compared outcomes following exposure to an expert tutorial presented verbally, with partial multimedia, or with full multimedia. The study tested whether:

• expert evidence enhances juror DNA knowledge;
• visual information enhances understanding compared to information conveyed verbally;
• strong DNA evidence increases conviction rates compared to inconclusive DNA evidence; and
• greater understanding of DNA evidence reduces convictions.

Method

The expert tutorial was developed in consultation with legal counsel and forensic and medical scientists to reflect core elements of DNA testimony routinely presented in Australian criminal trials. Six topics were addressed:

• the structure of DNA (nucleotides, base pairs);
• alleles and how they are measured;
• production of a DNA profile (laboratory processing, electropherogram);
• profile interpretation;
• laboratory report and random match probability (RMP) calculation; and
• significance of RMP and DNA match.

Next, a transcript of an actual homicide case containing DNA evidence was adapted for a simulated trial containing the major components of the case. Actors played the roles of the judge, prosecutor, defence attorney, expert and defendant. A still photograph of each active speaker accompanied the narration. The prosecutor and the defence made opening statements. The judge summarised evidence from five prosecution witnesses. In the No expert version, the judge informed jurors that the DNA tests were inconclusive (in practice, therefore, the No expert condition meant no reliable DNA evidence). In the four Expert versions, a forensic expert presented the tutorial on DNA sampling and the resulting RMP of one in one billion. On cross-examination, the defence highlighted the possibility of errors in DNA testing (transference, technical and human errors). Next, the judge summarised evidence from two defence witnesses. Closing statements by legal counsel were followed by judicial directions, including the routine instruction that jurors did not have to accept the expert evidence.

The simulation lasted approximately 35 minutes, 13 to 18 minutes of which comprised expert evidence. The tutorial on DNA lasted approximately 13 minutes and six minutes for RMP. A pilot test with 108 students yielded a mid-range conviction rate of 56 percent, suitable to model variations in conviction levels with a relatively small sample. Forensic experts who testify in Australian trials verified the content of a set of multiple-choice questions addressing topics used to measure jurors' relevant DNA knowledge.

An independent market research company gathered data in two stages. First, email invitations were sent to over 23,000 individuals in New South Wales, Queensland and Victoria. One-quarter opened the email and 5,185 proceeded to access the link describing the task. Respondents were screened for jury eligibility and offered an incentive to complete an online survey assessing DNA knowledge, expectations of DNA evidence in criminal trials, trust in science and demographic characteristics (gender, age, education and English fluency). Approximately two to four weeks later, persons who had completed this survey were invited via email to participate in a simulated online trial lasting approximately one hour, for which they were paid $40. After viewing the simulation, these virtual jurors completed the same 19 DNA knowledge questions as before, plus 10 previously unseen questions, rendered a verdict and reported their trust in expert evidence and the usefulness and ease of understanding of the DNA and RMP tutorials. Participants could not rewind or fast forward the video.

Stage One responses were received from 3,611 jury eligible individuals (70%). An administrator who was blind to the experimental design allocated 470 volunteers from Stage Two to five trial versions to obtain groups roughly equivalent in terms of age, education, gender and state of origin. Participation rates by age, education, gender and state were representative of the Australian population (ABS 2008).

Results

The influence of expert evidence on DNA knowledge

Overall, pre-trial DNA knowledge in the five jury groups was low. No significant differences emerged between groups [F(6, 455)=1.448, ns, η2= 0.19). On average, participants correctly answered 24 percent of the questions (M=4.5 correct of 19 questions). Following exposure to DNA expert testimony, DNA knowledge increased substantially to 59 percent (M=17.2 correct of 29 questions). The expert tutorial significantly increased understanding of DNA evidence (M=63%) compared to that of jurors not given the information (M= 42%) [t(468)=-12.2, p≤.0001,η2= 0.24]. These outcomes verified that the mock jurors attended to the complex information presented by the expert and learned as a result.

To discern sources of juror difficulty, DNA and RMP knowledge and learning gains were compared. The proportion of correct post-trial responses on these topics was similar, but learning gains for DNA (52%) exceeded those for RMP (18%), leaving considerable room to further improve RMP comprehension. Interestingly, as ratings of the ease of understanding of the DNA tutorial increased, jurors' overall learning and DNA learning scores decreased (Kendall's τ=-.11, p≤.05), revealing a significant gap between self-reported understanding and more objective measures of comprehension.

On average, participants anticipated that DNA evidence would be presented in 70 percent of criminal cases. The perceived trustworthiness of DNA evidence was very high—M=6.5 on a seven point Likert scale. However, repeated measures revealed a slight but significant drop following exposure to the tutorial to a mean of 6.2 [F(1, 395)=20.85, Wilk's λ=0.95, Partial η2= 0.05, p≤.0001). This suggests that far from being seduced by 'the white coat syndrome' and deferring mechanistically to the expert because of their field of expertise (Schuman & Champagne 1997: 255), jurors' scepticism may increase slightly when provided with expert testimony. Moreover, jurors with higher objective post-trial DNA knowledge scores rated the evidence as substantially less convincing (44%) than did jurors with lower post-trial DNA knowledge scores (52%). As shown in Table 1, very low levels of DNA knowledge were associated with the highest expectations of, and trust in, forensic expert evidence. Jurors with moderate post-trial DNA knowledge had lower expectations of, and faith in, scientific evidence and jurors with the highest post-trial DNA knowledge scores expressed the most scepticism regarding forensic experts and technology.

The influence of oral versus multimedia expert presentations

Overall, post-trial DNA knowledge, learning and the perceived usefulness and ease of understanding the expert evidence in groups exposed to oral versus multimedia tutorials were equivalent. Jurors exposed to the expert tutorial learned about the same amount irrespective of whether the oral tutorial was accompanied by multimedia, as shown in Table 2.

Evidence presented with multimedia was rated as slightly easier to follow (M=78%) than the oral tutorial (M=75%), but mock jurors exposed to the multimedia expressed more keenness (M=60%) to serve as jurors in the future than their counterparts exposed to traditional oral expert evidence (M=55%). The expert evidence was rated as no more convincing when presented with multimedia.

The influence of expert evidence and DNA knowledge on verdict

DNA evidence tripled the conviction rate (M=59%) compared with the identical case when DNA evidence was inconclusive (M=21%) (χ2=30.3, df=1, p≤.0001, n=462). Increases in post-trial DNA knowledge were associated with a reduction in the perceived culpability of the defendant. This pattern is illustrated in Table 3, showing that mock jurors with the lowest levels of comprehension of the DNA evidence rated the evidence as more convincing, were most confident in the guilt of the defendant and most prone to convict.

Note: Totals may not sum due to rounding

Meanwhile, conviction rates did not differ significantly by mode of expert presentation: 65 percent following traditional oral evidence, 54 percent with partial multimedia and 57 percent following full multimedia. Regardless of the expert presentation, mock jurors tended to use a 95 percent threshold of guilt. As is shown in Figure 1, the multimedia had the most dramatic impact on jurors with lower levels of post-trial DNA knowledge. The conviction rate among jurors in this group who were exposed to verbal evidence only was over 80 percent. It dropped to 64 percent when the expert evidence was accompanied by partial multimedia. When jurors with lower comprehension of DNA evidence were exposed to the full multimedia expert tutorial (on DNA and RMP topics), their verdicts were in the same range as those of jurors with higher levels of post-trial DNA knowledge, that is, 56 percent voted guilty.

Figure 1 Mean conviction rates by expert presentation mode and post-trial knowledge of DNA (%)

Discussion

The findings confirmed the power of a DNA match to significantly increase convictions, even in a weak circumstantial case; three times as many jurors voted to convict in the presence of a DNA match than in the absence of this evidence. This study tested conditions optimal for the 'white coat syndrome' effect; a well-credentialed, independent forensic expert led by the prosecution explained the elements often cited as the most complex and confusing, namely the science and the significance of a DNA match (Findlay 2008). The content of the expert testimony was uncontested and the defence used cross-examination, rather than a rebuttal expert, to challenge this evidence. Nonetheless, reliance on a single forensic expert whose testimony supported the Crown did not produce verdicts uniformly in line with the prosecution case. In general, this Australian jury-eligible sample was resistant to the 'white coat syndrome'. The exception was jurors with poor understanding of the DNA evidence following a traditional verbal presentation by an expert.

The pre-trial survey demonstrated that DNA-relevant knowledge in the Australian population was very limited. On average, people in New South Wales, Queensland and Victoria correctly answered one-quarter of the objective questions. However, they were capable of understanding the complex forensic science information. Exposure to a 20 minute cognitively-sequenced tutorial on DNA profiling and the significance of a random match significantly increased their knowledge on critical issues common to most DNA cases and the tutorial had a positive influence on their decision-making in a simulated homicide trial. The additional insight into DNA evidence reduced juror reliance on this evidence. These results are consistent with findings derived from juror interviews (Findlay 2008) and trial simulation experiments (Vidmar & Diamond 2001). Following a relatively brief tutorial on probabilistic scientific information presented by a DNA expert, jurors took the expert evidence into account in evaluating the weight of the trial evidence, but were not overwhelmed. Nonetheless, the tutorial was more effective at increasing knowledge about scientific methods to produce a DNA profile than knowledge of the significance of a random match. This finding is likely attributable to the content of the tutorial that conveyed more extensive, in-depth information about DNA than about the RMP. The smaller gains and lower post-trial knowledge scores regarding the RMP indicate that this topic requires more elaboration than it received in this expert tutorial.

Technologically sophisticated multimedia presentations do not exert an unduly persuasive or biasing influence on juror decisions, the study suggests. Where multimedia focused on DNA extraction and testing, the learning effect of the multimedia was similar to that of a verbal presentation with identical content. When DNA expert evidence is carefully sequenced in conformity with principles of cognitive instruction, the visual images may not add substantially more value to the tutorial. This finding replicated outcomes of studies in which exposure to animations on topics that were either readily accessible or well known to jurors produced effects indistinguishable from those produced by verbal evidence (Dunn, Salovey & Feigenson 2006). The multimedia tutorials were effective in facilitating the decision-making of participants whose comprehension of the expert evidence was lowest, bringing their verdicts more in line with those whose understanding of the significance of the expert evidence was more accurate.

In general, jurors' insight into their own learning, as reflected in self-reported ease of understanding and usefulness of the information, was unrelated to their objectively measured levels of knowledge and learning. Caution is warranted when interpreting jury responses provided in interviews and focus groups, as there is often little similarity between subjective self-reports and more objective measures of comprehension. An advantage of the trial simulation method applied in this study was the opportunity it offered to compare self-reports with actual behaviours and to discern the real (rather than self-reported) relationship between increased knowledge and verdict.

These findings are subject to a number of limitations. Although participants were community members drawn from three states, and represented a cross-section of educational backgrounds and age ranges, replication with a sample of actual jurors is recommended, as is testing of this tutorial as a pre-trial briefing before an actual trial. Given that the influence of DNA knowledge on verdict was tested in this study in the context of a single case, ideally, to determine whether the results generalise more broadly, replication in other case contexts should follow, including cases involving more contested expert evidence, where DNA evidence is exculpatory and in which the probability of a random match is larger.

Conclusion

A recent survey of forensic experts revealed widespread frustration with the lack of preparation by lawyers, the restricted time in advance of trial to plan in-court testimony and the limited opportunity to explain the scientific evidence adequately to jurors (Wheate 2008). A readily-available model tutorial covering the critical background information applicable in most DNA cases can allay some of these concerns, permitting the expert and the legal counsel to focus on issues unique to the particular case at hand. In 2008, Findlay (2008) recommended that juries be briefed before trial on uncontested complex information that may assist them in understanding the DNA evidence in a particular case. This study demonstrated that a 20 minute tutorial on complex scientific information assisted in resolving jurors' acknowledged difficulties. Whereas strategies previously implemented to assist jurors with complex DNA evidence, such as note-taking, the ability to ask questions, jury directions and deliberation, met with limited success (Dann, Hans & Kaye 2007), the generic expert tutorial on DNA profiling tested in this study dramatically increased juror understanding, whether the information was presented verbally or with multimedia.

The findings of this study, like others examining juror uses of expert evidence, showed that if jurors are given clear and well-sequenced complex information, they deal competently with it. One implication of these results is that the presence of complex information is not a basis to further curtail the use of a jury in Australia; in the right environment, jury-eligible participants learned substantially from an expert presentation. The objective DNA questions used in this study identified specific gaps in the DNA knowledge of jury-eligible participants that can assist forensic experts and legal counsel to hone their presentation in court. This model tutorial and these findings can be applied to train forensic scientists who serve as expert witnesses and to assist legal counsel and judges in conveying relevant DNA information more effectively to jurors.

These findings will assist courts and legal administrators to develop policies and practices for court procedures regarding the use of visual aids, the use of a single expert and uses of stipulated, agreed background informational material in DNA cases, presented either before or during trial. Generic DNA tutorials, such as the one developed for this study, can result in economies of time, reduce the need for multiple expert witnesses on these topics and promote better and more appropriate use of DNA evidence by juries in civil and criminal trials.

Acknowledgements

The support of Criminology Research Council grant 05/07–08 is acknowledged. Helpful advice was obtained from Anna Davey, Andrew Haesler, Chris Lennard, Dennis McNevin and David Tait. We are grateful to Project Manager Berenike Waubert de Puiseau and research assistants Yat Sang Cheung and Diana Matovic.

References

All URLS correct at 4 February 2010

• Australian Bureau of Statistics (ABS) 2008. Population by age and sex, Australian states and territories. cat. no. 3201.0. Canberra: ABS
• Australian Law Reform Commission (ALRC) 2003. Essentially yours: The protection of human genetic information in Australia. http://www.austlii.edu.au/au/other/alrc/publications/reports/96/
• Bornstein BH 2004. The impact of different types of expert scientific testimony on mock jurors' liability verdicts. Psychology, Crime & Law 10: 429–446
• Briody M 2004. The effects of DNA evidence on homicide cases in court. The Australian and New Zealand Journal of Criminology 37: 231–252
• Dann BM, Hans VP & Kaye DH 2007. Can jury trial innovations improve jury understanding of DNA evidence? Judicature 90: 152–156
• Dartnall S & Goodman-Delahunty J 2006. Enhancing juror understanding of probabilistic DNA evidence. Australian Journal of Forensic Sciences 38(2): 85–96
• Dunn ME, Salovey P & Feigneson N 2006. The jury persuaded (and not): Computer animation in the courtroom. Law & Policy 28: 228–248
• Easteal PW & Easteal S 1990. The forensic use of DNA profiling. Trends & Issues in Crime and Criminal Justice no. 26. Canberra: Australian Institute of Criminology. http://www.aic.gov.au/publications/current series/tandi/21-40/tandi26.aspx
• Edmond G & Mercer D 1999. Creating (public) science in the Noah's Ark case. Public Understanding of Science 8(4): 317–343
• Feigenson N & Spiesel C 2009. Law on display: The digital transformation of legal persuasion and judgment. New York: New York University Press
• Findlay M 2008. Juror comprehension and the hard case: Making forensic evidence simpler. International Journal of Law, Crime and Justice 36: 15–53
• Findlay M & Grix J 2003. Challenging forensic evidence? Observations on the use of DNA in certain criminal trials. Current Issues in Criminal Justice 14: 283–298
• Gans J & Urbas G 2002. DNA identification in the criminal justice system. Trends & Issues in Crime and Criminal Justice no. 226. Canberra: Australian Institute of Criminology. http://www.aic.gov.au/publications/current series/tandi/221-240/tandi226.aspx
• Haesler A 2008. Dealing with DNA in court: Its use and misuse. www.lawlink.nsw.gov.au/lawlink/pdo/ll_pdo.nsf/pages/PDO_dnadealingwithin...
• Hewson LE & Goodman-Delahunty J 2008. Using multimedia to support jury understanding of DNA profiling evidence. Australian Journal of Forensic Sciences 40: 55–64
• Kassin S & Dunn M 1997. Computer-animated displays and the jury: Facilitative and prejudicial effects. Law and Human Behavior 21: 269–281
• Lieberman et al. 2008. Gold versus platinum: Do jurors recognize the superiority and limitations of DNA evidence compared to other types of forensic evidence? Psychology, Public Policy, and Law 14: 27–62
• National Research Council 1996. The evaluation of forensic DNA evidence. Washington, DC: National Academy Press
• Schklar J & Diamond SS 1999. Jurors' reactions to DNA evidence: Errors and expectancies. Law and Human Behavior 23: 159–184
• Shuman DW & Champagne A 1997. Removing the people from the legal process: The rhetoric and research on judicial selection and juries. Psychology, Public Policy and Law: 3: 242–258
• Vidmar N & Diamond S 2001. Juries and expert evidence. Brooklyn Law Review 66: 1121–1179
• Walsh SJ et al. 2004. DNA profiling and criminal justice: A contribution to a changing debate. Australian Journal of Forensic Sciences 36: 34–43
• Wheate R 2008. Australian forensic scientists: A view from the witness box. Australian Journal of Forensic Sciences 40: 123–146
• Wheate R 2006. Australian juries and scientific evidence. Australian Journal of Forensic Sciences 38: 75–84
• Wood J 2003. Forensic sciences from the judicial perspective: The expert witness in the age of technology. Australian Journal of Forensic Sciences 35: 115–132

About the authors

Jane Goodman-Delahunty JD, PhD, MAPS, is a Professor at Charles Sturt University.

Dr Lindsay Hewson is currently an education design and media consultant in the medical and higher education sectors.