Error Aversions and Due Process

William Blackstone famously expressed the view that convicting the innocent constitutes a much more serious error than acquitting the guilty. This view is the cornerstone of due process protections for those accused of crimes, giving rise to the presumption of innocence and the high burden of proof required for criminal convictions. While most legal elites share Blackstone’s view, the citizen jurors tasked with making due process protections a reality do not share the law’s preference for false acquittals over false convictions.

Across multiple national surveys sampling more than 12,000 people, we find that a majority of Americans consider false acquittals and false convictions to be errors of equal magnitude. Contrary to Blackstone, most people are unwilling to err on the side of letting the guilty go free to avoid convicting the innocent. Indeed, a sizeable minority view false acquittals as worse than false convictions; this group is willing to convict multiple innocent persons to avoid letting one guilty person go free. These value differences translate into behavioral differences: we show in multiple studies that jury-eligible adults who reject Blackstone’s view are more accepting of prosecution evidence and are more conviction-prone than the minority of potential jurors who agree with Blackstone.

These findings have important implications for our understanding of due process and criminal justice policy. Due process currently depends on jurors faithfully following instructions on the burden of proof, but many jurors are not inclined to hold the state to its high burden. Courts should do away with the fiction that the reasonable doubt standard guarantees due process and consider protections that do not depend on jurors honoring the law’s preference for false acquittals, such as more stringent pretrial screening of criminal cases and stricter limits on prosecution evidence. Further, the fact that many people place crime control on par with, or above, the need to avoid wrongful convictions helps explain divisions in public opinion on important policy questions like bail and sentencing reform. Criminal justice proposals that emphasize deontic concerns without addressing consequentialist concerns are unlikely to garner widespread support.


Christopher Michael Sanchez had the right idea. During voir dire in his trial for assault on a public servant, Mr. Sanchez’s lawyer sought to ask the venire “to rate on a scale of one to five whether it agreed or disagreed with the statement that it is better for ten people [to] go free than one be convicted.”1Sanchez v. State, No. 08-17-00244-CR, 2019 WL 926139, at *1 (Tex. Ct. App. Feb. 26, 2019). The judge disallowed use of the scale but permitted counsel to ask prospective jurors whether they agreed or disagreed with “Blackstone’s ratio,”2Id. at *4–*5. that it is “better that ten guilty persons escape, than that one innocent suffer.”34 William Blackstone, Commentaries *352. “Blackstone’s ratio” is one label applied to William Blackstone’s famous dictum expressing the view that the law should accept multiple acquittals of the guilty to avoid one wrongful conviction. See id.

The appellate court concluded that this alteration in questioning was not so important as to have affected Mr. Sanchez’s substantial rights.4Sanchez, 2019 WL 926139, at *5. The court correctly noted that prospective jurors might have disagreed with Blackstone’s ratio because they rejected the idea that any error was acceptable or because they disagreed with the particulars of the math—that the posited ratio was too high or too low.5Id. Thus, no matter how the initial question was posed, further questioning was needed to understand how jurors’ views on the Blackstone ratio might have affected their interpretation of the state’s “beyond a reasonable doubt” burden of persuasion.6Id. (“Counsel would need to delve into a venireperson’s thought process, and no follow-up questions were asked here.”). Because counsel did not seek to ask further questions on the topic, the judge’s restriction on voir dire did no harm.7Id. (“We thus cannot tell how helpful or not the answer to the question would be in exercising peremptory strikes. Consequently, we cannot determine how helpful or not a scaled response to the same question would be.”).

Despite the somewhat flawed execution, Mr. Sanchez and his counsel were on to something important, for many prospective jurors do not share Blackstone’s view.8In the Sanchez case, for instance, a substantial number disagreed with Blackstone’s ratio. Of the sixty-one prospective jurors asked the question, “49 answered that they disagreed with the statement; 5 answered that they agreed with the statement; and 7 were undecided or equivocal.” State’s Brief at 29, Sanchez, 2019 WL 926139 (No. 08-17-00244-CR), 2018 WL 4381508. Still more troubling, those who do not share that view are unlikely to give the defendant the same benefit of the doubt as those who agree with Blackstone.

We establish both propositions empirically. Across several national surveys, we have found that far more Americans reject Blackstone’s view than endorse it: a majority equate the harms of false acquittal and false conviction, and a sizeable minority deem false acquittals more harmful to society than false convictions.9Theorists often label the error of convicting an innocent person a “type-1 error” and the error of acquitting a guilty person a “type-2 error,” terminology that is cryptic to those not familiar with the literature and perhaps implies one error is more fundamental than the other. To avoid confusion or a suggestion of priority between the errors, we use the terms “false conviction” and “false acquittal” to refer to the two respective factual errors that can occur in a criminal trial (i.e., “false” here means a trial outcome that deviated from ground truth with respect to who committed the acts in question). The majority of Americans are unwilling to trade multiple false acquittals to avoid one false conviction, and many are willing to accept multiple false convictions to avoid one false acquittal.10See infra Section I.B. These value differences matter greatly: those who see false acquittals on par with, or worse than, false convictions are more receptive to the prosecution’s evidence and are easier for the prosecution to persuade.11See infra Part III. We observe these value and behavioral differences in multiple samples of the U.S. population through various trial error aversions measurements, and the pattern holds across political groups.12See infra Part II. Perhaps still more surprising, we find that this is not necessarily a partisan preference. Majorities of Democrats, Republicans, and Independents all view false convictions and false acquittals to be errors of equal magnitude.13See infra Part II.

Our empirical evidence establishes that the error aversions held by the general public depart dramatically from long-accepted constitutional norms.14For prior scholarship exploring the relationship error aversions have with legal outcomes and showing that many laypeople do not share the assumptions of our constitutional system, see Gregory Mitchell & Brandon L. Garrett, The Impact of Proficiency Testing Information and Error Aversions on the Weight Given to Fingerprint Evidence, 37 Behav. Scis. & L. 195 (2019) (data available at The term “error aversion” simply refers to the desire to avoid an error. See infra note 27. Blackstone proffered his famous ratio in 1765,15See supra note 3. but the notion that false convictions constitute a greater injustice than false acquittals animated the law at least as early as biblical times. Blackstone’s 10:1 ratio is just one of many formulations of the idea that false convictions outweigh false acquittals morally and legally.16See generally Alexander Volokh, Aside, n Guilty Men, 146 U. Pa. L. Rev. 173 (1997) (tracing the history of the idea that trials should shift the risk of error in favor of the accused). We follow Professor Epps’ interpretation of the Blackstone principle—the idea that false convictions must outweigh false acquittals. Daniel Epps, The Consequences of Error in Criminal Justice, 128 Harv. L. Rev. 1065, 1068 (2015) (defining the “Blackstone principle” as the notion that “in distributing criminal punishment, we must strongly err in favor of false negatives (failures to convict the guilty) in order to minimize false positives (convictions of the innocent), even if doing so significantly decreases overall accuracy”). In the United States, the Supreme Court first invoked the Blackstone principle in 1895 to justify the presumption of innocence17Coffin v. United States, 156 U.S. 432, 453–56 (1895) (invoking the principle to explain the presumption of innocence and tracing it to Roman law). and again in 1970 to justify incorporating the requirement of the beyond-a-reasonable-doubt burden of persuasion in criminal cases under the Due Process Clause of the Fourteenth Amendment.18See In re Winship, 397 U.S. 358, 361–64 (1970). Justice Harlan’s concurrence in Winship contains the most oft-cited statements from the Court discussing the law’s preference for false acquittals. See id. at 372 (Harlan, J., concurring). Between its decisions in Coffin and Winship, the Court also invoked the Blackstone principle to justify the Fourth Amendment’s limits on search and seizure. See Henry v. United States, 361 U.S. 98, 104 (1959) (“Under our system suspicion is not enough for an officer to lay hands on a citizen. It is better, so the Fourth Amendment teaches, that the guilty sometimes go free than that citizens be subject to easy arrest.”). The fundamental due process protections for criminal defendants at trial—the presumption of innocence and the requirement of proof of guilt beyond a reasonable doubt—arise from the principle that the state should impose punishment only on the clearest of proof that the accused committed the crime charged, even if this high bar means some wrongdoers will escape punishment. In a very real sense, the Supreme Court understands due process as a manifestation of the Blackstone principle: “With reputation, liberty, and at times even life on the line, every legal and moral precept counsels caution in bringing down the hammer of justice on a criminal defendant.”19J. Harvie Wilkinson III, The Presumption of Civil Innocence, 104 Va. L. Rev. 589, 597, 600–03 (2018).

Although occasionally questioned,20See, e.g., Ronald J. Allen & Larry Laudan, Deadly Dilemmas, 41 Tex. Tech L. Rev. 65, 68 (2008) (“At some point in this country’s history, perhaps discussing just one side of the equation was laudable, but now is the time to consider the other side as well.”); Jeremy Bentham, A Treatise on Judicial Evidence 198 ((M. Dumont ed. & trans., London, J.W. Paget 1825) (“All these candidates for the prize of humanity have been outstripped by I know not how many writers, who hold, that, in no case, ought an accused person to be condemned, unless the evidence amount to mathematical or absolute certainty. According to this maxim, nobody ought to be punished, lest an innocent man be punished.”); Epps, supra note 16, at 1069 (“This Article seeks to give the Blackstone principle the careful attention it deserves.”). for the most part, judges, lawyers, and legal scholars accept as correct the proposition that the need to minimize wrongful convictions justifies procedural asymmetries that favor the defendant.21See Epps, supra note 16, at 1069 (“[S]erious and sustained discussions of the principle’s costs and benefits are few and far between. Most simply treat it as a self-evident truth.”). Indeed, one scholar labeled Blackstone’s ratio the “Mount Everest of legal mantras.”22Laura I. Appleman, A Tragedy of Errors: Blackstone, Procedural Asymmetry, and Criminal Justice, 128 Harv. L. Rev. F. 91, 91 (2015). Perhaps because legal education inculcates this mantra,23Indeed, legal education inculcates the Blackstonian view: “Enroll in law school and you will be taught, within the first year, a revered maxim of criminal law: ‘[B]etter that ten guilty persons escape, than that one innocent suffer.’ ” Joel S. Johnson, Note, Benefits of Error in Criminal Justice, 102 Va. L. Rev. 237, 238 (2016) (alteration in original) (footnote omitted). rarely do judges, lawyers, or legal scholars question whether nonlawyers agree with it.24See, e.g., Epps, supra note 16, at 1151 (“Most of us, if required to decide whether someone was guilty of a crime, would almost certainly choose to err strongly against being responsible for wrongly imposing a harsh penalty.”). The prevailing wisdom seems to be that while jurors may not always agree a high burden of proof should apply in every case, jurors will follow the standard jury instructions and err in favor of the accused.25For an excellent discussion of the realities of jury trials and how to align jury behavior with the legal ideal, see generally Jack B. Weinstein & Ian Dewsbury, Comment on the Meaning of ‘Proof Beyond a Reasonable Doubt’, 5 Law, Probability & Risk 167 (2006).

Our first results showing widespread rejection of the Blackstone ratio were so surprising and potentially disruptive that we tested their robustness multiple times, using a series of large samples drawn from the entire U.S. population and multiple measurement methods.26We describe our methods, the populations sampled, and the limitations of these designs in Section I.B. The underlying data is also all available on the Open Science Framework. The picture remained consistent and clear: far more Americans view false acquittals and false convictions to be errors of equal magnitude than those who view false convictions to be the more serious error, and a sizeable minority consider false acquittals to be the more serious error. We detail these results in Part I and explain why prior studies missed this important finding.

In Part II, we examine the demographic, experiential, and ideological correlates of these different error aversions.27Preference and aversion are two sides of the same coin, with the former often used to refer to positive desires or outcomes one wants to experience, and the latter used to refer to negative desires or outcomes one wants to avoid. We usually speak of people’s aversions to false convictions and false acquittals, but we occasionally speak in terms of error preferences as well. We find that people’s error aversions defy simple ideological assumptions: a majority of Democrats, Republicans, and Independents all see false convictions and false acquittals as equally harmful, and many Democrats deem false acquittals more harmful than false convictions. Concerns about the criminal justice system are better predictors of error aversions: those equally averse to the two trial errors tend to have greater fears of being a crime victim and of being falsely accused of a crime, and these fears transcend party labels.

In Part III, we address the behavioral consequences of these different error aversions. We show that a person’s error aversions are important predictors of how they will behave as jurors. For example, in one of our recent studies, the conviction rate among people who prioritize the avoidance of false acquittals was 58 percent, compared to a conviction rate of 25 percent among those who prioritize the avoidance of false convictions, even though these two groups were exposed to the same evidence.28Brandon L. Garrett, William E. Crozier & Rebecca Grady, Error Rates, Likelihood Ratios, and Jury Evaluation of Forensic Evidence, 65 J. Forensic Scis. 1199, 1203–04 (2020). We find similar results in other studies examining how jurors respond to expert evidence and eyewitness evidence.29See, e.g., Brandon L. Garrett, Alice Liu, Karen Kafadar, Joanne Yaffe & Chad S. Dodson, Factoring the Role of Eyewitness Evidence in the Courtroom, 17 J. Empirical Legal Stud. 556, 571 (2020); Mitchell & Garrett, supra note 14, at 206.

In Part IV, we turn to the larger implications of our findings for legal doctrine and crime policy. Herbert Packer famously described two competing models of criminal justice: (1) the crime control model, emphasizing the need to repress criminal conduct and an outright “presumption of guilt,” and (2) the due process model, which emphasizes the presumption of innocence, the risk of convicting the innocent and procedural protections for the accused.30Herbert L. Packer, Two Models of the Criminal Process, 113 U. Pa. L. Rev. 1, 9–14 (1964). Our results suggest that most people view both models as equally important. This finding has substantial implications for legal doctrine, from evidence rules to criminal procedure more broadly, which assume the trial is the “main event” and a beyond-a-reasonable-doubt standard is the key protection for the accused.

Put simply, our findings suggest that legal doctrines that assume the median juror is more averse to false convictions than false acquittals proceed from an empirically false premise. Lawyers should rethink how they select jurors and present evidence, and judges should reconsider how due process protections are implemented. Courts should not presume jurors will follow an instruction to find guilt beyond a reasonable doubt. Jurors may fully understand the burden of persuasion and seek to implement it in good faith, but their error aversions may affect how they view evidence and how they decide whether it exceeds the prosecution’s burden. As a result, jury instructions regarding burdens of proof may not be adequate to secure the values underlying due process protections. Instead, we suggest courts consider nontrial protections, like more stringent pretrial screening of criminal cases, similar to the process courts already use for civil cases. Or courts could more carefully limit what evidence gets admitted. For far too long, constitutional criminal procedure, evidence law, and trial practice have assumed jurors will impartially test a prosecution case. We call into question that assumption and suggest a different path for criminal procedure.

The error aversions we document may also affect a range of broader policy decisions that define our criminal justice system. Public policy advocates who assume the median voter will support initiatives to minimize the risk of false convictions, regardless of impact on crime control, ignore the reality that most Americans care equally about convicting the innocent and freeing the guilty.31To our knowledge, no research has examined Americans’ support for specific criminal justice and policing policies as a function of Americans’ error aversions. Givati, however, examined support for police spending as a function of greater concern about false convictions versus false acquittals and found that those more concerned about false acquittals were willing to spend more on policing. See Yehonatan Givati, Preferences for Criminal Justice Error Types: Theory and Evidence, 48 J. Legal Stud. 307, 321, 327–28 (2019). In contrast, Williamson and colleagues examined Australians’ support for police funding as a function of the same error aversions and found that those more concerned with false convictions supported greater funding of the police. See Harley Williamson, Mai Sato & Rachel Dioso-Villa, Wrongful Convictions and Erroneous Acquittals: Applying Packer’s Model to Examine Public Perceptions of Judicial Errors in Australia, Int’l J. Offender Therapy & Compar. Criminology 15 (Dec. 29, 2021), Voters often determine criminal justice policy through ballot measures or elections of public officials like district attorneys.32For information on the movement to accomplish criminal justice reform through such electoral efforts, see, for example, Angela J. Davis, Reimagining Prosecution: A Growing Progressive Movement, 3 UCLA Crim. Just. L. Rev. 1 (2019). Policy debates that pivot between extremes of being tough on crime versus protecting the rights of the accused overlook the largest group of voters, who worry equally about crime control and due process. Large majorities express great concern about crime but also about police use of excessive force; they support efforts to hold bad cops accountable but resist efforts to defund the police.33William Saletan, Americans Don’t Want to Defund the Police. Here’s What They Do Want., Slate (Oct. 17, 2021, 7:00 PM), []. In short, people care about public safety and fairness.

We conclude by emphasizing that despite sometimes heated legal and political rhetoric, our criminal justice system does not operate as a zero-sum game, even though in any given case the prosecution or defense wins. The public is not well-served by false dichotomies. Convicting the wrong person is not just a fairness concern but also a public safety concern. When an innocent person languishes in prison, a guilty person goes free.34For insights into the role of wrongful convictions in exposing failure to convict actual culprits, as well as convicting the innocent, see, for example, Brandon L. Garrett, Convicting the Innocent 5 (2011) (“In 45% of the 250 postconviction DNA exonerations (112 cases), the test results identified the culprit.”). Unnecessarily jailing the innocent harms the person and the community and produces no gain in public safety.35For studies finding that cash bail imposition can increase reoffending while imposing other social and sentencing harms, see, for example, Paul Heaton, Sandra Mayson & Megan Stevenson, The Downstream Consequences of Misdemeanor Pretrial Detention, 69 Stan. L. Rev. 711, 747 (2017); Will Dobbie, Jacob Goldin & Crystal S. Yang, The Effects of Pretrial Detention on Conviction, Future Crime, and Employment: Evidence from Randomly Assigned Judges, 108 Am. Econ. Rev. 201, 224–26 (2018). From bail reform, to sentencing reform, to protections against wrongful convictions, a range of proposed changes can improve both fairness and public safety.36Scholars and policymakers have been engaged in a rethinking of public safety from a broader perspective, taking into account fairness to defendants, community harm, and more. See, e.g., Phillip Atiba Goff, Elizabeth Hinton, Tracey L. Meares, Caroline Nobo Sarnoff & Tom R. Tyler, Just. Collaboratory & Ctr. for Policing Equity, Re-imagining Public Safety: Prevent Harm and Lead with the Truth (2020), []; Monica C. Bell, Police Reform and the Dismantling of Legal Estrangement, 126 Yale L.J. 2054 (2017); Policing Project & Just. Collaboratory, Reimagining Public Safety: First Convening Report (2021), []. This reframing should be attractive to people who value due process and public safety. Our findings therefore support the view that making fairness and public safety benefits clear to the public will be crucial to the success of future criminal justice reforms.

I. The Distribution of Trial Error Aversions in the General Public

Although extremely influential in formulating jury instructions at criminal trials, Blackstone’s ratio is never explicitly stated. Jurors are not told that the law prefers to err on the side of letting the guilty go free or that all ambiguities and doubts should be resolved in favor of the accused, even if that means allowing a guilty person to go free. Rather, judges typically instruct jurors in a barebones manner that the “defendant is presumed innocent of . . . the charges” and that this presumption “is not overcome unless . . . you are convinced beyond a reasonable doubt that the [defendant] is guilty as charged.”37See Comm. on Fed. Crim. Jury Instructions of the Seventh Cir., The William J. Bauer Pattern Criminal Jury Instructions of the Seventh Circuit 9 (2020), [].

Given the abstract nature of this language, it is perhaps not surprising that jurors often fail to understand that the burden of production lies solely with the state and that the defendant has no obligation to put forth any evidence.38See Joel D. Lieberman, The Psychology of the Jury Instruction Process, in 1 Psychology in the Courtroom 129, 132 (Joel D. Lieberman & Daniel A. Krauss eds., 2009) (summarizing studies showing that many jurors fail to understand the presumption of innocence and allocation of burdens). Nor is it surprising that jurors vary widely in their interpretations of the level of subjective certainty required by the reasonable doubt instruction.39Id. at 133–34; see also Mandeep K. Dhami, On Measuring Quantitative Interpretations of Reasonable Doubt, 14 J. Experimental Psych. 353, 361 (2008) (“[A]ll three methods showed wide interindividual variability in interpretations of reasonable doubt . . . .”); Mandeep K. Dhami, Samantha Lundrigan & Katrin Mueller-Johnson, Instructions on Reasonable Doubt: Defining the Standard of Proof and the Juror’s Task, 21 Psych., Pub. Pol’y & L. 169, 169 (2015) (“[T]here is considerable inter-individual variability in interpretations, meaning that different people interpret RD differently . . . .”); Katrin Mueller-Johnson, Mandeep K. Dhami & Samantha Lundrigan, Effects of Judicial Instructions and Juror Characteristics on Interpretations of Beyond Reasonable Doubt, 24 Psych. Crime & L. 117, 127 (2018) (“[I]nterpretations of BRD ranged from around .5 to 1 across people . . . .”). For the Due Process Clause to consistently protect rights, it is crucial that the legal profession understand why jurors apply different thresholds for conviction and whether legal procedures can reduce this variability.

Efforts to harmonize juror interpretations of judicial instructions through linguistic simplification can produce less compliance with instructions.40Reducing conceptual complexity can increase compliance, but reducing linguistic complexity alone will likely not suffice, and reducing the amount of information given may increase noncompliance. See Chantelle M. Baguley, Blake M. McKimmie & Barbara M. Masser, Deconstructing the Simplification of Jury Instructions: How Simplifying the Features of Complexity Affects Jurors’ Application of Instructions, 41 Law & Hum. Behav. 284, 300 (2017) (“[O]ur analysis also shows that simplifying certain features of complexity unintentionally and adversely affects the punitiveness of jurors’ verdicts.”). Reducing conceptual complexity, as opposed to linguistic complexity, is more difficult because legal concepts do not usually have simpler analogs and because simplifying a concept may alter its intended meaning. Further, jury researchers agree that differences in education levels and other demographic differences cannot explain why jurors interpret and apply judicial instructions differently.41See, e.g., Joel D. Lieberman, The Utility of Scientific Jury Selection: Still Murky After 30 Years, 20 Current Directions in Psych. Sci. 48, 49 (2011) (“The effect of demographic characteristics on verdict inclinations has been investigated for a wide variety of factors, including occupation, age, socioeconomic status, ethnicity/race, and gender. However, these factors typically account for less than 2% of verdict variance when examined independently, and less than 5% when combined together.” (citations omitted)). Juror gender, however, has been found to be more predictive than other demographics “in cases involving domestic homicide and/or child or female victims.” See Dennis J. Devine & David E. Caughlin, Do They Matter? A Meta-Analytic Investigation of Individual Characteristics and Guilt Judgments, 20 Psych. Pub. Pol’y & L. 109, 124 (2014). Instead, they appear to primarily flow from differences in “legal personality,” which involves personal beliefs and attitudes pertaining to civil liberties, the interests of victims, and the rights of the accused.42Lieberman, supra note 41, at 49. In particular, differences in jurors’ levels of cynicism about the justice system and its protection of offenders and their beliefs about the frequency of criminal behavior and the efficacy of punishment translate into pro-prosecution versus pro-defense biases; these biases in turn affect how jurors weigh evidence and decide whether the evidence is sufficient for conviction.43See, e.g., Devine & Caughlin, supra note 41, at 122 (noting that juror authoritarianism and juror trust in the legal system yield “effects on juror guilt judgments large enough to have some practical significance”); Samantha Lundrigan, Mandeep K. Dhami & Katrin Mueller-Johnson, Predicting Verdicts Using Pre-trial Attitudes and Standard of Proof, 21 Legal & Criminological Psych. 95, 103 (2016) (finding that pretrial attitude measures of pro-prosecution versus pro-defense biases accounted for more than 21 percent of the variance in juror verdicts). Jurors, in short, react differently to the same judicial instructions because they bring different preconceptions and goals to the jury task.44It is important to note that jurors with different legal personalities often agree in their verdicts where the evidence in a case clearly supports guilt or a lack of guilt; in general, “extralegal” factors such as juror personality or race of the defendant tend to have their strongest effects where the evidence is most ambiguous. See, e.g., Len Lecci, Christopher Beck & Bryan Myers, Assessing Pretrial Juror Attitudes While Controlling for Order Effects: An Examination of Effect Sizes for the RLAQ, JBS, and PJAQ, 31 Am. J. Forensic Psych., no. 3, 2013, at 41, 46 (“[I]n studying the effects of pretrial bias on juror judgments, the general tendency is for evidence to carry the day. In other words, pretrial biases are most likely to exert their influence when the case is equally strong for the prosecution and the defense.” (citations omitted)).

These differences in legal personality explain why people may differ in their aversions to the two possible errors at trial: false convictions versus false acquittals. Jurors averse to false acquittals should be more skeptical of defense evidence and have a lower threshold for conviction.45See Phoebe C. Ellsworth, Some Steps Between Attitudes and Verdicts, in Inside the Juror: The Psychology of Juror Decision Making 42, 56 (Reid Hastie ed., 1993) (The “greater the [anticipated] regret the juror feels for a mistaken conviction relative to a mistaken acquittal, the higher will be his or her threshold of conviction”); Lundrigan et al., supra note 43, at 96 (“For example, a juror with a pro-prosecution bias would be expected to have a lower conviction threshold . . . than a juror with a pro-defence bias, and consequently the former would be more likely to return a guilty verdict compared with the latter.”). Likewise, to the extent that jurors believe that few defendants who reach trial have been falsely accused and thus discount the need for defendant protections, they should be less concerned with the risk of false convictions and more willing to convict. Conversely, jurors averse to false convictions should be more skeptical of prosecution evidence and should have a higher threshold for conviction.

If most jurors have legal personalities that lead to strong aversions to wrongful convictions, then we should expect juries to fulfill their duties in ways that ensure constitutionally guaranteed due process for criminal defendants: these jurors will hold the government to its high burden of proof and convict only where the proof of guilt is strong.46Those with the strongest aversions to false convictions might, however, hold the government to too high a burden and acquit those who should have been convicted under a reasonable doubt burden. If many jurors have legal personalities that result in strong aversions to wrongful acquittals, however, then we should worry that juries will not require strict proof of guilt, will not presume innocence, and will not disregard evidence such as a defendant’s criminal history that might cause them to have public safety concerns.47See, e.g., Lundrigan et al., supra note 43, at 105 (“We found that the more biased jurors are towards conviction, the lower their quantitative interpretation of [beyond a reasonable doubt].”).

In the two Sections that follow, we first survey prior research on the public’s trial error aversions and the limitations of this research. Then, we turn to our own studies, presenting a very different picture of Americans’ aversions to false convictions versus false acquittals.

A. Prior Survey Data on Trial Error Aversions

At first glance, prior research seems to support the view that a majority of Americans share legal elites’ stronger aversion to false convictions. The primary source of evidence on the public’s trial error aversions has been the General Social Survey (GSS), which is a “nationally representative survey of adults in the United States conducted since 1972.”48About the GSS, Gen. Soc. Survey, []. The GSS “collects data on contemporary American society in order to monitor and explain trends in opinions, attitudes and behaviors.”49Id. Participants are adults surveyed in face-to-face interviews, and the GSS uses multistage sampling to produce a group of respondents proportional to the population living in the continental United States.50 James A. Davis & Tom W. Smith, The NORC General Social Survey: A User’s Guide 31 (Peter V. Marsden ed., 1992). The GSS, which includes hundreds of questions on a wide range of topics, is considered “an important contributor to the statistical and scientific investigation of American society,” and has served as the basis for thousands of journal articles, books, and dissertations.51Stephen Adair, Immeasurable Differences: A Critique of the Measures of Class and Status Used in the General Social Survey, 25 Human. and Soc’y 57, 58 (2001).

The key source of data on Americans’ error aversions comes from a single question on the GSS. Since 1985, the GSS has periodically asked Americans the following question: “All systems of justice make mistakes, but which do you think is worse, to convict an innocent person, or to let a guilty person go free?”52Givati, supra note 31, at 317–18. The question on trial error aversions was included in the 1985, 1990, 1996, 2006, and 2016 iterations of the GSS. Id. at 318. Collectively, these surveys appear to show that “74 percent of Americans think that convicting an innocent person is worse than letting a guilty person go free, while 26 percent hold the opposite view.”53Id. If correct, then the American public seems to share legal elites’ stronger aversion to false convictions.

The GSS employs a forced-choice format to measure error aversions (i.e., respondents must choose one of the errors as worse than the other), and other surveys using the same measurement approach reach similar conclusions to those found on the GSS. For instance, the Cato Institute’s 2016 Criminal Justice Survey asked respondents to choose whether it would be worse to have 20,000 people in prison who are actually innocent or to have 20,000 people escape imprisonment despite being actually guilty.54 Emily Ekins, Cato Instit., Policing in America 59 (2016), []. Sixty percent of the respondents stated that false convictions would be worse than false acquittals.55Id. at 60.

But this forced-choice measurement approach rules out the possibility that the error of false convictions and error of false acquittals are deemed equally aversive and does not measure the relative strength of the aversions. For some topics, a binary forced-choice approach may be appropriate. For example, the approach is valuable where a researcher wants to know which of two consumer products is favored or where a mock jury researcher is studying whether juries acquit or convict.56 Encyclopedia of Survey Research Methods 290 (Paul J. Lavrakas ed., 2008) (“Although useful for some survey items, the forced choice format has disadvantages. The primary disadvantage is that it can contribute to measurement errors, nonresponse errors, or both.”); Arnold Lau & Courtney Kennedy, Pew Rsch. Ctr., When Online Survey Respondents Only ‘Select Some That Apply’ 3 (2019), [] (noting limitations of “select all that apply” questions as compared with forced-choice questions). For a discussion of the use of forced-choice surveys and their limitations in the context of personality tests, see Yue Xiao, Hongyun Liu & Hui Li, Integration of the Forced-Choice Questionnaire and the Likert Scale: A Simulation Study, Frontiers in Psych., 2017, at 1, 1 (“The model may encounter underidentification and non-convergence and the test may show low test reliability in simple test designs . . . .”). But when trying to assess the relative ranking and intensity of values or policy preferences, this approach can systematically bias results and lead to incorrect predictions about what people believe and how people will make decisions.57There is more extensive literature regarding the problem of forced choice assessments of individual preferences, particularly if “no choice” is a practically important option. For example, in consumer preferences research, where consumers do have the real-world option to not buy anything, researchers have raised concerns regarding forced-choice methodologies. See, e.g., Ravi Dhar & Itamar Simonson, The Effect of Forced Choice on Choice, 40 J. Mktg. Rsch. 146 (2003) (describing that where “in many real-world situations, buyers are not forced to choose . . . . and they have the option not to purchase at all, defer purchase, or purchase elsewhere,” then studies that fail to include “no-choice option[s]” may be “systematically biased and lead to incorrect predictions”); see also G. David Hughes, Some Confounding Effects of Forced-Choice Scales, 6 J. Mktg. Rsch. 223 (1969).

Our first inkling that the forced-choice approach to measuring error aversions may produce misleading results arose by chance in the first of a series of studies we undertook to examine how jurors perceive forensic evidence. In the first of these studies, we included two questions on error aversions.58Brandon Garrett & Gregory Mitchell, How Jurors Evaluate Fingerprint Evidence: The Relative Importance of Match Language, Method Information, and Error Acknowledgment, 10 J. Empirical Legal Stud. 484, 502 (2013) (Study 2). One question asked respondents to grade the seriousness of the criminal justice system falsely convicting an innocent person, and another question asked respondents to grade the seriousness of the criminal justice system failing to convict a guilty person.59Id. (“Participants were also asked to respond to two questions designed to measure aversions to Type I and Type II errors: ‘How serious an error is it for the criminal justice system to convict an innocent person?’ and ‘How serious an error is it for the criminal justice system to fail to convict a guilty person?’ ”). Respondents answered both questions using six-point scales, with one signifying that the error was not serious and six signifying that the error was extremely serious.60Id.All but six of the 689 persons participating in the study answered both questions, and, to our surprise, the mean rating for both errors was near the top of the scale (5.78 and 5.10, respectively, for false convictions and false acquittals), and the modal rating for both errors was six (i.e., most deemed both errors to be extremely serious).61These results have not been previously reported, but the data was collected in connection with the research reported in Garett & Mitchell, supra note 58 (data available at The modal response is the response that “has the highest frequency of occurrence” in a data set. Rebecca M. Warner, Applied Statistics 1023 (2008). Nearly half of our sample (333 respondents, or 48 percent) gave the same rating to the two errors.62Garrett & Mitchell, supra note 58. In other words, most people rated both errors to be very serious, and many people rated the errors to be equally serious.

This discovery prompted a closer examination of GSS data on error aversions, and this closer look led to the realization that the forced-choice approach to measuring aversions obscures the fact that many people are strongly averse to both false convictions and false acquittals. Evidence in support of this conclusion can be found in how many respondents refused to categorize one error as more serious than the other. As shown in Table 1, every year in which the GSS has asked its forced-choice question about trial errors, many respondents indicated that they could not choose the worse error or gave no answer.63Unfortunately, the GSS does not ask additional questions that would allow researchers to determine precisely why respondents could not choose or gave no answer, nor why the percentages in these categories varied over time.

Table 1: General Social Survey Data on Trial Error Aversions

As shown in Table 2, this pattern repeats itself on the International Social Survey (ISS), the global analog to the GSS.64About ISSP, ISSP, [] (“The ISSP is a cross-national collaboration programme conducting annual surveys on diverse topics relevant to social sciences.”). The ISS asks the same forced-choice error aversion question, and again we see large numbers of respondents unwilling to choose between the errors. These data suggest that many respondents viewed the errors as equally serious and thus could not choose one error over the other.65Cf. Moulin Xiong, Richard G. Greenleaf & Jona Goldschmidt, Citizen Attitudes Toward Errors in Criminal Justice: Implications of the Declining Acceptance of Blackstone’s Ratio, 48 Int’l J. L. Crime & Just. 14, 18 (2017) (“What does selection of Can’t Choose mean in the context of the aforementioned surveys? This response may reflect conflict in the beliefs of the respondents, specifically, that Type I and Type II errors are equally problematic and constitute a miscarriage of justice, making it difficult for them to select one or the other option. It may reflect the view that it is impossible to select one or the other because there are too many factors that can impact the decision, such as the seriousness of the crime, criminal history of the defendant or other variables.”).

Table 2: International Social Survey Data on Trial Error Aversions

Furthermore, answering that false convictions were worse does not mean that those respondents considered false acquittals to be minor errors. Many people likely consider both errors to be quite serious and harmful (as data from our initial study asking about both errors showed). Data from forced-choice questions like those used on the GSS and ISS provide no insight into the relative strength of the aversions.

B. New Survey Data on Americans’ Trial Error Aversions

Given these reasons for being skeptical about the existing data on the public’s trial error aversions, we began measuring error aversions using a question that does not force respondents to choose one error over the other:

Which of the following errors at trial do you believe causes more harm to society?

  • Erroneously convicting an innocent person
  • Failing to convict a guilty person
  • The errors are equally bad

Using this approach, we have consistently observed large numbers of respondents who rate the errors as equally harmful.

Indeed, as shown in Table 3, most Americans express equal concern about both errors and reject Blackstone’s principle, while sizeable minorities see either false convictions or false acquittals as the more serious error. These data come from national samples recruited by Qualtrics (a survey research company) to be representative of the adult population in the United States with respect to gender, race/ethnicity, age, education, income, regional location, and political identity. Over 12,000 persons have answered our error aversion question in seven different studies, and 61 percent of our respondents stated that the two errors are equally harmful. Only 25 percent viewed false convictions to be the more harmful error, as constitutional tradition holds. A large minority, 14 percent, viewed false acquittals as the more harmful error.

Table 3: Trial Error Aversions When Respondents Are Not Forced to Choose between the Errors

Footnotes for Table 3

Garrett et al. (2018);66Brandon Garrett, Gregory Mitchell & Nicholas Scurich, Comparing Categorical and Probabilistic Fingerprint Evidence, 63 J. Forensic Scis. 1712, 1714 (2018) (data available at Mitchell & Garrett (2019);67Mitchell & Garrett, supra note 14, at 206. Garrett et al. (2020);68See Garrett et al., supra note 29, at 570.
Garrett et al. (2020);69These results have not been previously reported, but the data on error aversions was collected in connection with the research reported in a firearm forensics study. Brandon L. Garrett, Nicholas Scurich & William E. Crozier, Mock Jurors’ Evaluation of Firearm Examiner Testimony, 44 Law & Hum. Behav. 412 (2020) (data available at Mitchell & Tetlock (2021);70Gregory Mitchell & Philip E. Tetlock, Are Progressives in Denial About Progress? Yes, but So Is Almost Everyone Else (2021) (unpublished manuscript) (on file with authors and available upon request) (data available at Mitchell & Garrett (2021);71Gregory Mitchell & Brandon L. Garrett, Battling to a Draw: Defense Expert Rebuttal Can Neutralize Prosecution Fingerprint Evidence, 35 Applied Cognitive Psych. 976, 981 (2021) (data available at
Garrett et al. (2021).72Brandon L. Garrett, William E. Crozier, Karima Modjadidi, Alice J. Liu, Karen Kafadar, Joanne Yaffe & Chad S. Dodson, Sensitizing Jurors to Eyewitness Confidence Using Reason-Based Judicial Instructions, J. Applied Rsch. Memory & Cognition (June 23, 2022),

To ensure that our results were not an artifact of our own measurement method, we measured error aversions in a variety of alternative ways. 73The data from these alternative measures of error aversions was collected in connection with the research reported in an eyewitness face recognition study. Adele Quigley-McBride, William Crozier, Chad S. Dodson, Jennifer Teitcher & Brandon Garrett, Face Value? How Jurors Evaluate Eyewitness Face Recognition Ability, J. Applied Rsch. Memory & Cognition (July 11, 2022), This data, like that summarized in Table 3, was obtained from a national sample of adults recruited by Qualtrics. First, we expanded the number of response options. Those who choose the “equally bad” category might differentiate between the errors once they were allowed to distinguish the severity of the errors. We expanded the options as follows:

Which of the following errors at trial do you believe causes more harm to society?74The information in parentheses following each response option indicates how many respondents chose each option.

  • Convicting an innocent person is much more harmful (308, or 13%)
  • Convicting an innocent person is somewhat more harmful (156, or 7%)
  • Failing to convict a guilty person is much more harmful (192, or 8%)
  • Failing to convict an innocent person is somewhat more harmful (206, or 9%)
  • The two errors are equally bad (1449, or 63%)

This approach not only confirmed our prior finding that most respondents deem the errors to be of equal magnitude but it also revealed variation among those who deem one error weightier than the other. As we suspected, the forced-choice approach for measuring error aversions obscures the fact that those who rate false convictions or false acquittals as the more serious error do not all agree on the level of harm associated with that error.75Cf. Matthew D. Adler, Happiness Surveys and Public Policy: What’s the Use?, 62 Duke L.J. 1509, 1552 (2013) (“Numerous studies using standard preference data . . . have confirmed the common-sense point that different individuals often have different rankings of commodity bundles, income-leisure bundles, different degrees of risk aversion, and so forth.”).

To further explore the relative magnitude of the two aversions, we utilized two different ratio-based approaches to measure error aversions: a willingness-to-pay approach and an error-distribution approach. To measure willingness-to-pay, we asked respondents how much in additional taxes they would be willing to pay each year to reduce the number of wrongly convicted innocent persons and wrongly acquitted guilty persons.76The questions in full read: (a) “How much more in taxes would you be willing to pay each year if that money were used to reduce the number of innocent persons who are wrongfully convicted by the courts?” (b) “How much more in taxes would you be willing to pay each year if that money were used to reduce the number of guilty persons who are wrongly acquitted by the courts?” For an overview of utilizing work hours to carry out such an approach, see, for example, Daniel Kahneman, Ilana Ritov & David Schkade, Economic Preferences or Attitude Expressions?: An Analysis of Dollar Responses to Public Issues, 19 J. Risk & Uncertainty 203 (1999). As shown in Table 4, more people were willing to pay more to reduce false convictions than false acquittals, and on average, people were willing to pay $69.49 more to reduce false convictions than false acquittals. Most people in our sample, however, were either unwilling to pay any tax increase to reduce either error or were willing to pay equal amounts to reduce both errors (see the last two rows of Table 4, as well as the modal responses in the first three rows). This alternative measurement method thus confirmed that most Americans care equally about the two errors, though some within this group are so sufficiently worried about both errors as to devote more tax dollars to reducing them.

Table 4: Willingness to Pay Increase in Taxes as Measure of Error Aversions

Our second ratio-measurement approach asked respondents for their ideal distribution of trial errors using the following question:

In any criminal trial, there is always the risk that the jury will reach the wrong decision. Sometimes that wrong decision results in the conviction of an innocent person, and sometimes that wrong decision results in the release of a guilty person.

Assume that each year, 100 wrong decisions are made by juries in criminal trials. If you could distribute these wrong decisions between wrongly convicting innocent persons and wrongly acquitting guilty persons, how would you distribute these two mistakes?

Type in below the number of false convictions and number of false acquittals that you would prefer each year. Your two numbers must together total up to 100.77The online survey would not allow participants to proceed unless the two numbers entered by a participant summed to 100.

False Convictions: _______

False Acquittals: _______

The average ratio of false convictions to false acquittals was 39:61, but the median and modal ratios were 50:50, indicating that most commonly, respondents distributed the errors equally. Figure 1 illustrates the distribution of false conviction to false acquittals, where zero reflects an even distribution of errors, numbers to the left of zero reflect a preference for more false acquittals, and numbers to the right of zero reflect a preference for more false convictions.

Chart, histogram
Description automatically generated Figure 1: Number of False Convictions Preferred Minus Number of False Acquittals Preferred

Finally, we examined whether error aversions vary with the nature of the underlying wrongdoing, and we altered the wording of the question to focus respondents on their concerns about making an error when serving as a juror in a particular case. For instance, the following question asked about error aversions in a murder trial:

Imagine you are a member of a jury in a trial of a man [accused of first-degree murder. If you and your fellow jurors convict this man of murder, he will receive a sentence of life in jail without the possibility of parole].78The words in brackets varied depending on the nature of the wrongdoing and possible penalty; Table 4 specifies the cases and penalties examined. We held the gender of the defendant constant across cases, hence our use of gendered language here. The jury may make the correct decision about what actually occurred, or it may make an error and either convict an innocent man or let a guilty man go free. Which of these two possible errors would worry you more as you serve on the jury?

  • I would worry much more about convicting an innocent man
  • I would worry a bit more about convicting an innocent man
  • I would be equally worried about both possible errors
  • I would worry a bit more about letting a guilty man go free
  • I would worry much more about letting a guilty man go free

Table 5 summarizes our findings across the different cases that we examined, which ranged from first-degree murder to misdemeanor driving under the influence and which also included a civil sexual harassment claim.79These data were collected as part of a study regarding face memory ability, but not reported in that work in progress. See Quigley-McBride et al., supra note 73 (reporting results of three online studies, with 3,143, 1,156, and 3,180 participants, respectively). For this summary, we grouped together those who worried much more or a bit more about false convictions or false acquittals.

Table 5: Error Aversions across Different Types of Cases

Again, we found that most people are equally concerned about the two potential errors regardless of the wrong the defendant was accused of committing and regardless of the penalty that might be imposed on the defendant.80We did not systematically examine different penalties for each of the wrongs, so we cannot disentangle the influence of nature of the crime/tort or penalty level on responses. These results suggest that the aggregate pattern of error aversions we observe will be fairly stable across types of wrongs and penalties, but more study is necessary to determine just how stable error aversions are at the individual and group level. Further, asking respondents to imagine their concerns as jurors in a particular case did not alter the pattern that we have consistently observed.

C. Summary

In sum, across several large national samples and several different ways of measuring error aversions, we consistently find that most Americans consider false convictions and acquittals to be equally harmful and worrisome errors. Smaller groups see false convictions or false acquittals as the greater concern, but within these groups, individuals differ in the degree to which they believe the harms of one error outweigh the harms of the other.

Our results stand in stark contrast to the results obtained when error aversions are measured using a forced-choice format, an approach that our data strongly suggest leads to misleading conclusions about the distribution of error aversions among the general public. Contrary to the picture painted by surveys that assume one error ranks above the other, our surveys consistently reveal that the great majority of Americans reject Blackstone’s principle that false convictions merit greater concern than false acquittals.

II. Individual Characteristics and Error Aversions

Can we predict who will agree or disagree with Blackstone’s principle? Do those who rate the errors as equally harmful inhabit the center of the political spectrum? Do error aversions depend on one’s experiences with crime and the legal system? To answer these questions, we collected information about the demographics of our respondents, their political views, and their experiences with and beliefs about the legal system.81The results presented here come from multiple surveys, some of which asked different background questions. Accordingly, the number of respondents will vary across the analyses and tables presented. It turns out that there are significant differences between those who agree with Blackstone and those who disagree.

A. Demographic and Political Differences

A regression equation seeking to predict error aversions based on respondents’ self-reported age, gender, race/ethnicity, income, education, and political party preference (Democrat, Republican, or Independent) found that each of these variables, except for race/ethnicity and income, significantly improved prediction.82We employed a multinomial regression because the variable to be predicted was divided into three categories (false convictions worse, false acquittals worse, and errors equally bad). To conduct these analyses, we combined data from the studies reported in Table 3, where respondents had provided the same background information. As shown in Table 6, women were significantly more likely than men to rate false convictions and false acquittals as errors of equal harm,83A chi-square test rejected the null hypothesis that error aversions were evenly distributed among women and men (χ2(2) = 173.74, p < .001). Only a handful of respondents identified as nonbinary, agendered, or gender queer; these respondents predominantly rated both errors as equal. In an earlier study, Professor Givati wrote that “[u]sing GSS data, I find that women care less about convicting the innocent than men.” Givati, supra note 31, at 310. Our results using the expanded measure of error aversions show that most women are equally concerned about the two errors rather than less concerned about false convictions. persons under the age of 30 were less likely to equate the errors compared to older persons,84A chi-square test rejected the null hypothesis that error aversions were evenly distributed across age groups (χ2(4) = 25.07, p < .001). and those with post-college education were more likely to rate false convictions as the more serious error.85A chi-square test rejected the null hypothesis that error aversions were evenly distributed across educational groups (χ2(4) = 26.43, p < .001).

Table 6: Error Aversions by Demographic Categories and Political Party Affiliation

One might expect that conservatives would be more concerned about law and order, and thus especially sensitive to letting the guilty go free, while liberals would be more concerned about the rights of the accused and the prospect of wrongful convictions. But our results defied that simple dichotomy. A greater percentage of Democrats were most averse to false convictions, but Republicans were outnumbered by Democrats and Independents among those most averse to false acquittals. And a majority of all three groups rated both errors as equally harmful, with Republicans and Independents significantly more likely to be in this group than Democrats.86A chi-square test rejected the null hypothesis that error aversions were evenly distributed across political groups (χ2(4) = 113.53, p < .001).

The results summarized in Table 6 demonstrate that these demographic and political differences occur at the margins, but what stands out is our primary finding that the majority of Americans agree convictions of the innocent and acquittals of the guilty are equally important. Across political and demographic differences, most Americans believe both types of error equally matter, rejecting the Blackstone principle and the foundational premise of the Due Process Clause. A key question is whether this flows from experiences with the criminal justice system or, conversely, whether this correlates with different views about the criminal justice system. We turn to that question next.

B. Experiences with, and Perceptions of, the Criminal Justice System

Broad demographic and political categories do a poor job differentiating among those with different error aversions, but perhaps the source of these differences can be found in individuals’ experiences with, and perceptions of, the criminal justice system and crime. To dig deeper into the possible origins of the different error aversions, we asked a series of questions about respondents’ experiences with, and beliefs about, crime and the functioning of the criminal justice system.87This data was collected as part of the research reported in a study on fingerprint experts. Mitchell & Garrett, supra note 71. To avoid having these estimates influence error aversion responses, we asked these questions after participants answered our error aversion question.

First, to examine the relationship between error aversions and experiences with the criminal justice system, we asked respondents whether they themselves or a close family member had ever been arrested, falsely accused of a serious crime, or the victim of a serious crime. Second, to examine the relationship between error aversions and beliefs about the operation of the criminal justice system, we asked respondents to estimate the number of false convictions, false acquittals, and unsolved crimes per 100 crimes, and we asked respondents whether they believe the criminal justice system gives too much or too little attention to the interests of criminal defendants and crime victims.

Surprisingly, different experiences with the criminal justice system did not correlate with different error aversions. Although a substantial number of our respondents reported that they or family members had been arrested or victimized, none of these experiences significantly predicted error aversions.88Of the 1716 participants in this study, 454 (27%) reported that they or a family member had been arrested, 350 (20%) reported that they or a family member had been a crime victim, and 152 (9%) reported that they or a family member had been falsely accused of a crime. But different beliefs about the functioning of the criminal justice system did correlate with differences in error aversions. Estimates of the number of false convictions, false acquittals, and unsolved crimes, as well as perceptions that the criminal justice system attends too much or too little to the interests of defendants or victims, all significantly or marginally predicted error aversions.89A multinomial regression predicting whether respondents stated that false convictions were worse, false acquittals were worse, or that the errors were equally bad found that estimates of false acquittals per 100 trials and views about the interests of criminal defendants and crime victims were associated with error aversions below the usual .05 statistical significance level. Estimates of false convictions were associated with error aversions at the p = .09 level and estimates of unsolved crimes were associated with error aversions at the p = .05 level.

Examining these specific beliefs in more detail sheds further light on the possible origins of the different error aversions. As shown in Table 7, when it comes to beliefs about the treatment of criminal defendants, those who worry most about false convictions believe that the law gives too little protection to criminal defendants, those who worry most about false acquittals believe the law gives too much protection to criminal defendants, and those who worry about both errors largely believe the law gives about the right amount of protection to criminal defendants. All three groups believe the law gives too little attention to the interests of crime victims, but the only significant gap in beliefs was between those most worried about false convictions and those who worry about both errors.90Participants were asked whether the law gives too little (scored as -1), too much (scored as 1), or about the right amount of protection to criminal defendants (scored as 0), and participants were asked whether the law gives too little (-1), too much (1), or about the right amount of attention to victim interests (0). The mean scores in Table 7 below 0 indicate too little protection or attention, and mean scores above 0 indicate too much protection or too much attention. Mean scores in Table 7 with different superscripts were significantly different from one another at the .05 level, as determined by a post hoc Tukey test.

Table 7: Participant Beliefs about the Interests of the Accused and Victims

Although estimates of the error rates improved prediction of which error aversion individuals would hold when those estimates were added to a regression equation, most of the observed differences in estimates across the error aversion groups did not rise to the level of statistically significant differences when considered on their own. As shown in Table 8, those most concerned about false convictions gave higher estimates of the number of false convictions and lower estimates of the number of false acquittals than those most concerned with false acquittals. Those equally concerned about the two errors gave estimates that fell between the other groups’ estimates. Only the higher estimate of false convictions given by those most averse to false convictions differed significantly from the estimates of other groups.91The difference in each group’s mean estimates were tested using a post hoc Tukey test. The mean estimate of false convictions given by those most averse to false convictions was significantly different from the estimate given by those most averse to false acquittals (p = .03) and those equally averse to the errors (p = .04). The difference in estimates by those averse to false acquittals and those equally averse to the errors was not statistically significant. With regard to estimates of the number of false acquittals, the difference in estimates given by those most averse to false acquittals was marginally significant when compared to the estimates given by those equally averse to the errors (p = .067). No other comparisons were marginally or statistically significant. Interestingly, all error aversion groups gave, on average, similar estimates of the number of unsolved crimes, and all three groups estimated false acquittals to be higher than false convictions.

Table 8: Participant Estimated Error Rates and Unsolved Crime Rates by Error Versions

Finally, we asked respondents how often they worried that they or a family member would be a victim of a crime or falsely accused of a crime. Both worries significantly predicted error aversions.92A multinomial regression predicting whether respondents stated that false convictions were worse, false acquittals were worse, or that the errors were equally bad found that fear of crime and fear of false accusations were both associated with error aversions below the usual .05 statistical significance level. As shown in Table 9, those who worried most about false acquittals also worried more about being a crime victim than those who saw the errors as equally bad, and those who worried most about false convictions worried more about being falsely accused of a crime compared to those holding other error aversions.93Participants were asked to rate how frequently they worry about being a victim of crime and about being falsely accused of a crime on a 1 (never worry about this possibility) to 6 (worry about this possibility on a daily basis) scale. Thus, higher scores in Table 9 reflect greater worry. Mean scores in Table 9 with different superscripts were significantly different from one another at the .05 level, as determined by a post hoc Tukey test.

Table 9: Participant Fears of Crime and False Accusations of Committing a Crime

By tying the demographic and system-perception portraits together, we can paint a richer portrait of the persons who hold the different error aversions. Those who weigh false convictions more heavily tend to be those who are highly educated, male Democrats who believe that the law does not provide adequate protections for criminal defendants or that the false conviction rate is relatively high. Those who weigh false acquittals more heavily tend to be those who believe the false conviction rate is relatively low and that the law gives too much protection to defendants and too little attention to the interests of victims. Those who weigh the errors equally tend to be Independents over the age of 30 (especially female Independents over the age of 50) who worry about being a crime victim or believe that the law does a good job protecting the rights of criminal defendants but not with respect to the interests of victims.

Notwithstanding these differentiating factors, it is important not to overstate the identifiable differences among those who hold different error aversions. Indeed, wagering that a randomly chosen American is more likely to see false convictions and false acquittals as equally bad errors would, over time, be the smart bet. In our samples, most Democrats and Republicans fell into this middle ground category, as did persons of all ages, races, and ethnicities. While it is true that more women than men fell into this middle ground category, almost half of the men in our samples fell into this category too.

Thus, our detailed analyses of the characteristics of the people who express different error aversions leave us with an important takeaway: large numbers of Americans across a range of demographic and ideological differences share the view that avoiding convictions of the innocent and acquittals of the guilty are equally important goals for the law to have. As we discuss next, these findings have important implications for what happens at trial.

III. Juror Behavior and Error Aversions

A number of prior studies have found that the legal personalities associated with pro-prosecution versus pro-defense biases predict juror behavior in mock trials.94See, e.g., Len Lecci & Bryan Myers, Individual Differences in Attitudes Relevant to Juror Decision Making: Development and Validation of the Pretrial Juror Attitude Questionnaire (PJAQ), 38 J. Applied Soc. Psych. 2010, 2028 (2008) (“Our findings suggest that attitudes relating to a tendency to convict, confidence in the judicial system, cynicism toward the defense, racial bias, beliefs regarding the innate quality of criminal behavior, and the belief that individuals do not receive equal protection under the law (i.e., social justice) are relevant factors in how final legal judgments are reached.”); Len B. Lecci & Bryan Myers, Predicting Guilt Judgments and Verdict Change Using a Measure of Pretrial Bias in a Videotaped Mock Trial with Deliberating Jurors, 15 Psych., Crime & L. 619, 628 (2009) (“The present study demonstrates the predictive validity of the PJAQ [measuring pro-prosecution vs. pro-defense bias] using ecologically valid stimuli (e.g. realistic trial videotape) and legal procedures (e.g. jury deliberation).”); Lecci et al., supra note 44, at 50–52 (finding that three alternative measures of legal personality predicted verdict tendencies); Lundrigan et al., supra note 43, at 105 (“We found that the more biased jurors are towards conviction, the lower their quantitative interpretation of [the beyond a reasonable doubt instruction].”); Lisa L. Smith & Ray Bull, Validation of the Factor Structure and Predictive Validity of the Forensic Evidence Evaluation Bias Scale for Robbery and Sexual Assault Trial Scenarios, 20 Psych., Crime & L. 450, 452 (2014) (showing that jurors predisposed to favor the prosecution rated the prosecution’s DNA evidence more favorably). But does the aspect of legal personality that we focus on—trial error aversions—also predict juror behavior? Across many different studies involving several thousand jury-eligible adults, we find that jurors’ error aversions do predict how individuals will analyze and weigh evidence to reach a decision on conviction or acquittal. We first describe our findings on the conviction rates associated with the different error aversions, and then we develop two possible overlapping explanations for these findings.

A. Error Aversions and Conviction Rates

A striking demonstration of the importance of error aversions can be seen in a comparison of conviction rates across the groups. Table 10 presents the conviction rates of the three error aversion groups from our mock juror studies in which participants were asked to vote for conviction or acquittal:95In these studies, we randomly assigned participants to different trials in which they would be exposed to different types of evidence in support of the prosecution or defense. Each mock juror participated in only one trial, and mock jurors did not deliberate before providing their responses (for additional details, please consult the studies cited in Table 10). To ensure that persons with one particular error aversion were not by chance consistently assigned to trials in which the evidence of guilt was weak or strong, we examined the distribution of participants’ error aversions across the trials and found that none of the error aversion types was overrepresented in any particular trial (i.e., there were no significant differences in the percentage of mock jurors holding the different error aversions across the trials). those most worried about false convictions are consistently less prone to convict defendants, while those most worried about false acquittals are more prone to convict, and those who worried equally about the two errors consistently exhibit conviction rates between the two extremes. Consistent with these differences in conviction proneness, we also observe that those most averse to false acquittals rate the prosecution’s evidence against the defendant to be stronger than those who equate the errors or those who worry more about false convictions, and this group is more trusting of the prosecution’s forensic evidence and more skeptical of the defense’s evidence.96See Garrett et al., supra note 28, at 1203 (“[P]eople who were more concerned about failing to convict a guilty person were more likely to vote guilty, believe the case was stronger, have a higher opinion of the reliability of voice and fingerprint evidence, and believe more people brought to criminal trial are guilty . . . .”); Mitchell & Garrett, supra note 14, at 206 (“[T]hose with stronger aversions to false acquittals versus false convictions gave more weight to the fingerprint evidence regardless of experimental condition . . . .”); Mitchell & Garrett, supra note 71, at 982–84 (discussing greater skepticism to a fingerprint expert for the defense among those most concerned about false acquittals).

Table 10: Conviction Rates by Error Aversion

Footnotes for Table 10:

Garrett et al. (2018);97These results have not been previously disclosed, but the data were collected in connection with the research reported in a study of categorical and probabilistic fingerprint evidence. Garrett et al., supra note 66. Garrett et al. (2020);98Garrett et al., supra note 29, at 570. Garrett, Crozier & Grady (2020);99Garrett et al., supra note 28, at 1204 tbl.2. Mitchell & Garrett (2021).100These results have not been previously disclosed, but the data were collected in connection with the research reported in a study on trends in progress among progressives. Mitchell & Tetlock, supra note 70.

Jurors’ trial error aversions, in short, are good proxies for pro-prosecution versus pro-defense biases that potential jurors bring into the courtroom. Those more concerned about false convictions will be more accepting of defense evidence and more skeptical of the state’s evidence, while those more concerned about false acquittals will be more likely to exhibit the opposite tendencies.101Our evidence strongly suggests that error aversions affect how jurors analyze and weigh evidence, but we also have some evidence suggesting that error aversions affect the level of subjective certainty required for conviction. In particular, we found in one study that error aversions affected receptivity to prosecution evidence and the level of subjective certainty about guilt needed for jurors to vote for conviction. Those most concerned about false convictions rated the prosecution’s case as weaker and required greater subjective certainty that the defendant committed the crime before voting for conviction (on average, those who worried about false acquittals voted to convict once they believed it was 71 percent likely the defendant committed the crime, whereas those worried about false convictions voted to convict once they believed it was 81 percent likely the defendant committed the crime) (based on data from Mitchell & Garrett, supra note 14). In another study, however, we found smaller differences in the level of subjective certainty required for conviction (on average, all of the error aversion groups voted for conviction only once the likelihood of guilt was approximately 80 percent), but those who worried more about false acquittals viewed the prosecution’s case as much stronger than those who worried more about false convictions (based on data from Mitchell & Garrett, supra note 71). In other words, in trials in this study, it was harder to convince those averse to false convictions of the defendant’s guilt and easier to convince those averse to false acquittals of the defendant’s guilt, but the groups had similar subjective certainty thresholds for voting to convict.

B. Evidence Skepticism or Burden of Proof Adjustments?

The differences in conviction rates we observe across the error aversion groups could be the product of two mechanisms: (1) an evidence skepticism effect, in which false-conviction avoidant jurors are more receptive to pro-defense evidence and arguments, and false-acquittal avoidant jurors are more receptive to pro-prosecution evidence and arguments, or (2) a burden of proof adjustment effect, in which jurors adjust their subjective threshold to convict up or down depending on which error they seek to avoid.

Support for the evidence skepticism mechanism is provided by the fact that, despite seeing the same evidence, those averse to false convictions tend to rate the strength of a prosecution case as weaker than those averse to false acquittals. But we have also observed differences in the subjective certainty thresholds associated with a vote for conviction across the error aversion groups.102This data was collected as part of a study on using defense expert rebuttal to neutralize prosecutorial fingerprint evidence. Mitchell & Garrett, supra note 71. Because jurors remain free to determine what level of subjective certainty equates to reasonable doubt about guilt, jurors can vary the stringency with which they hold the prosecution to its burden of proof.103See, e.g., United States v. Hernandez, 176 F.3d 719, 728 (3d Cir. 1999) (“Reasonable doubt is not an easy concept to understand, and it is all the more difficult to explain.”); State v. Bennett, 165 P.3d 1241, 1248 (Wash. 2007) (“We recognize that the concept of reasonable doubt seems at times difficult to define and explain.”). See generally Michael D. Cicchini, Instructing Jurors on Reasonable Doubt: It’s All Relative, 8 Calif. L. Rev. Online 72 (2017). We suspect that, at times, both effects occur, depending on the strength of the evidence and the magnitude of worry about false convictions or false acquittals.

When jurors’ subjective thresholds for conviction have been measured, one finds “sizable variability in interpretations of the standard when it is left undefined.”104Dhami et al., supra note 39, at 176; see also C.M.A. McCauliff, Burdens of Proof: Degrees of Belief, Quanta of Evidence, or Constitutional Guarantees? 35 Vand. L. Rev. 1293, 1325 tbl.2 (1982) (reporting a survey showing variations in interpretation of the beyond-a-reasonable-doubt standard, from 50% to 100%, with the most common response at 90% and next most common at 95%). Our research suggests that error aversions may influence where different jurors set those thresholds, causing some jurors to view the beyond-a-reasonable-doubt standard as less demanding. But we have also found that those with different error aversions often have similar subjective certainty thresholds for convicting. For example, as shown in Figure 2 below, in one of our mock juror studies, those who voted to acquit typically estimated the likelihood of offense to be below 50 percent, and those who voted to convict typically estimated the likelihood of offense to be above 75 percent across all three error aversions.105Mitchell & Garrett, supra note 71. Still, in this same study, those most averse to false acquittals viewed the prosecution’s evidence as stronger and the defense’s evidence as weaker than those most averse to false convictions, even though both groups were viewing the same evidence.106See Mitchell & Garrett, supra note 72, at 981–82 (finding that those most averse to false acquittals were less influenced by defense evidence than those most averse to false convictions). Thus, even if those with different error aversions have similar subjective certainty thresholds for voting for conviction, it may still be easier for the state to convince those most averse to false acquittals that the evidence exceeds this threshold. In other words, error aversions appear to relate to differences in conviction proneness.

Chart, bar chart
Description automatically generated Figure 2: Error Aversions and Subjective Certainty Thresholds to Convict

Judges have long suspected that both processes may be at work. For instance, anticipating the empirical evidence that we marshal here, Judge Jack Weinstein, writing with Ian Dewsbury, speculated that jurors “may evaluate the same evidence differently and adjust the burden of proof based on personal experience.”107Weinstein & Dewsbury, supra note 25, at 171. And consistent with our findings, they attributed this variance to differences in the “degree of confidence in the police, the prosecutor, the court[,] and the justice system as a whole.”108Id. at 172.

To be sure, where jurors perceive evidence of guilt to be particularly strong or weak, differences in jurors’ error aversions are unlikely to result in differences in trial verdicts.109See, e.g., Leticia De La Fuente, E. Inmaculada De La Fuente & Juan García, Effects of Pretrial Juror Bias, Strength of Evidence and Deliberation Process on Juror Decisions: New Validity Evidence of the Juror Bias Scale Scores, 9 Psych., Crime & L. 197, 206 (2003) (“[I]n our study, . . . the bias scores collected by the [Juror Bias Scale have] been shown to be valid predictors of the verdicts of mock jurors in the case of ambiguous evidence, but not in those of clear evidence.”); Mitchell & Garrett, supra note 14, at 208–09 (discussing how evidence that the prosecution’s fingerprint examiner scored poorly on a proficiency test convinced even those jurors most concerned about false acquittals that the state’s case was weak). In ambiguous cases, however, jurors’ error aversions may play an important role in determining whether a defendant is acquitted or convicted. In light of the recent Supreme Court decision declaring that the Sixth Amendment to the Constitution requires unanimous jury verdicts in all trials of serious offenses,110Ramos v. Louisiana, 140 S. Ct. 1390, 1397 (2020) (“So if the Sixth Amendment’s right to a jury trial requires a unanimous verdict to support a conviction in federal court, it requires no less in state court.”). a single juror particularly worried about false convictions may be sufficient for the defendant to win acquittal. Conversely, a single juror worried about the consequences of false acquittals may be the difference between a hung jury and a verdict of acquittal.

Even though error aversions may not matter in every case, they could be crucial in the marginal case. Our research validates the voir dire undertaken by counsel for Christopher Michael Sanchez, the case with which we opened this Article.111See supra text accompanying notes 1–8. Existing multi-question measures of legal personality cannot practically be used during voir dire because they are too lengthy and time-consuming.112See Lecci et al., supra note 44, at 59 (discussing how standardized measures of legal personality might be used as pretrial questionnaires, in expanded voir dire, or in abbreviated fashion during ordinary voir dire). But jurors’ error aversions can be quickly and easily assessed during voir dire by asking the venire for a showing of hands of those who believe that it is worse to convict an innocent person or to acquit a guilty person and those who cannot choose which error is worse. This simple inquiry will reveal much about how those in the venire view the rights of criminal defendants versus the interests of crime victims and about how jurors will approach the evidence about to be presented at trial.

While our empirical evidence of behavioral tendencies associated with the different error aversions cannot support the conclusion that any particular juror should be excused for cause due to a pro-prosecution or pro-defense bias, the evidence should raise concern when attorneys are deciding how to exercise peremptory strikes. Furthermore, judges confronted with many prospective jurors who reject Blackstone’s principle should consider instructions designed to help these jurors understand their obligations under current due process law. We hope that other attorneys will follow the lead of Mr. Sanchez’s counsel and that other courts will follow the lead of the judges in Mr. Sanchez’s case, who recognized that trial error aversions are a proper area of inquiry during jury voir dire.

Such voir dire should lead to a more balanced jury, but still, the question remains: should the legal system accept the possibility that juries comprised of those strongly averse to false acquittals are likely to convict some defendants on evidence that the median juror would find quite weak and not free of doubt? This prospect of jury nullification of the defendant’s rights moves us beyond questions of trial strategy to questions of fundamental system design.

IV. The Larger Consequences of the Public’s Error Aversions

The discovery of large numbers of prospective jurors who place false acquittals on par with, or even above, false convictions in the hierarchy of trial errors raises foundational questions about the meaning of due process under the U.S. Constitution and how to preserve it. Historically, Professor Packer’s “due process” model of the criminal justice system has been more influential within the academy and courts than the crime control model,113See Packer, supra note 30, at 9–14; see also Herbert L. Packer, The Limits of the Criminal Sanction 153–73 (1968). On the influence of Packer’s two-model conceptualization, see, for example, Hadar Aviram, Packer in Context: Formalism and Fairness in the Due Process Model, 36 Law & Soc. Inquiry 237, 243 (2011) (reviewing Packer, supra note 30) (“The academic fascination with Herbert Packer’s two models of the criminal process has yielded an amazing thread of scholarship involving a variety of academic disciplines.”). which Packer described as operating on the “presumption of guilt” for those who are not quickly screened as unlikely to have offended.114Packer, supra note 30, at 11–12 (“If there is confidence in the reliability of informal administrative factfinding activities that take place in the early stages of the criminal process, the remaining stages of the process can be relatively perfunctory without any loss in operating efficiency. The presumption of guilt, as it operates in the Crime Control Model, is the expression of that confidence.”). Whereas under the crime control model, “no errors should suffice for reversal if the appellate court concludes on a review of all the evidence that the factual guilt of the accused was adequately established,”115Id. at 54. under the due process model, “any error abridging basic rights of the defendant . . . should be ground for reversal irrespective of the strength of the rest of the case.”116Id. at 55. These different approaches to errors at the trial level flow from different assumptions about the reliability of the factfinding process: the crime control model places great faith in the ability of police and prosecutors to make accurate decisions, but the due process model “stresses the possibility of error” in the factfinding process.117Id. at 14. How far the law should go to avoid error depends on our confidence in the reliability of the system, our willingness to forgo efficiency and finality to avoid error in individual cases, and our relative aversion to false convictions versus false acquittals. Our results suggest that most people seek to navigate a middle ground between Packer’s archetypal models, a path that controls crime without treading on the rights of the truly innocent.118Professor Packer emphasized his two models were “normative” and “represent an attempt to abstract two separate value systems that compete for attention in the operation of the criminal process.” Id. at 5. Professor Packer surely would not be surprised to find many people hold values falling in between a value system that only emphasizes crime control or one that only emphasizes the rights of the accused. People do worry about errors in the factfinding process, but many worry that these errors often benefit rather than harm the accused, and most people see just as much harm to society from letting the guilty go free as from imprisoning the innocent.

That most people do not fully share core due process commitments under the U.S. Constitution raises several possible responses. One response to this new evidence of public sentiment at odds with the intuitions of legal elites would be to reconsider the balance of protections in place to protect the accused while permitting vigorous investigation and prosecution of crimes.119Recently, a few legal scholars have questioned whether criminal law goes too far in protecting against false convictions. E.g., Allen & Laudan, supra note 20; Epps, supra note 16. For the most part, these critiques proceed from the premise that false acquittals likely cause more harm than most theorists assume rather than any empirical analysis of the relative error preferences of citizens. Perhaps the current balance of interests under the due process calculus places too much weight on the interests of the accused and not enough weight on the interests of past and future victims of the guilty. For those who believe that due process should reflect public values, our data, therefore, provide grounds for reconsidering the nature of due process in criminal adjudications.

Alternatively, if the Constitution dictates the preference for wrongful convictions over wrongful acquittals regardless of the general public’s values, then we need to redouble our efforts to secure the due process model. For those who believe Blackstone’s principle is incorporated into the Constitution, our data should be alarming and should lead to a reconsideration of how to institutionalize Blackstone’s principle to ensure that all jury trials resolve reasonable doubts in favor of the accused.120The law seeks to institutionalize an aversion to false convictions through the “interlaced principles of juror unanimity, the presumption of innocence, and proof beyond reasonable doubt.” State v. Gaiter, No. F12-8535, 2016 WL 2626005, at *8 (Fla. Cir. Ct. May 9, 2016).

In the Sections that follow, we first discuss the implications of our findings for the heavy reliance on the burden of proof as a protection against false convictions and consider alternative means of ensuring that only the truly guilty are convicted. We then turn to the broader implications of our data for criminal justice reforms and policymaking.

A. Rethinking the Burden of Proof as the Primary Safeguard of Defendants

Our research suggests that most jurors come to the courtroom predisposed to give equal weight to prosecution and defense evidence and are just as likely to resolve doubts in favor of the prosecution as they are to resolve them in favor of the accused. As a result, those equally concerned about false convictions and false acquittals can be more easily persuaded of guilt than those more concerned about false convictions. Differences in subjective thresholds for conviction and differences in how evidence is assessed and weighed may both contribute to differences in conviction proneness, a prospect that raises serious doubts about the ability of a burden of persuasion to guard against false convictions. If jurors with different error aversions can watch the very same trial and come away with very different impressions of how strong the evidence of guilt is, then prescribing the burden of proof in strict, quantitative terms is likely to have little impact on the outcome. Furthermore, no matter how the burden of proof is described, ultimately, every individual juror must translate that verbal description into a subjective threshold for conviction. Currently, we have little reason to believe that a single instruction can produce the same high, subjective threshold across all jurors.121See, e.g., Kwangbai Park, Yoori Seong, Minchi Kim & JongHan Kim, Juror Adjustments to the Reasonable Doubt Standard of Proof, 22 Psych., Crime & L. 599, 613 (2016) (“This study indicated that participants who voted to convict were more likely to lower their interpretation of the legal standard than those who voted to acquit, regardless of the method used to measure the interpretation.”).

If the law, therefore, is serious about erring on the side of avoiding false convictions, it would do well to focus on what evidence the prosecution can present rather than on the burden of proof.122For discussion of the manner in which innocence is underprotected by constitutional criminal procedure, see infra Section IV.B. To be sure, exclusionary rules also flow from the Due Process Clause, as well as other amendments, particularly the Fourth Amendment. Richard M. Re, The Due Process Exclusionary Rule, 127 Harv. L. Rev. 1885, 1887 (2014) (describing the “historically evolving interrelationship between the Fourth Amendment and the Due Process Clauses”). Reliability of evidence, however, is principally an evidence law concern and not a constitutional or due process concern. See, e.g., Sandra Guerra Thompson, Daubert Gatekeeping for Eyewitness Identifications, 65 SMU L. Rev. 593, 596 (2012) (developing how policing reliability is the “principal role of the rules of evidence”). Critics of constitutional rules of criminal procedure have long argued that the courts should focus on truth-seeking and substantive justice rather than procedural ideals.123For the argument that exclusionary rules should focus on accurate evidence and privacy, see Akhil Reed Amar, Against Exclusion (Except to Protect Truth or Prevent Privacy Violations), 20 Harv. J.L. & Pub. Pol’y 457, 457–58, 460 (1997). For the argument that, more broadly, constitutional criminal procedure should focus on substance over process, see William J. Stuntz, The Collapse of American Criminal Justice 79 (2011). The Constitution generally imposes few substantive restrictions on the admissibility of evidence.124See generally Alex Stein, Constitutional Evidence Law, 61 Vand. L. Rev. 65 (2008). Making it harder to introduce evidence of a defendant’s prior convictions and other bad acts,125 Fed. R. Evid. 404 (regulating admissibility of character evidence, including prior bad acts). further restricting the use of inculpatory hearsay when declarants are not available,126 Fed. R. Evid. 801–04 (regulating admissibility of hearsay evidence). limiting the use of unreliable eyewitness testimony,127The Supreme Court adopted a “reliability” test in the eyewitness area, but it has been long viewed as ineffectual in practice and focusing on a range of factors not associated with reliability. Manson v. Brathwaite, 432 U.S. 98, 110–14 (1977); see, e.g., Brandon L. Garrett, Eyewitnesses and Exclusion, 65 Vand. L. Rev. 451, 452 (2012); Gary L. Wells & Deah S. Quinlivan, Suggestive Eyewitness Identification Procedures and the Supreme Court’s Reliability Test in Light of Eyewitness Science: 30 Years Later, 33 Law & Hum. Behav. 1, 9–18 (2009). curtailing interview practices that lead to questionable confessions,128For an explanation of the Supreme Court’s turn away from reliability in its Fifth Amendment jurisprudence concerning confessions, see, for example, Richard A. Leo, Steven A. Drizin, Peter J Neufeld, Bradley R. Hall & Amy Vatner, Bringing Reliability Back In: False Confessions and Legal Safeguards in the Twenty-First Century, 2006 Wis. L. Rev. 479, 493–94. and rigorously regulating questionable expert evidence offered by the prosecution129 Fed. R. Evid. 702 (regulating admissibility of expert evidence). For an overview of the reluctance of courts to limit forensic expert evidence introduced in criminal cases, despite common situations in which there is lacking evidence of its reliability, see, for example, Comm. On Identifying the Needs of the Forensic Scis. Cmty., Nat’l Rsch. Council, Strengthening Forensic Science in the United States 4 (2009), [], and Peter J. Neufeld, The (Near) Irrelevance of Daubert to Criminal Justice and Some Suggestions for Reform, 95 Am. J. Pub. Health S107, S107–S110 (2005). would likely do more to protect against wrongful convictions than expanding on the meaning of the presumption of innocence or reasonable doubt burden. We expect jurors to bring their unique perspectives and values to the jury room, and it is naïve to think that judicial instructions can meaningfully alter jurors’ legal personalities.

Of course, we could impose a rule that jurors who express a preference for avoiding acquittals of the guilty will be excluded from juries for good cause, just as jurors opposed to the death penalty can be excluded from death-qualified juries.130See Witherspoon v. Illinois, 391 U.S. 510, 513–14 (1968). Given the diversity of citizens who inhabit the large middle ground category of those equally averse to false convictions and false acquittals, such an automatic exclusion would not systematically exclude members of any identifiable group. But that exclusion would beg the question of whether to exclude those in the dominant middle ground category as well. While this group tends to be less extreme in its reactions to prosecution versus defense evidence, we have found consistent differences between those averse to false convictions and those equally averse to false convictions and false acquittals. Given that this middle ground group constitutes the majority of prospective jurors, their exclusion from jury service would be impractical. Further, excluding jurors who do not embrace Blackstone’s principle could well lead to juries that are too averse to false convictions and refuse to convict when there is any doubt rather than reasonable doubt.

Ultimately, if the law is committed to erring on the side of letting the guilty go free, the most realistic means of doing so will involve stricter evidentiary rules for the prosecution or, as we discuss next, by focusing on pretrial mechanisms that more stringently test the state’s case.

B. Implications for Constitutional Criminal Procedure

Justice Harlan’s concurrence in In re Winship provides the classic rationale for the reasonable doubt standard: “I view the requirement of proof beyond a reasonable doubt in a criminal case as bottomed on a fundamental value determination of our society that it is far worse to convict an innocent man than to let a guilty man go free.”131In re Winship, 397 U.S. 358, 372 (1970) (Harlan, J., concurring). The assumption that the presumption of innocence and reasonable doubt burden serve their intended functions is the primary justification for not imposing additional safeguards against wrongful conviction. As Justice Sandra Day O’Connor put it, “Our society has a high degree of confidence in its criminal trials, in no small part because the Constitution offers unparalleled protections against convicting the innocent.”132Herrera v. Collins, 506 U.S. 390, 420 (1993) (O’Connor, J., concurring). With that faith in hand, the Supreme Court saw no need to recognize a post-trial right to raise a claim of actual innocence under the Constitution.133Id. at 420–21.

A wide range of constitutional criminal procedure doctrines exists to deny relief to criminal defendants who challenge the reliability and sufficiency of evidence introduced by the prosecution.134 Garrett, supra note 34, at 7–8; George C. Thomas III, The Supreme Court On Trial 168–80 (2008); Darryl K. Brown, The Decline of Defense Counsel and the Rise of Accuracy in Criminal Adjudication, 93 Calif. L. Rev. 1585, 1592–1613 (2005); Keith A. Findley, Toward a New Paradigm of Criminal Justice: How the Innocence Movement Merges Crime Control and Due Process, 41 Tex. Tech L. Rev. 133, 141–47 (2008); Joseph D. Grano, Kirby, Biggers, and Ash: Do Any Constitutional Safeguards Remain Against the Danger of Convicting the Innocent?, 72 Mich. L. Rev. 717, 722–25 (1974); Dan Simon, The Limited Diagnosticity of Criminal Trials, 64 Vand. L. Rev. 143, 146–47 (2011). Constitutional criminal procedure, put simply, greatly protects the choices of police, prosecutors, and jurors lest the guilty avoid conviction due to “mere technicalities” and rules that place the interests of criminals above those of the public. One can plausibly argue that the Supreme Court, in the decades following the Warren Court’s strong embrace of the Blackstone principle, has been engaged in the anti-Blackstone project of revising criminal procedure rules and post-conviction standards to focus more on ensuring that the guilty not go free than that the innocent not be convicted.135See, e.g., Carol S. Steiker, Counter-Revolution in Constitutional Criminal Procedure? Two Audiences, Two Answers, 94 Mich. L. Rev. 2466 (1996). At a doctrinal level, one can argue that those who worry that Blackstone’s principle too strongly drives constitutional criminal procedure ignore “the past twenty to thirty years of Supreme Court rulings on the protections of the Fourth and Fifth Amendments.”136See Appleman, supra note 22, at 94.

Our response to continuing concern about the influence of Blackstone’s principle on criminal procedure is that the key mechanism put in place to make Blackstone’s prescription a reality—the state’s high burden of proof—likely does not function as the Supreme Court assumes. The strong testing of the state’s evidence that the Court envisions likely only applies to the minority of Americans who weigh false convictions more heavily than false acquittals. If the law truly prefers Blackstone’s principle, then it should reconsider how it institutionalizes that ratio in criminal procedure.

One solution, as we have discussed, is that defense lawyers should query potential jurors on their views about Blackstone’s principle, and they should favor those who agree with Blackstone over those who do not. Perhaps a simpler approach, such as asking the primary error aversion question we have used in our studies, would be acceptable to judges. The goal, after all, is to identify fair and impartial jurors and, conversely, jurors who may have a bias that justifies exclusion for cause or by peremptory challenge.137See United States v. Wood, 299 U.S. 123, 145–46 (1936) (“Impartiality is not a technical conception. It is a state of mind. For the ascertainment of this mental attitude of appropriate indifference, the Constitution lays down no particular tests and procedure is not chained to any ancient and artificial formula.”). The Supreme Court has held that the “essential demands of fairness” under the Due Process Clause can require, for example, that the defense be permitted inquiry into racial prejudice among potential jurors.138Ham v. South Carolina, 409 U.S. 524, 526 (1973) (quoting Aldridge v. United States, 283 U.S. 308, 310 (1931)). That said, inquiry into other topics may not be permitted, much less required, and generally speaking, judges have a great deal of discretion to manage and limit the voir dire process.139Jeffrey M. Gaba, Voir Dire of Jurors: Constitutional Limits to the Right of Inquiry into Prejudice, 48 U. Colo. L. Rev. 525, 531 (1977); Terry Fox, Judicial Restrictions on Voir Dire: Have We Gone Too Far?, 97 Denv. L. Rev. 327 (2020); see also Fed. R. Crim. P. 24.

Assuming that voir dire for error aversions is permitted, defense lawyers and prosecutors may contest what follows. Judges may not agree that a juror can be stricken for cause, for example, if large numbers of the venire candidly disclose that they fail to accept the Blackstone principle, and where those error aversions reflect general preferences and not partiality toward the litigants in the specific case.140Swain v. Alabama, 380 U.S. 202, 220 (1965) (describing for cause challenges as appropriate for “narrowly specified, provable and legally cognizable bas[es] of partiality”). Judges will also have to rule on objections if prosecutors seek to strike jurors that espouse the Blackstone principle.

At trial, even if voir dire does not fully address the concerns we have raised, a range of strategies could address the error aversions of jurors. Lawyers could acknowledge error aversions in their closing arguments. Defense lawyers, for example, could do more than highlight that the burden of proof in criminal cases reflects a strong view that acquitting the guilty is of less concern than the harm of convicting the innocent. Defense lawyers could also highlight that there is not necessarily a tradeoff between the two types of errors because if an innocent person is convicted for a crime that occurred, then the guilty person remains free. By framing the issue in this way, defense counsel might appeal to jurors who view both types of errors as equally important. The judge could also offer an instruction to that effect. Such an instruction would only complement the presumption of innocence and burden of proof instructions already provided. Indeed, such an instruction may far more clearly and directly address often quite confusing instructions geared toward explaining how high the level of certainty must be to convict “beyond a reasonable doubt.”141See Jon O. Newman, Taking “Beyond a Reasonable Doubt” Seriously, Judicature, Summer 2019, at 33, 34 (describing the “imprecise meaning” of the phrase “beyond a reasonable doubt” and the “defects” of standard jury charge language).

Alternative means of institutionalizing the Blackstone principle matter even more during a time when criminal trials are vanishing, and our system is primarily one of plea bargaining.142Marc Galanter, The Vanishing Trial: An Examination of Trials and Related Matters in Federal and State Courts, 1 J. Empirical Legal Stud. 459, 493 (2004). As the U.S. Supreme Court has recognized: “[C]riminal justice today is for the most part a system of pleas, not a system of trials.”143Lafler v. Cooper, 566 U.S. 156, 170 (2012); see also Missouri v. Frye, 566 U.S. 134, 144 (2012) (“ ‘To a large extent [, horse trading] determines who goes to jail and for how long. . . . It is not some adjunct to the criminal justice system; it is the criminal justice system.’ ”) (quoting Robert E. Scott & William J. Stuntz, Plea Bargaining as Contract, 101 Yale L.J. 1909, 1912 (1992)). It is widely recognized that plea bargaining often does not occur in the shadow of likely trial outcomes144See Stephanos Bibas, Plea Bargaining Outside the Shadow of Trial, 117 Harv. L. Rev. 2463, 2471–72 (2004). and that, in turn, protections at the plea negotiation stage are far more important than trial protections.145See id.; see also William J. Stuntz, Plea Bargaining and Criminal Law’s Disappearing Shadow, 117 Harv. L. Rev. 2548, 2549 (2004); Darryl K. Brown, Judicial Power to Regulate Plea Bargaining, 57 Wm. & Mary L. Rev. 1225, 1228 (2016). Our results bolster efforts to refocus the discussion on what pretrial protections might better accomplish accuracy goals, such as enhanced criminal discovery during the plea process and enhanced pretrial screening of the reliability of evidence.146See, e.g., Leo et al., supra note 128; Brown, supra note 134, at 1590–91; Brandon L. Garrett, Evidence-Informed Criminal Justice, 86 Geo. Wash. L. Rev. 1490 (2018). The prosecution need not provide disclosure of exculpatory evidence to the defense during the plea negotiation process and defendants need not be made aware of the full extent of the evidence against them before trial. United States v. Ruiz, 536 U.S. 622 (2002); see also Mary Prosser, Reforming Criminal Discovery: Why Old Objections Must Yield to New Realities, 2006 Wis. L. Rev. 541, 572.

Reforms can also occur at the appellate level, with appellate courts more carefully scrutinizing convictions.147See Jon O. Newman, Beyond “Reasonable Doubt”, 68 N.Y.U. L. Rev. 979, 1000–02 (1993). Under the Supreme Court’s Jackson v. Virginia standard, appellate courts currently view the evidence in the light most favorable to the prosecution and assume that jurors reasonably determined guilt.148443 U.S. 307 (1979). Yet “it is a romantic notion that the jury should be an infallible determiner of credibility,” a notion that other countries do not entertain as they examine more searchingly whether a conviction is “unsafe or unsatisfactory.”149Newman, supra note 147, at 998, 1001 (citing Criminal Appeal Act 1968, c.19, § 2(1)(a) (UK)), []. That standard has been strengthened in subsequent amendments; for an overview of the evolution of new evidence of innocence claims in England and in other countries, see Brandon L. Garrett, Towards an International Right to Claim Innocence, 105 Calif. L. Rev. 1173 (2017). A middle ground approach that balances error concerns would be to apply special appellate scrutiny in cases that pose a demonstrably higher risk of wrongful conviction, such as cases based on a single eyewitness identification or cases involving recovered memories.150Newman, supra note 147, at 998 (“If we are not going to be rigorous in enforcing the ‘reasonable doubt’ standard in all cases, at least we should do so in those cases where we know the risk of convicting the innocent is higher than ordinary.”).

Whatever approach is taken, our data provide a strong rationale for courts to reconsider how to police accuracy and fairness in criminal adjudications and to stop placing so much weight on the presumption of innocence and burden of persuasion. Yet these reforms need not be led by the courts. Public movements to reform police practices, decriminalize marijuana, and elect prosecutors committed to rethinking the severity of prosecutorial decisionmaking demonstrate the public will embrace common sense reforms that balance fairness to litigants against public safety.151See, e.g., Note, The Paradox of “Progressive Prosecution”, 132 Harv. L. Rev. 748 (2018) (critiquing progressive prosecution movement); The President’s Task Force on 21st Century Policing, Final Report 9–11 (2015), [] (describing police reform agenda); Robert A. Mikos, The Evolving Federal Response to State Marijuana Reforms, 26 Widener L. Rev. 1 (2020). The median voter, just like the median juror, cares both about treating those accused of crimes fairly and about punishing those who pose real threats to public safety.

C. Implications for Crime Policy

The views of the American public concerning the criminal system have been much scrutinized in recent years. In the summer 2020, the largest public protests in American history responded to police brutality and racialized policing and incarceration.152See Derrick Bryson Taylor, George Floyd Protests: A Timeline, N.Y. Times (Nov. 5, 2021), []; Larry Buchanan, Quoctrung Bui & Jugal K. Patel, Black Lives Matter May Be the Largest Movement in U.S. History, N.Y. Times (July 3, 2020), []. At the time, polling suggested that “Americans are largely united behind the idea that action is required.”153Colleen Long & Hannah Fingerhut, AP-NORC Poll: Nearly All in US Back Criminal Justice Reform, AP (June 23, 2020), []. Prior surveys had found that large majorities of Americans believed criminal justice reforms were needed, including a focus on rehabilitation and not just punishment.154Overwhelming Majority of Americans Support Criminal Justice Reform, New Poll Finds, Vera (Jan. 25, 2018), []; see also Barack Obama, Commentary, The President’s Role in Advancing Criminal Justice Reform, 130 Harv. L. Rev. 811, 816 (2017).

Once one considers what actions are needed, however, the public’s views become more complex. The remarkable time period from 2020 to 2021 provides a case in point. The sizable uptick in gun violence during the pandemic, and perhaps shifting politics, may have impacted public views and heightened concern with public safety. Surveys suggest that strong majorities continue to support shifting police budgets to community policing and social services.155E.g., Chris Jackson, As Public Safety Tops the Agenda, Americans Want Both Order and Justice, Ipsos (July 8, 2021), []. And strong majorities continue to support banning abusive police tactics and penalizing police for racially biased conduct.156See Saletan, supra note 33. But sizable majorities of the public also oppose defunding the police and support public safety measures.157Id. These public opinion polls suggest that neither due process nor public safety is the exclusive interest of the broad American middle.

While journalists have at times called these surveys conflicted and have highlighted racial and partisan divides on particular issues like police use of force,158E.g., Susan Page, Calls To ‘Defund the Police’ Clash with Reality for Many Americans, City Polls Show, USA Today (Dec. 1, 2021, 12:02 PM), []. the public opinion landscape is consistent with our finding that most Americans seek to balance fairness and public safety when it comes to criminal justice. Indeed, the public’s error aversions hold relevance not only for criminal procedure but also penal policy and electoral politics. Voters’ error aversions provide a window into their attitudes and beliefs about crime, crime control, and the protection of suspects against false accusations and false convictions.

The median voter is likely to see false convictions and false acquittals as equally serious and thus should support criminal justice reforms that they believe reduce the risk of false convictions without increasing the risk of false acquittals. Likewise, they should support district attorneys and other elected officials who seek to right racial injustices and other criminal justice mistakes without denying the importance of the police in maintaining public order. To these citizens, increased accuracy in criminal justice should be the goal, for it benefits both crime control and due process when the police and courts reach accurate determinations of who committed what crimes. A false conviction means not only that an innocent person suffers but also that a guilty person remains on the streets, perhaps to offend again.159James R. Acker, The Flipside Injustice of Wrongful Convictions: When the Guilty Go Free, 76 Alb. L. Rev. 1629, 1631 (2013) (“Wrongful convictions entail profound social costs in addition to the hardships borne by the unfortunate individuals who are erroneously adjudged guilty. When innocents are convicted, the guilty go free. Offenders thus remain capable of committing new crimes and exposing untold numbers of additional citizens to continuing risk of victimization. Public confidence in the administration of the criminal law suffers when justice miscarries.” (footnotes omitted)).

The lesson for those advocating reforms to penal policy is to consider and articulate how the reform will impact accuracy in criminal justice, highlighting the two types of errors that matter to the public.160Recall that the median participant in our studies was willing to pay an equal amount in increased taxes to reduce both false convictions and false acquittals. The increased costs of such reforms, therefore, should not be an insurmountable hurdle, especially where those costs are unlikely to be large. See supra Section I.B. Some policies, such as ensuring access to DNA testing for the convicted and extending the circumstances under which actual innocence can be asserted as a basis for relief, should enjoy widespread support because they clearly promote accuracy in adjudication without reducing safety.161See, e.g., Andrew E. Taslitz, Convicting the Guilty, Acquitting the Innocent: The ABA Takes a Stand, Crim. Just., Winter 2005, at 18, 18 (describing resolutions in 2004 “designed to improve the justice system’s accuracy in convicting the guilty while acquitting the innocent”). Indeed, post-conviction DNA testing is one reform that rapidly spread across the country, with adoption in all fifty states in recent years.162Access to Post-Conviction DNA Testing, Innocence Project, [].

Other reform efforts that flow primarily from deontological concerns, like the inherent wrong of racial bias in policing or invasions of privacy from modern surveillance techniques, will probably not move most voters unless one explains how the reform will promote justice without unduly reducing public safety or emboldening the guilty. For such issues, framing may deeply matter, not just on public opinion surveys or mock jury experiments, but when lawmakers put proposals before legislatures and the public.

It is exactly that broader conceptualization of public safety and the costs of traditional policing that have occurred in recent years.163See supra notes 34–36; see also Barry Friedman, What Is Public Safety?, 102 B.U. L. Rev. 725 (2022). Convincing the public that overpolicing, pretrial detention, lengthy sentences, and racial disparities associated with those policies do not promote crime control goals but do raise grave due process concerns should increase support for reforms.164See Russell M. Gold & Ronald F. Wright, The Political Patterns of Bail Reform, 55 Wake Forest L. Rev. 743 (2020); Ekow N. Yankah, Pretext and Justification: Republicanism, Policing, and Race, 40 Cardozo L. Rev. 1543, 1547–49 (2019). Conversely, it may be difficult to convince voters of the need for reforms using arguments about the disparate impacts of criminal justice policies, like with the possible racial impacts from the use of algorithms in policing and sentencing,165For measuring racial disparities in pretrial outcomes, see, for example, Crystal S. Yang & Will Dobbie, Equal Protection Under Algorithms: A New Statistical and Legal Framework, 119 Mich. L. Rev. 291, 343–357 (2020). where the public is convinced (rightly or wrongly) that such policies improve public safety.

The rise of mass incarceration provides another telling example. For decades, voters and lawmakers supported and enacted more severe sentencing laws across the United States.166See Carol S. Steiker, Keeping Hope Alive: Criminal Justice Reform During Cycles of Political Retrenchment, 71 Fla. L. Rev. 1363, 1364 (2019) (“Between 1975 and 1996, the most frequently enacted sentencing law change was the adoption of mandatory minimum penalties.”). Only when it became apparent that many of these policies were both unfair and counterproductive did bipartisan interest in sentencing reform arise.167See Jessica Kelley & Arthur Rizer, Keep Calm and Carry On with State Criminal Justice Reform, 32 Fed. Sent’g Rep. 86, 87 (2019) (describing how “ ‘law and order’ states started to move the rudder on criminal justice reform”); Steiker, supra note 166, at 1368 (“To a degree unthinkable in previous decades, left–right coalitions at all levels of government began to unite on a variety of criminal justice reforms, agreeing on the fundamental premise that punishments had become too harsh and rehabilitative options too scarce.”). To succeed in changing course, however, policy advocates must continue to make clear that overly harsh sentencing and incarceration approaches implicate both fairness and crime control. Our findings suggest criminal justice proposals that treat due process and accuracy as complements, rather than competitors, will have a much greater chance of adoption.


The median juror rejects William Blackstone’s 10:1 ratio in favor of a 1:1 ratio. They believe that false convictions and false acquittals constitute serious mistakes that should both be minimized to the greatest extent possible. They reject the pure due process model that underlies our constitutional criminal procedure protections and beyond-a-reasonable-doubt standard. Yet other jurors disagree even more strongly with Blackstone and consider false acquittals to be the more serious mistake. Some jurors do agree with Blackstone and view false convictions as the graver error, but the percentage of the population in agreement with Blackstone is a minority that is much smaller than previously assumed.

These differences in error aversions have direct implications for how jurors perform their duties and for the efficacy of constitutional protections of criminal defendants’ due process rights. Jurors who disagree with Blackstone are more open to evidence from the prosecution, more skeptical of evidence from the defense, and more prone to conviction than jurors who agree with Blackstone.

This empirical reality should inform trial practice and the content of criminal procedure. Defense lawyers should query potential jurors on their views about Blackstone’s ratio and should favor those who agree with Blackstone over those who do not. Judges charged with developing criminal procedure law should recognize that the presumption of innocence and reasonable doubt standard fail to produce the desired error tradeoff in jurors. And the Supreme Court, policymakers, and the public should consider alternative ways of ensuring that the innocent not be imprisoned or put to death if Blackstone’s ratio is to remain a foundation for our criminal justice system.

As Judge Jon O. Newman put it in his Madison Lecture, “American courts have permitted their unbounded enthusiasm for the jury to dilute the rigor of their enforcement of the ‘reasonable doubt’ standard as a rule of law.”168See Newman, supra note 147, at 1002. Our findings suggest there is far more that courts can do, but also far more for lawyers and policymakers to do as well to assure fair and accurate justice. The inquiry into whether there is sufficient evidence to convict a person is inherently “subjective.”169Id. There is a constitutional command, however, and a public responsibility to ensure that sufficient evidence supports criminal convictions.

The movement to address longstanding inequities in our criminal system could not be more urgent today. Our findings regarding the centrality of error aversions help to inform that work. For those who are not trained in the law, Blackstone’s ratio, despite its centrality to our constitutional criminal procedure, is not self-evident. It is a mistake to assume the general public will share the concerns of progressives who seek to right the wrongs of our criminal justice system by sharply limiting the policing function, granting defendants more rights, and abolishing prisons. Americans across the political spectrum want both fairness and safety—they want to avoid convicting the innocent and acquitting the guilty. Courts and reform advocates should take these error aversions into account when evaluating how our legal system functions and how best to reform it.

 *   L. Neil Williams Professor of Law & Director, Wilson Center for Science and Justice, Duke University School of Law.

 **   Joseph Weintraub–Bank of America Distinguished Professor, University of Virginia School of Law. Order of authorship determined alphabetically. Our thanks to Jeffrey Bellin, Christopher Buccafusco, James Doyle, David Faigman, Adam Gershowitz, Yehonatan Givati, John Monahan, Adele Quigley-McBride, Roseanna Sommers, Jennifer Teitcher, Jenia Turner, Christopher Slobogin, and participants in an American Psychology-Law Society Roundtable and at a faculty workshop at William & Mary School of Law for helpful comments on earlier versions of this Article. Eric Tucker provided excellent research assistance for this Article. Self-citations by Garrett: 9; self-citations by Mitchell: 4; colleague citations by Garrett: 3; colleague citations by Mitchell: 2. For an explanation of this citation disclosure, see Andrew T. Hayashi & Gregory Mitchell, Maintaining Scholarly Integrity in the Age of Bibliometrics, 69 J. Legal Educ. 138, 150–51 (2021).