The Missing Algorithm: Safeguarding Brady Against the Rise of Trade Secrecy in Policing

Deborah Won* October, 2021

Trade secrecy, a form of intellectual property protection, serves the important societal function of promoting innovation. But as police departments across the country increasingly rely on proprietary technologies like facial recognition and predictive policing tools, an uneasy tension between due process and trade secrecy has developed: to fulfill Brady’s constitutional promise of a fair trial, defendants must have access to the technologies accusing them, access that trade secrecy inhibits. Thus far, this tension is being resolved too far in favor of the trade secret holder—and at too great an expense to the defendant. The wrong balance has been struck.

This Note offers three contributions. First, it explains the use of algorithms in law enforcement and the intertwined role of trade secrecy protections. Second, it shows how trade secrecy clashes with the Due Process Clause—the Constitution’s mechanism for correcting the power asymmetry between the state and the defendant—and argues that due process should not waver simply because a source of evidence is digital, not human. Third, it proposes a solution that better balances a defendant’s due process rights with intellectual property protections.

Introduction

Algorithmic policing is on the rise. Investigative tools like facial recognition, DNA genotyping, and predictive policing systems are increasingly—and effectively—being marketed by private technology companies as the best way to police efficiently under tight budget constraints.¹ But because algorithmic systems are built by humans, they exhibit human fallibilities, including racial and gender bias, inconsistency, and error.² Despite their similarities, algorithmic systems and humans are treated differently when used against a criminal defendant in court. Humans are subject to adversarial scrutiny; algorithmic systems are not.³ The difference is attributable to trade secrecy, a form of intellectual property protection designed to maintain “standards of commercial ethics” and “encourage[] invention.”⁴ When invoked in the criminal context, however, trade secrecy shields algorithmic systems from adversarial scrutiny—even when the Constitution mandates it.⁵

It is perplexing that algorithmic systems receive heightened protection as intellectual property given that defendants’ due process rights do not change simply because the investigatory source is digital, not human. This is particularly troubling given the government’s affirmative duty under Brady v. Maryland to disclose helpful evidence to the criminal defendant.⁶ The Brady obligation, established by the Supreme Court to ensure that the defendant receives a full and fair trial under the Due Process Clause, requires that the prosecutor turn over any information “favorable” to the defendant that is in the prosecutor’s constructive “possession,” so long as that evidence is “material” to the defendant’s case.⁷ Favorable information includes both impeachment⁸ and exculpatory⁹ evidence. The prosecutor must actively search for favorable evidence not just in their own possession, but also in the possession of any member of the “prosecution team.”¹⁰ The prosecutor’s duty to search and disclose is “ongoing.”¹¹

However, prosecutors need only disclose favorable evidence that is “material.” The standard for determining materiality may differ depending on whether a prosecutor’s Brady compliance is evaluated before or after the trial’s conclusion. The post-trial materiality standard is well established. A defendant that raises a Brady challenge after trial must show prejudice—that there is a “reasonable probability” disclosure of the evidence would have resulted in a different verdict.¹² The pretrial materiality standard, however, lacks uniformity. Some courts have imported the post-trial prejudice requirement into the pretrial standard, while others have rejected it and require only that the evidence be favorable.¹³ Those courts with a mirrored pretrial standard therefore allow prosecutors to determine whether evidence is Brady material based on their own predictions of the eventual trial’s outcome. The pretrial prejudice requirement thus makes it easy for prosecutors to manipulate Brady’s materiality threshold; prosecutors can cobble together any number of arguments that disclosing evidence would not affect the ultimate verdict. Thus, to maintain the central tenet of Brady—that defendants be treated fairly as they face the machinery of the state—defendants should not be required to show prejudice before trial to obtain favorable evidence.¹⁴

This Note argues that for purposes of Brady disclosures, courts should view law enforcement algorithms as analogous to human witnesses and should accordingly implement an at-trial “missing algorithm” remedy when trade secrecy is invoked. Part I provides the factual and legal background of law enforcement algorithms and trade secrecy protections to place the question in context. Part II analyzes Brady and its progeny and concludes that algorithmic information falls within a prosecutor’s duty to disclose favorable material to the defendant. Part III proposes that courts adopt a missing algorithm rule, allowing juries to draw reasonable and limited inferences to safeguard defendants’ due process rights when their access is limited by intellectual property protections.

I. Law Enforcement Algorithmic Systems and Trade Secrecy

Machine-learning algorithms are increasingly executing government functions. One context in which algorithmic systems are proliferating is law enforcement and prosecution.¹⁵ As public attention to automated decisionmaking has increased, terms like “algorithm,” “machine learning,” and “predictive policing” have become buzzwords, often used loosely and interchangeably.¹⁶ Section I.A provides definitions of those terms and a high-level explanation of how an “algorithmic system” works. Section I.B summarizes U.S. trade secrecy law and explains which components of an algorithmic system can be shielded under trade secret protections.

A. What Are Algorithmic Systems?

To understand what a law enforcement algorithmic system is, it is necessary to first unpack what an algorithmic system is. An algorithmic system involves several separate technical components. As the term suggests, the foundation of the system is the “algorithm,” a specified series of logical steps used to accomplish some task.¹⁷ The algorithm is operationalized by source code. Source code is a series of letters, numbers, and punctuation that give the computer instructions on how to act in accordance with the algorithm.¹⁸ Systems vary greatly in how many lines of source code they contain.

A “machine-learning” algorithm is an algorithm that is “taught” on training data to perceive patterns and to subsequently become better at discerning new patterns when exposed to new information.¹⁹ Training data is a collection of examples from which the algorithm is instructed to extract logical rules.²⁰ “Verification” and “test” data sets are then used to score and refine the performance of the algorithm.²¹

In addition, an algorithmic system requires inputs to produce a desired output. The input is the information fed into the algorithm, and the output is the information created by applying the algorithm to the input data—for example, whether a person’s picture has a match in a facial recognition database. Though every algorithmic system requires some kind of input and produces an output, they vary widely depending on the purpose or design of the system and can also vary over time.²²

These technical components—the algorithm, training data, input, and output—are limited by policy decisions implemented by the designer or the user.²³ Indeed, policies, which are the result of human choices, “govern how both technical and human components of th[e] system should behave.”²⁴ These policy limits might take the form of certain prohibitions built into the algorithm, or they might instruct the user on how to act upon or interpret an algorithm’s output.²⁵

A “law enforcement algorithmic system,” as used in this Note, is an algorithmic system that is used by government entities like police departments for surveillance, investigation, or prosecution purposes.²⁶ One increasingly common example of a law enforcement algorithmic system is facial recognition technology.²⁷ A facial recognition algorithm, broadly speaking, is trained to identify faces by analyzing images in a historical dataset.²⁸ A police officer or analyst can then input an image into the algorithm.²⁹ The output is a series of similar photos, usually with a probability ranking to denote the likelihood of a “match.”³⁰ Depending on the policies in place, the police officer may or may not act upon the similar photos by finding and detaining any identified individuals.

Each technical component of an algorithmic system involves human judgment. The sequential logical steps embodied by the algorithm and operationalized by the source code are written and designed by humans.³¹ The training data is selected by humans. The input is chosen and the output is interpreted by humans. Thus, each component risks human error and human bias.³²

For example, the historical datasets on which facial recognition and other machine-learning algorithms are trained are often skewed by race and gender.³³ One dataset, deemed the “gold standard benchmark for face recognition,” was found to be approximately 83.5 percent white and 77.5 percent male.³⁴ Unsurprisingly, researchers have found that algorithmic systems are one hundred times more likely to misidentify African and Asian Americans than white individuals,³⁵ and are also more likely to misidentify women than men.³⁶ Illustrating this problem, one study tested Rekognition, Amazon’s facial recognition tool, and found that it incorrectly identified twenty-eight members of Congress as people from a criminal database.³⁷ Nonwhite members of Congress were disproportionately misidentified, at about 40 percent of false positives, despite making up only about 20 percent of Congress. That false positives are much higher for racial minorities is particularly concerning given that minorities are also more likely to be subjected to facial recognition searches for law enforcement purposes.³⁸

Training data may also be racially skewed because of past discriminatory police practices. When Black men are disproportionately targeted and arrested by police for drug crimes, or when predominantly Black neighborhoods are disproportionately targeted for unjustified police scrutiny and intrusion—to list just a few of the multitude of examples—those racially biased arrests and police contacts create skewed historical data in which Black men are overrepresented.³⁹ That overrepresentation, often interpreted as indicating greater criminality, may reinforce the biases underlying the discriminatory practices and produce a flawed, circular justification for increasing the policing of already overpoliced communities.⁴⁰ Then, when that biased historical data is used to train algorithms, the discrimination becomes “baked in” and can perpetuate and exacerbate the bias—a “garbage in, garbage out” problem.⁴¹

For instance, in 2012 Chicago deployed the Strategic Subject List (SSL), an algorithmic tool that ranked individuals at risk of becoming either victims or offenders in a shooting or homicide.⁴² The tool assigned individuals a risk tier, ranging from “very low” to “very high,”but did not differentiate between potential “victims” and “offenders.”⁴³ The tool was problematic not only because it weighed arrests rather than convictions as a factor—meaning that the list likely reflected the Chicago Police Department’s discriminatory practices of targeting nonwhite individuals and communities—⁴⁴ but also because the factor that most affected a person’s risk score was age.⁴⁵ Indeed, 56 percent of all Black men under thirty in Chicago were listed and assigned risk scores by the SSL.⁴⁶ Notably, Black men under thirty were exactly the same demographic targeted by the Chicago Police Department’s unlawful stop-and-frisk practices.⁴⁷ From these facts, researchers concluded that the tool likely ingested racially skewed data.⁴⁸

Inclusion on the list had serious consequences—two-thirds of the individuals on the list were flagged to receive heightened police scrutiny and were more likely to be arrested as a direct result of being included on the list.⁴⁹ While it is unclear how officers actually used SSL predictions in the field,⁵⁰ one Chicago Police Department directive encouraged “[t]he highest possible charges” for all individuals on the list, regardless of their risk score, so long as they had received notice about being on the list and were subsequently arrested.⁵¹ The SSL is thus a cautionary example of how a tool trained on “dirty data” risks “perpetuating additional harm via feedback loops”⁵²—harm laden with grave privacy and civil rights implications for those targeted individuals.⁵³

Furthermore, even a simple “accident” in the source code, such as a mistaken ampersand, can result in substantive differences.⁵⁴ One study found that 33 percent of “highly experienced” programmers failed to properly use parentheses, which in complex programs could result in “tens of thousands of errors.”⁵⁵ For example, source code errors in STRmix, a probabilistic DNA genotyping program,⁵⁶ produced incorrect results—stating a percentage likelihood that the defendant’s DNA matched a sample at the crime scene when that percentage was off by a factor of ten—in sixty criminal cases.⁵⁷ Prosecutors were forced to replace twenty-four expert statements after that coding-error discovery.⁵⁸ In one New York trial involving a different DNA genotyping program, the judge granted the defense access to the program despite the government’s objections to access on “proprietary” grounds.⁵⁹ Review of the source code uncovered that the program had dropped essential data from its calculations in ways that would have unpredictably changed the program’s DNA probability results.⁶⁰ Once the defense expert witness testified to that effect, the prosecution withdrew the DNA evidence against the defendant.⁶¹

Additionally, inputs are selected by humans, who may feed nonsensical data to the algorithm.⁶² Yet another variation of the garbage in, garbage out problem, such inputs would only produce nonsensical outputs.⁶³ For example, when the New York Police Department (NYPD) was investigating a beer theft, it first fed the facial recognition tool a store surveillance image of the actual culprit.⁶⁴ But when that input produced no match, NYPD decided to input an image of celebrity Woody Harrelson because one officer remarked that the culprit resembled a Woody Harrelson with long hair.⁶⁵ Based on the Woody Harrelson “matches,” the police arrested the defendant (who was not Woody Harrelson).⁶⁶ While NYPD declined to admit what happened to the non-Woody Harrelson after the arrest,⁶⁷ the incident illustrates how prone to human error, and thus how fallible, algorithms can be. In addition to celebrity images, records show that police departments use other concerning inputs such as artist sketches and graphically modified images.⁶⁸

Similarly, outputs may also be selectively chosen by humans in problematic ways. For example, Willie Lynch was convicted in Florida for selling fifty dollars’ worth of drugs based on a facial recognition database called Face Analysis Comparison Examination System (FACES).⁶⁹ During a drug sale, police officers surreptitiously photographed the seller with a cell phone camera. The photos were blurry in places and taken at an oblique angle.⁷⁰ The officers, not knowing who the individual was, did not make an arrest on the scene.⁷¹ Instead, they sent the cell phone photos to an analyst.⁷² That analyst fed the low-quality photos into FACES, which produced five possible African American suspects, including Lynch.⁷³ FACES only designated Lynch at a one-star level of confidence—the lowest number of stars for “likelihood” that the match was accurate.⁷⁴ Despite the low rating, the analyst selected Lynch and submitted his identification information to the officers. No other matches were submitted. The officers promptly arrested Lynch. At trial, Lynch raised just one defense—that he was mistakenly identified as the seller.⁷⁵ Lynch was convicted, and on appeal he asked to see the other matches produced by FACES because it would help his defense.⁷⁶ The Florida appeals court denied Lynch’s request. The court reasoned that because Lynch could not show that the other algorithm-produced matches resembled him, he could not show that access to those photos would support his argument that he was mistakenly identified.⁷⁷ Of course, the court did not address how Lynch would have been able to show that those photos resembled him without any access to the photos in the first place.⁷⁸ Willie Lynch was sentenced to eight years of incarceration and is currently in prison.

This Section has shown that the potential for error—the same flaws, biases, and lapses in judgment that humans exhibit—is embedded in every component of algorithmic systems. However, while that commonality would urge the same scrutiny for both human and algorithmic error in criminal cases, trade secrecy shrouds the latter.

B. What Are Trade Secrets and What Can They Shield from Disclosure?

There are four forms of intellectual property: copyright, patents, trademarks, and trade secrets.⁷⁹ One central purpose of intellectual property protection is to promote innovation by ensuring the creator has the financial incentives to innovate given the significant investment of resources required to do so.⁸⁰ All four forms of intellectual property thus provide some legal mechanism that protects the innovator’s ability to reap the financial benefits of their investment.⁸¹

Trade secrecy’s mechanism targets misappropriation: the improper use, acquisition, or disclosure of a trade secret.⁸² A trade secret designation offers “powerful legal protections to companies that want to keep their business practices a secret.”⁸³ Trade secret holders receive two types of protection. First, substantive trade secret protection allows proprietors to recover monetary damages for trade secret misappropriation and to secure injunctive relief against future misappropriation.⁸⁴ Second, evidentiary trade secret protection allows proprietors to shield their trade secret when in court, preventing business competitors from exploiting court proceedings to obtain commercially valuable information.⁸⁵

Trade secrets are the youngest and least restrained form of intellectual property protection.⁸⁶ For example, while patent protection requires one to undergo a formal application and approval process and has a set expiration date,⁸⁷ trade secret protection requires no formal process and can extend indefinitely.⁸⁸ There is no uniform definition of trade secrets, but they are generally described as “confidential, commercially valuable information.”⁸⁹ To procure protection, the purported trade secret needs to meet just two requirements: (1) it derives independent economic value from being generally not known and (2) reasonable efforts have been made by the trade secret holder to maintain its secrecy.⁹⁰

The trade secret evidentiary privilege raises the defendant’s burden for discovery. When it is invoked, the defendant must make a “particularized showing” that what they seek is “necessary to the defense.”⁹¹ If the defendant cannot show necessity, then the proprietor can shield the algorithmic information entirely; the defendant is not allowed access even with a protective order.⁹²

Thus, trade secrecy can be invoked to shield algorithmic systems from scrutiny at all stages of a case or proceeding.⁹³ Specifically, the algorithm and source code, training data, user manuals, and audit information can be claimed as proprietary under the expansive definition of trade secrets.⁹⁴ Law enforcement technology proprietors and prosecutors have already begun to invoke trade secrecy to avoid disclosure and scrutiny of their algorithmic systems.⁹⁵

The invocation of trade secrecy to shield law enforcement algorithmic systems is likely to increase. In 2014, the Supreme Court made it more difficult to patent software in Alice Corp. v. CLS Bank International,⁹⁶ thereby encouraging developers to use trade secrecy rather than patents to protect their intellectual property.⁹⁷ And in 2016, Congress established the first federal cause of action for trade secret misappropriation by passing the Defend Trade Secrets Act.⁹⁸ The availability of a federal cause of action enables proprietors to more easily and efficiently pursue out-of-state and out-of-country infringers, making trade secrecy an appealing choice.⁹⁹

Additionally, state courts are increasingly adopting a criminal trade secret evidentiary privilege.¹⁰⁰ At least twenty-one states have a codified trade secret privilege in their evidence code, and many states recognize a common law privilege.¹⁰¹ While the trade secret privilege was often invoked in civil cases, it was not until 2015 that the evidentiary privilege was first extended to a criminal case.¹⁰² Since that first case, courts across the country are beginning to expressly allow the privilege to withhold evidence from defendants.¹⁰³ Indeed, in a span of just five years, from 2013 to 2018, courts in no fewer than five states denied access to algorithmic information due to trade secrecy.¹⁰⁴ These developments, alongside the lack of any formal claim requirements or expiration date,¹⁰⁵ heavily incentivize algorithm developers to rely more on trade secret protections than on other branches of intellectual property.¹⁰⁶

Because evidentiary privileges “apply to all stages of a case or proceeding,”¹⁰⁷ trade secrecy protections can be invoked before trial, at trial, and after trial.¹⁰⁸ Before trial, proprietors can invoke the privilege against defendants seeking information about the algorithmic systems used in the law enforcement investigation—information that the defendant could have used in suppression hearings.¹⁰⁹ At trial, the proprietor (or the prosecutor on behalf of the proprietor) can invoke the privilege to shield an algorithmic system used as evidence.¹¹⁰ After trial, the proprietor can invoke the privilege in bail or sentencing proceedings if the defendant seeks access to a risk-assessment algorithmic tool.¹¹¹ In addition, proprietors may invoke the privilege if a convicted defendant seeks an appeal.¹¹²

Law enforcement algorithmic tools are increasingly being used across the country. Surveillance technology companies like Palantir and PredPol that market to government entities and police departments claim that their machine-learning technology is the best way to efficiently police under tight budget constraints.¹¹³ Thus, products like facial recognition technology have proliferated. For example, from 2010 through 2016, facial recognition technology resulted in approximately 2,800 arrests.¹¹⁴ In 2018 alone, over 8,000 cases used facial recognition as an investigative tool.¹¹⁵ And companies are developing new uses and new tools to capture the lucrative law enforcement market.¹¹⁶ Because law enforcement algorithmic tools are becoming more prevalent and the incentives for proprietors to choose trade secrecy over other intellectual property protections are increasing, courts will have to grapple more and more with the invocation of trade secrets in criminal proceedings.

II. When Due Process and Trade Secrets Clash

Trade secrets can be invoked to shield from disclosure significant parts of a law enforcement algorithmic tool, including the algorithm, the implementing source code, and the training data (collectively shorthanded here as “algorithmic information”). In the criminal context, trade secrecy protections allow law enforcement and prosecutors to shirk their Brady disclosure responsibilities when they use algorithms to identify, investigate, and prosecute a suspect. This Part discusses the constitutional tension between intellectual property and a defendant’s due process rights and explains why the principles of Brady require that the latter, not the former, prevail.

A. The Brady Doctrine: Human Witnesses and Their Algorithmic Counterparts

In Brady v. Maryland, the Supreme Court held that a prosecutor cannot withhold evidence “favorable to an accused” because such suppression violates the defendant’s due process rights under the Fifth and Fourteenth Amendments.¹¹⁷ The Brady doctrine thus imposes upon prosecutors a duty to learn of and disclose to the defendant any information in their possession that is “favorable” and “material either to guilt or to punishment, irrespective of the good faith or bad faith of the prosecution.”¹¹⁸

The Brady doctrine represents a limited and necessary departure from the pure adversarial model of the United States legal system.¹¹⁹ Such a departure is warranted because the prosecutor represents not an ordinary party but a sovereignty; as such, the prosecutor’s purpose is “not that it shall win a case, but that justice shall be done.”¹²⁰ The Supreme Court created this duty to disclose because “[s]ociety wins not only when the guilty are convicted but when criminal trials are fair; our system of the administration of justice suffers when any accused is treated unfairly.”¹²¹ Ultimately, the Brady requirement is a “rule of fairness”—an essential tenet of our criminal justice system.¹²² The Brady doctrine also serves to safeguard against the inherent power asymmetry between the state and defendant, helping to equalize an otherwise deeply skewed system.¹²³

1. The Rise of Trade Secrecy in Policing

Despite the necessity of the Brady requirement for a fair criminal system, trade secret protections directly clash with, and pose a threat to, the constitutional defenses erected by Brady. While research in this area is limited by barriers to access,¹²⁴ researchers have been able to identify flaws in proprietary algorithms. These flaws, discussed in Part I, are pervasive in law enforcement algorithmic tools. For example, law enforcement algorithms have been found to be systematically biased based on race and gender due to the use of data skewed by past discriminatory police practices in algorithm development.¹²⁵ Because the systems are built and trained upon biased data, they allow police departments to recast past discriminatory practices as infallible new technology, thereby deepening and perpetuating those discriminatory practices.¹²⁶ Law enforcement algorithms can also have mistakes in their source code: even an error as simple as a misplaced ampersand can significantly alter the algorithm’s operation.¹²⁷ Despite mounting evidence of their fallibility, police departments across the country continue to acquire and implement proprietary algorithmic tools—often in secret.¹²⁸

Prosecutors and proprietors are already resisting disclosure of information pertaining to law enforcement algorithms in criminal cases. For example, the New York Police Department (NYPD) used facial recognition technology to find a man who stole a pair of socks in a department store using an image from the store’s surveillance camera.¹²⁹ Multiple potential matches were generated.¹³⁰ From those matches, the NYPD detective selected a photo of Andre¹³¹ and texted an officer who had witnessed the incident, “Is this the guy?”¹³² The officer texted back in the affirmative. Andre’s attorneys argued that the single photograph and text-message confirmation was the “sole basis” for the arrest. When Andre tried to obtain information about NYPD’s use of the facial recognition program, NYPD and prosecutors resisted, claiming that disclosure would violate the trade secrecy of the program’s owner, a third-party vendor. Before the disclosure issue was resolved, Andre took a plea deal.¹³³

Had the other suspect matches come from a human eyewitness rather than from a facial recognition program, the prosecutors would not have been able to shield the eyewitness from scrutiny.¹³⁴ Brady would have required the prosecutors to disclose that the eyewitness had identified multiple possible suspects other than the defendant,¹³⁵ as well as the eyewitness’s level of certainty (or uncertainty) about each identification.¹³⁶ This would be true even if the eyewitness’s testimony were not introduced by the prosecutor at trial and only the witnessing police officer were to testify.¹³⁷ That the prosecutor declined to use the eyewitness at trial does not negate the Brady value of the eyewitness’s uncertainty or unreliability. Further, had the prosecutor introduced the eyewitness at trial (or had the testifying police officer improperly or sloppily relied on such an unreliable investigative lead, as seems likely in Andre’s case),¹³⁸ the prosecutor would be required to disclose information challenging the credibility of the eyewitness—including evidence that the witness was racially biased¹³⁹ or had previously given inconsistent statements.¹⁴⁰ Thus, had the NYPD relied on a human eyewitness in its investigation, Brady would have afforded Andre the evidence he needed to present a full defense. But since the NYPD relied on a proprietary tool and could therefore invoke trade secrecy, Andre was denied that same evidence.¹⁴¹

Cases like Andre’s illustrate the costs of weakening Brady protections in the face of new law enforcement technology.¹⁴² The costs will continue to mount as trade secret protections are increasingly raised by prosecutors and faced by defendants.¹⁴³

Technological advances will also accelerate the rate of trade secret invocations in criminal proceedings. Law enforcement algorithm tools, the components of which can be claimed as trade secrets,¹⁴⁴ are proliferating.¹⁴⁵ These tools are being marketed as a cost-effective investigation solution to law enforcement agencies, who are among the biggest consumers of algorithmic technologies like facial recognition programs.¹⁴⁶ Thus, trade secret protections are likely to be increasingly invoked against defendants’ requests for law enforcement algorithmic information.

As more proprietary tools are adopted by law enforcement, more criminal defendants will need—and more proprietors will resist—access to algorithmic systems. Indeed, a similar trend can be traced with tools that have had more time to be challenged in courts.¹⁴⁷ For example, the use of probabilistic DNA genotyping programs has rapidly expanded in the last ten years.¹⁴⁸ Defendants across the country have requested access to those programs used against them; proprietors have routinely resisted disclosure, citing trade secrecy and commercial concerns.¹⁴⁹ Courts have consistently denied defendants’ access requests.¹⁵⁰

2. The Need to Treat Algorithmic and Human Sources Alike

That algorithmic information enjoys heightened protection as a trade secret is perplexing given that a defendant’s due process rights remain the same regardless of the source of evidence against them—whether that source is digital or human. Developers argue that the disclosure of trade secrets to criminal defendants would jeopardize innovation in criminal justice technology.¹⁵¹ However, while those innovation concerns have merit,¹⁵² procedural mechanisms like protective orders can ensure that trade secrets remain shielded from the public while still allowing access by the defendant.¹⁵³ In fact, protective orders are routinely granted to shield proprietary technology in civil proceedings, showing that such orders are effective.¹⁵⁴ And there are other ways to incentivize innovation of law enforcement algorithms: alternate forms of intellectual property, such as patents; exclusive government contracts contingent on transparency; tax incentives; and prize competitions.¹⁵⁵ Ultimately, the stakes for criminal defendants are far too great to prioritize intellectual property concerns at the expense of constitutional protections. As shown in the remainder of this Section, barring defendant access to algorithmic information on the basis of trade secret protections is in tension with the Brady doctrine. Differentiation between algorithmic and nonalgorithmic material is unwarranted in light of the principles and rationale underlying Brady and its progeny.

Defendants’ inability to access favorable information¹⁵⁶ about law enforcement algorithms is detrimental to due process for the same reasons that defendants’ inability to access that information about nonalgorithmic sources is detrimental. For example, courts have held that an eyewitness’s inconsistent descriptions of the perpetrator must be disclosed under Brady.¹⁵⁷ Similarly, algorithms used in criminal proceedings produce inconsistent results, even when conducted by the same program.¹⁵⁸ Courts have also required disclosure of information that links someone other than the defendant to the crime, as when a witness identifies different suspects.¹⁵⁹ Like in Willie Lynch’s case, facial recognition programs can also produce multiple suspects other than the defendant.¹⁶⁰

Courts also emphasize that the prosecution must disclose any information relating to potential witness bias pursuant to Brady. This includes whether the witness received any benefits (such as monetary or sentence-reduction benefits)¹⁶¹ or whether the witness has shown animosity towards the defendant’s race or other characteristics.¹⁶² Likewise, the proprietors who contract algorithmic systems to law enforcement agencies have financial incentives to meet law enforcement expectations that the tools they pay for will help maximize the number of successful prosecutions. Thus, proprietor incentives to design overbroad tools (or to ignore errors that create overbreadth)¹⁶³ pose the same risk to algorithms’ credibility that human witnesses’ incentives pose. Law enforcement algorithms have also been shown to produce racially biased and gendered results.¹⁶⁴ It is important for defendants to understand how eyewitness identifications were made because unconscious racial and gender bias may have influenced eyewitnesses’ recollections, and police officers’ own racial, gender, and confirmation biases may have led them to overly rely on biased eyewitnesses’ leads. The same is true for law enforcement algorithms.

These parallels show that algorithmic information has the same underlying flaws as human witnesses or statements that raise due process concerns and must likewise be subject to Brady. As this Section has shown, analogizing algorithmic systems to human witnesses—particularly in the Brady context—is an apt conceptual framework. Indeed, practitioners and scholars have compared algorithmic systems to their human counterparts in a variety of contexts. One scholar, likening algorithmic systems’ outputs to “machine testimony,” argues that defendants should be able to cross-examine machine witnesses under the Confrontation Clause.¹⁶⁵

If courts allow prosecutors to invoke trade secret protections for law enforcement algorithms in criminal proceedings, the constitutional safeguards promised by Brady will be undermined. This Brady carveout could incentivize law enforcement and prosecutors to increasingly exploit algorithmic tools to circumvent the duty to investigate and disclose favorable material. Allowing such leeway will not only diminish defendants’ due process protections but also cause expressive harm by signaling that the innovation incentives of trade secret protections outweigh defendants’ constitutional rights.¹⁶⁶ The choice to protect a proprietor’s intellectual property right at the expense of a defendant’s due process right conveys to defendants and the broader public that their government values commercial interests over the accused’s interest in a full and fair trial.¹⁶⁷ That message of inferiority, in addition to inflicting psychological trauma on the specific defendant,¹⁶⁸ in and of itself creates expressive harm by indicating disregard for defendants and by forcing them into a disvalued relationship with their government.¹⁶⁹

B. Additional Forms of Resistance to Brady Disclosure of Algorithmic Systems

Despite the similarities between human and algorithmic witnesses, the latter have proven particularly resistant to Brady disclosure. In addition to the trade secret privilege discussed above, resistance generally comes in two other forms: (1) that the algorithm is not “in possession” of the prosecutor and (2) that the algorithm is not “material.”¹⁷⁰ While resistance through trade secrets is the primary focus of this Note, these additional two challenges are worth discussing here because they are often raised in conjunction with or as alternatives to the trade secret privilege.¹⁷¹ As a result, courts sometimes are not clear whether or to what extent their rulings are based on trade secret grounds or on more traditional Brady grounds.¹⁷² This issue is exacerbated by the relative newness of the trade secret privilege.¹⁷³

1. Prosecutor’s Knowledge or Possession

Prosecutors may argue that an algorithm is not Brady material because it is not in the prosecutor’s “know[ledge]” or “possession,”¹⁷⁴ but rather possessed by a third-party proprietor.¹⁷⁵ However, this element of Brady is construed permissively—constructive knowledge or possession is sufficient.¹⁷⁶ Information is constructively within the prosecutor’s knowledge or possession if the “prosecution team,” which includes “others acting on the government’s behalf,”¹⁷⁷ knows or possesses it.¹⁷⁸ In other words, even if the prosecutor does not actually know or possess the exculpatory information, knowledge or possession is imputed to the prosecutor if any entity assisting them does know or possess it.¹⁷⁹

Courts have imputed information known or possessed by law enforcement witnesses and investigators,¹⁸⁰ crime labs,¹⁸¹ crime lab technicians,¹⁸² nurse examiners,¹⁸³ and expert witnesses¹⁸⁴ to prosecutors. While the law is quite unsettled as to the scope of the prosecution team,¹⁸⁵ courts largely agree that whether a witness or source “constitutes a state actor for purposes of Brady” requires an inquiry into “what the person did, not who the person is.”¹⁸⁶ Thus, even if the entity is privately owned or employed, the function served by the entity remains the primary focus.¹⁸⁷

For example, in Bracamontes v. Superior Court, a California state court held that two private forensic labs were part of the prosecution team for Brady purposes because the lab had “assisted in the government’s investigation” by conducting DNA testing “in an effort to identify or exclude suspects.”¹⁸⁸ The court emphasized that the lab received monetary payment from the government for its services.¹⁸⁹ Further, the court expressly disagreed with the prosecutor’s attempt to distinguish between private and government-run crime labs. In “both cases,” the work was “conducted on behalf of the government.” The private labs thus bore “the same relationship to the prosecution” as government labs, making it “reasonable to impute the private party’s knowledge to the prosecution.”¹⁹⁰ Similarly, in State v. Farris, the West Virginia Supreme Court held that a private, out-of-state forensic psychologist was part of the prosecution team because the psychologist had conducted her examinations “at the request” of the prosecutor’s investigation team.¹⁹¹ Thus, the knowledge she obtained was imputed to the prosecutors.¹⁹²

Algorithmic systems should likewise be evaluated based on what they do, not who their proprietors are.¹⁹³ If the algorithmic system directly assists in the investigation of a specific case, it is, like its human counterparts, “acting on the government’s behalf.” For example, an algorithmic system that examines DNA evidence “in an effort to identify or exclude suspects”¹⁹⁴ is part of the prosecution team even if the system is privately owned, just like the lab in Bracamontes.¹⁹⁵ A predictive system used to determine the scope of a criminal conspiracy that examines a party “at the request” of investigators and provides an “expert opinion,” as the forensic psychologist did in Farris, is part of the prosecution team.¹⁹⁶ A facial recognition system that searches and identifies a primary suspect is part of the prosecution team because it not only assists but “effectively commence[s] the prosecution of th[e] case.”¹⁹⁷ On the other hand, systems that are not directly “involved in the investigation of the case”¹⁹⁸ or are used merely at trial for “perceptual content,”¹⁹⁹ as with content used to prove a physical fact rather than the truth of the substance contained within,²⁰⁰ may be less likely to constitute part of the prosecution team. But law enforcement algorithms deployed to assist in a specific criminal investigation should largely be considered as “acting on the government’s behalf.” Ultimately, a “case-by-case analysis” is required, algorithmic source or not.²⁰¹

Like the forensic psychologist in Farris and the crime labs in Bracamontes, both of which were private entities, that an algorithmic system is owned by a private company should not obviate the necessary inquiry into the algorithm’s role in the investigation. This extension makes sense for two related reasons. First, the underlying question of the “knowledge or possession” requirement is “whether the [g]overnment should be held responsible for the actions” of the entity.²⁰² When the government has affirmatively “instructed” or “contracted” the entity to conduct an investigative task, it is reasonable that the government be held responsible, at least as to the scope of its request.²⁰³ Second, a permissive “knowledge or possession” requirement ensures that prosecutors do not have a perverse incentive to shield themselves from Brady material.²⁰⁴ Otherwise, Brady protections “might be nullified simply by keeping the prosecutor ignorant of information adverse to the government’s case.”²⁰⁵ Likewise, Brady protections would be rendered toothless if prosecutors could avoid Brady obligations by outsourcing investigative tasks to privately owned algorithms. Indeed, many scholars have lamented this as a crucial problem of privatizing state functions—a problem that is growing.²⁰⁶ Opaque privatization risks “corporate capture” and the unaccountable exercise of public power.²⁰⁷ Thus, when the government decides to contract with the proprietor of a law enforcement algorithm for use in a public, investigatory function, the algorithm should be considered within the knowledge or possession of the prosecutor.

2. Material Either to Guilt or to Punishment

The materiality requirement—or its manipulation—may pose another hurdle to defendants’ access to algorithmic information under Brady. Prosecutors are only obliged to disclose evidence “material either to guilt or to punishment” to the defendant.²⁰⁸ Materiality thus serves two purposes. Before trial, it is the threshold that instructs prosecutors on whether evidence must be disclosed (material) or need not be disclosed (not material).²⁰⁹ After trial, it imposes on the defendant a requirement to prove harm, that is, to show that there was a reasonable probability that the withheld evidence would have changed the outcome of the trial had it been disclosed.²¹⁰ Thus, “materiality” may be defined differently before and after trial, which may affect the defendant’s ability to access algorithmic information under Brady. While the post-trial definition is firmly established, the pretrial standard is not.

After trial, the standard for materiality imposes a prejudice requirement—the disclosure must have created a “reasonable probability” of a different verdict.²¹¹ Appellate judges should assess the trial record as a whole,²¹² including trial transcripts, to decide whether the disclosure would have changed the outcome.²¹³ However, appellate courts are hesitant to provide the extraordinary post-trial remedy of overturning convictions based on a Brady challenge.²¹⁴

The pretrial standard, in contrast to the established post-trial standard, is contested.²¹⁵ The Supreme Court has suggested in dicta that the pretrial standard is the same as the post-trial one.²¹⁶ Prosecutors, erroneously citing that dicta as though it were binding precedent, may assert that they do not have a pretrial or at-trial duty to disclose favorable evidence unless they determine that there is a “reasonable probability” the evidence would change the outcome of the trial.²¹⁷ However, this lack of differentiation between pretrial and post-trial standards has been widely critiqued.²¹⁸ Scholars lament that such a pretrial standard is “literally impossible” to implement because “[f]rom the pretrial perspective, it is absurd to ask a prosecutor to determine anything about the outcome of a trial that has not yet occurred.”²¹⁹ Drawing on the many unknowns of the future trial, prosecutors can “easily craft” some argument that disclosure of favorable information is unlikely to influence the trial outcome.²²⁰ This is particularly true with opaque and complex algorithms. Given the unworkability of the standard, many courts have waived the materiality threshold, requiring the government to disclose all favorable information in its possession.²²¹ This is the prudent approach.²²² Unfortunately, some courts continue to apply the prejudice standard before trial, which allows prosecutors to “hid[e] behind the cloak of materiality” to shirk their Brady disclosure obligations at trial.²²³ It would be quite difficult, then, for a defendant to overcome such a high, prosecutor-defined pretrial standard to obtain algorithmic evidence in time for trial. However, given the broad criticism of a pretrial prejudice requirement and the courts’ warning that “arguable cases” should err on the side of disclosure,²²⁴ algorithmic information should rarely be excluded before or at trial on the basis of a prosecutor-crafted assessment of materiality.²²⁵

III. The Missing Algorithm Instruction: An At-Trial Safeguard

As shown in Part II, law enforcement algorithms are akin to human witnesses in that they perform similar functions in criminal investigations and proceedings and raise similar issues related to credibility, reliability, and bias. This analogy to human witnesses is not entirely novel; it has been raised by practitioners and scholars²²⁶ and was acknowledged as “creative” by at least one court.²²⁷ This Part extends the analogy to justify a possible solution for situations in which the prosecutor or proprietor invokes a trade secret privilege despite a defendant’s request for disclosure of the algorithmic information.

Extending the analogy of algorithm as human witness, this Note proposes that courts adapt the “missing witness” evidentiary rule²²⁸ to “missing algorithms.” This adapted rule would deploy a jury instruction for missing algorithmic evidence to remedy conflicts between Brady’s constitutional mandate and trade secrecy. This remedy provides a due process safeguard at trial without violating trade secrecy and without necessitating a solution as extraordinary as excluding algorithmic evidence altogether²²⁹ or overturning²³⁰ a conviction on appeal.

A. The Missing Witness Instruction

The “missing witness” rule refers to the permissible inference a jury may draw from the absence of a potential witness who might have knowledgeable facts at issue in the case.²³¹ Take, for example, a defendant who is on trial for a robbery of a store. The prosecutor calls one eyewitness, the store manager, to testify before the jury. However, there were two eyewitnesses on the scene that saw the robbery occur—the store owner also witnessed the incident. The store owner, hostile to the robber, would be expected to testify favorably for the prosecution. But the prosecutor declines to call the store owner to the stand. The missing witness rule would allow the jury to infer that the prosecution refused to call the store owner because the owner’s testimony would have been unfavorable (perhaps, for example, the store owner’s description of the robber was inconsistent with the store manager’s).²³²

The missing witness rule can be traced to the Supreme Court case Graves v. United States,²³³ and both the defense and prosecution are allowed to invoke it in criminal trials.²³⁴ There are two key prerequisites for a missing witness instruction: (1) the witness must have been “peculiarly” available to the opposing party (the witness is available to the prosecution but not reasonably accessible to the defense through other channels), and (2) the witness would have elucidated “noncumulative” information (the information is not already in the defense’s possession).²³⁵ One of the rationales²³⁶ underlying this rule is spoliation, the suppression of evidence by a party.²³⁷ The rule is intended to deter concealment of evidence and to encourage disclosure of all relevant evidence.²³⁸ The reasoning is that when a party fails to produce a witness that knows facts about the case and the witness was “peculiarly available” to that party but not the opposing party, the jury could “naturally” draw an adverse inference from that failure.²³⁹ The instruction is permissive, not mandatory—juries are permitted to draw the inference but are not required to do so. If allowed by the court, the inference is typically included in the closing argument to the jury and addressed by a jury instruction.

B. The Missing Algorithm Instruction

Similarly, a “missing algorithm” jury instruction would only be granted when two prerequisites are met: (1) the algorithmic system is peculiarly available to the prosecution, and (2) the algorithmic system would elucidate noncumulative information.²⁴⁰ In addition, because the instruction would only be a remedy for Brady material, the instruction would only be available for algorithmic information that is “favorable” under Brady.²⁴¹

The missing algorithm instruction makes logical sense when trade secrecy is invoked to exclude an algorithm. In a criminal trial, the prosecution must prove each element of its case against the defendant beyond a reasonable doubt.²⁴² When law enforcement substantially relies on an algorithm in its investigation or when the prosecution introduces an algorithm’s output at trial and examining the algorithm may reveal bias or unreliability or lack of credibility, Brady requires that the defendant be given access to that impeachment material to help build their defense and persuade the jury that reasonable doubt exists.²⁴³ When Brady material is not disclosed, “irrespective of the good faith or bad faith of the prosecution,” a Brady “suppression” occurs.²⁴⁴ And “justice suffers” when such a suppression occurs—even when a third-party proprietor has commercial interests at stake.²⁴⁵ Thus, as with missing witness instructions, a jury should be permitted to draw a reasonable adverse inference from that suppression.

The missing algorithm instruction is also reasonable because government entities wield significant market power. As the primary consumers of law enforcement technologies, government entities can choose which proprietors to work with and how to craft their contractual relationships to ensure greater transparency.²⁴⁶ For example, a government could condition its purchase of an algorithmic system on access to all documentation relating to the reliability of the system. The government could also secure limited permission to disclose any Brady information contained therein to defendants under a protective order.²⁴⁷ In fact, some governmental entities already appear to be considering adjusting their contractual requirements to enhance algorithmic transparency.²⁴⁸ The availability of a missing algorithm instruction would incentivize more governmental entities to do so.

Allowing the jury to draw an adverse inference from Brady suppression when the government raises a valid trade secret privilege may appear harsh, particularly since the trade secret holder is a private proprietor who is not a party in the criminal case. For example, the private proprietor might resist disclosure even when the prosecutor wishes to disclose. However, as discussed in Part II, it is reasonable—and arguably necessary in many cases—to treat the algorithmic system as acting on behalf of the government.²⁴⁹ A prosecutor, in their unique role as a representative of the government,²⁵⁰ cannot instruct a crime lab to help in its investigation or at trial, then shield the crime lab from adversarial scrutiny.²⁵¹ Likewise, the prosecutor cannot have it both ways with algorithmic systems. The government should be thinking about its Brady duties when it uses an algorithmic system to assist directly in an investigation or trial, and it should only work with proprietors that will allow the government to disclose the information it is constitutionally obliged to share with the defense. If the government imprudently contracts with a proprietor that refuses to allow the required disclosures, the missing algorithm inference can remedy that due process failure at trial.

Further, any harshness is mitigated by the prosecutor and proprietor’s viable alternative—disclosing the algorithm under a protective order. Protective orders are issued routinely for trade secrets in the civil context.²⁵² These orders even enable disclosure of a party’s trade secret to its direct competitor.²⁵³ Such routinely accepted safeguards in the civil context should likewise be sufficient to protect trade secrets in criminal trials.²⁵⁴ In light of the availability of protective orders, one scholar has suggested getting rid of the trade secret privilege in the criminal context altogether.²⁵⁵

While removing the privilege from the criminal context altogether would promote transparency and accountability, trends suggest the use of trade secrets to prevent Brady disclosures is on the rise.²⁵⁶ Proprietors are aggressively and increasingly raising the privilege or asking governments to sign nondisclosure agreements at the outset.²⁵⁷ And courts appear to be increasingly sympathetic to trade secret arguments at each stage of a criminal case.²⁵⁸ In light of this trend, a missing algorithm instruction would provide a much-needed due process backstop. While the remedy does not provide the immediate transparency that full disclosure in all cases would, it provides longer-term incentives for the government to choose transparent practices and tools. This, in turn, would incentivize proprietors to develop transparent solutions.²⁵⁹

Outside of protective orders, the remedies currently available to defendants are either to request complete exclusion of the algorithmic evidence at trial or dismissal of the charges²⁶⁰ or to file a post-trial appeal seeking to overturn the conviction altogether. Neither is a satisfactory solution. The former risks providing the defendant with a windfall and punishing the prosecutor for raising a privilege.²⁶¹ Further, the former would rarely provide defendants relief given how difficult it is for defendants to exclude evidence at trial successfully.²⁶² Nor would the former provide any relief for defendants deprived of Brady evidence not introduced at trial (like a facial recognition tool that provided many other possible suspects). The latter may force the defendant to spend years in prison while the appeal is pending²⁶³ and requires the appellate court to conduct a difficult hindsight review of whether the admission of the evidence was prejudicial.²⁶⁴

The missing algorithm instruction provides a more balanced alternative to those “extreme” solutions.²⁶⁵ It permits the jury, rather than requiring the judge, to balance the defendants’ access to algorithms and proprietors’ trade secrecy concerns. Allowing the jury to draw an inference in favor of the defendant is not a punitive measure against the prosecutor;²⁶⁶ it is a due process safeguard, one that is appropriate given the prosecutor’s ability to allow access under a protective order as an alternative.

A missing algorithm instruction might take the following form:

In order for the defendant to receive a fair trial, the government is required to inform the defense of any information known to the government that casts doubt on the credibility of the government’s own evidence. In this case, the government failed to turn over promptly information favorable to the defense, namely [algorithmic material], of which the defense learned only on [date], when [means of disclosure]. The government has declined to disclose this information because of trade secret (intellectual property) concerns. The government had the option to disclose the information under a protective order. Although this denial of access does not necessarily bear on the guilt or innocence of the defendant, you may, if you think it appropriate in light of all the evidence, take into account the possible harm to the defense caused by this denial when evaluating whether the government has proven the defendant’s guilt beyond a reasonable doubt.²⁶⁷

This instruction clearly conveys the government’s disclosure duty—and its breach of that duty—to the jury. But it also conveys the reason for that breach and asks the jury to consider any inferences in consideration of that fact. Additionally, the jury is instructed to consider “all the evidence.” That evidence would include any validation studies for the algorithm the prosecutor may have produced at trial. Whether the jury would deem the validation studies sufficient would depend on whether the studies were independently conducted, the studies’ methodology, and other indicia of their credibility.²⁶⁸ The missing algorithm instruction would therefore encourage prosecutors to submit those studies at the outset and reward governmental entities that contract with transparent and credible proprietors. Further, “all the evidence” means that an adverse inference would not be drawn when the prosecution has presented a strong case bolstered by other witnesses and evidence. In contrast, an adverse inference is more likely to be drawn when the algorithm provides the key or sole piece of evidence.²⁶⁹

The instruction would be requested by the defense in a motion before or at trial. The timing would largely depend on when the suppression came to light. If requested at trial, the defense would make its request outside of the jury’s presence. Ultimately, the decision to grant this remedy would be within the trial court’s discretion. The trial court could also give more specific instructions and limit the defendant’s closing argument as to the scope of the adverse inference that can be drawn.

Admittedly, a missing algorithm instruction is far from a cure-all. One significant situation in which this remedy would not be curative is when prosecutors do not even disclose the use of law enforcement algorithms.²⁷⁰ Unfortunately, this has happened in many cases in which a novel algorithm aided the investigation.²⁷¹ As a result, most defendants do not know that an algorithm was involved in their investigation unless that information is serendipitously discovered.²⁷² For example, Willie Lynch discovered that facial recognition technology was used to identify him only eight days before his trial.²⁷³ Many defendants never find out even after they are convicted.²⁷⁴ One striking example: in Pinella County, Florida, law enforcement used facial recognition technology for fifteen years without ever providing the public defender’s office any indication of its use in Brady disclosures.²⁷⁵ Fortunately, organizations are becoming increasingly vigilant about law enforcement uses of algorithmic tools.²⁷⁶ And the availability of a missing-algorithm-instruction remedy may encourage more defense attorneys in pretrial discovery to learn whether an algorithm was used against their client.

Another shortcoming of this remedy is that defendants will still have to satisfy the Brady hurdles of “knowledge or possession” and “materiality.” But as explained in Part II, the human-witness analogy—which highlights the common need for due process safeguards between algorithmic tools and human witnesses—helps show why law enforcement algorithms should in most cases clear these two hurdles. Defendants who can overcome these Brady prerequisites will be able to request the jury-instruction remedy at trial instead of having to wait until an appeal. And courts are more likely to grant this less extreme remedy than to overturn a conviction entirely.²⁷⁷

Critics of this remedy may raise two additional concerns. First, critics may argue that the defense will be able to “create” reasonable doubt among jurors by overinflating how favorable or material the algorithmic information is.²⁷⁸ However, “[d]isclosure law cannot be predicated on the assumption that juries are [so] irrational” that a missing algorithm instruction would induce them to arbitrarily or capriciously discount the prosecution’s case.²⁷⁹ And the risk of creating reasonable doubt can be mitigated by a more tailored jury instruction.²⁸⁰

Second, critics may argue that the traditional missing witness rule is disfavored in some jurisdictions and should be limited rather than extended. But courts’ concerns over this rule usually involve adverse inferences drawn against the defendant—not the prosecutor.²⁸¹ Those concerns make sense given that defendants do not bear the burden of proof at trial and are not required to produce any evidence or any witnesses. Allowing the jury to draw an adverse inference against the defendant would risk shifting the burden of proof to the defendant. But law enforcement algorithms serve the prosecution team. The prosecutor bears the burden of convincing a jury that there is no reasonable doubt of guilt, and failing to produce the witness whose testimony was relied upon in the investigation or at trial may very well raise reasonable doubt. Therefore, the missing algorithm instruction would merely notify the jury that it should decide, in light of the other evidence presented, whether or not the prosecution team’s decision to disallow access and prevent the defendant from testing the algorithm’s credibility is significant enough to raise doubt.

Conclusion

As privately developed law enforcement algorithms proliferate, due process protections must keep pace. While algorithmic systems may be a cost-effective solution for efficient investigations and prosecutions, “[s]ociety wins not only when the guilty are convicted but when criminal trials are fair.”²⁸² Yet trade secret evidentiary privileges are increasingly shielding algorithms from scrutiny. To ensure the continued fairness of the criminal justice system, a new due process safeguard is necessary. The missing algorithm instruction, an at-trial solution, may provide just that.

* J.D., May 2021, University of Michigan Law School. Thank you to Professor Sherman Clark, Nate Wessler, and the Michigan Law Review Notes Office for their feedback and contributions. I am also grateful to Steve Kane and Professors Vivek Sankaran and Bridgette Carr for their mentorship. Finally, I thank my family—including my friends—for their unconditional support.