Legal Ethics of AI Snake Oil: Navigating the Hype, Harm, and Hope of Legal AI

Drew Simshaw* June, 2026

AI Snake Oil. By Arvind Narayanan and Sayash Kapoor. Princeton: Princeton University Press. 2024. Pp. x, 348. $24.95.

Introduction

Advancements in artificial intelligence (AI) have taken the world by storm. AI’s impact on legal services has been no less significant for licensed legal professionals, courts, and consumers. But ethical issues abound concerning competent use of technology, candor to courts, preservation of client confidentiality, and proper supervision of AI vendors, to name just a few.¹ A robust literature has developed addressing the impact of technological developments—including AI—on the access-to-justice gap,² which has sparked renewed discourse surrounding both the substance and process of potential reforms to the regulation of legal services.³

Given that AI comes with high expectations, enormous potential, and significant risks, the landscape can be daunting for consumers, licensed legal professionals, and regulators questioning how they can harness AI’s potential while avoiding its pitfalls. While these important questions have dominated the discourse, there is more to consider under the surface. Institutions within legal services are broken in a way that AI alone cannot fix, and emerging exploitation harms from AI are largely being ignored.

AI Snake Oil provides an invaluable resource on all these fronts. Notably, it is not written by, or even for, lawyers. It is written by computer scientists for a general audience. The authors are two of TIME’s 100 Most Influential People in AI.⁴ Arvind Narayanan⁵ is a professor of computer science at Princeton University and director of its Center for Information Technology Policy, and Sayash Kapoor⁶ is a PhD candidate in computer science at Princeton and former software engineer at Facebook. The book’s title—AI Snake Oil—refers to “AI that does not and cannot work” (p. 28). While much of the book focuses on identifying and responding to AI snake oil, this is just one of its many valuable contributions to a growing interdisciplinary literature on AI. As computer scientists, the authors credibly and persuasively push back on AI alarmism from other scientists, exposing it as rooted in bias and, in some cases, designed to add an aura of grandeur to those scientists’ research (p. 153). And despite the public’s broader focus on sensationalist headlines like impending existential threats from AI, the authors stress that “the biggest risks to humanity will arise from people misusing AI, not from AI going rogue” (p. 174). Their approach, therefore, is not purely technical. The book also surfaces helpful economic, social, and political dynamics currently undervalued in society. As this Review will demonstrate, these considerations are also undervalued in legal services and the literature on the legal ethics of AI, and they are an indispensable part of framing and achieving worthwhile goals for the future of legal services.

Part I of this Review analyzes the authors’ “look behind the screen” of AI within the context of emerging legal AI and the existing scholarship on legal ethics and AI. Part II examines the book’s even more important contribution—its nontechnical framework for understanding not just the supply of AI snake oil, but the demand for it. The authors argue that the adoption of AI snake oil exposes broken institutions, ones that cannot be fixed by AI. Applying this framework to legal services forces a necessary “look in the mirror” to bring attention to flaws in the landscape that lead to desperate legal service providers and consumers looking for a “quick fix” to underlying problems that are often marginalized or ignored. Finally, Part III takes “a look beyond” the dominant perspectives and discourse to address AI’s exploitative impacts and extends the authors’ concern for AI’s underlying labor exploitation to the underappreciated environmental impacts of AI. It ultimately argues that lawyers should be involved in the development of best practices for more sustainable AI and that some of those practices should rise to the level of professional obligations for lawyers.

I. A Look Behind the Screen—Legal AI’s Supply and the Scope of Its Power

Although AI Snake Oil does not focus on AI in legal services, the authors do recognize legal technology as a mature industry. In this light, it is increasingly accepted that legal practice requires some level of technological competence.⁷ In 2014, the American Bar Association (ABA) issued Formal Opinion 512, which says lawyers can usually satisfy this requirement through self-study or consultation with others with greater knowledge of technology.⁸ AI Snake Oil’s broader analysis can serve as a valuable resource for these competence efforts. This Part takes a “look behind the screen” to help understand how various forms of AI work and how understanding these systems adds important perspectives to the growing literature on AI and legal ethics.

Broadly speaking, the authors do not get bogged down in trying to find the perfect definition of AI. They ultimately propose that a tool is more likely to qualify as AI if its tasks would require creative effort or training for a human to perform, if its behavior indirectly emerged through “learning” from examples or similar processes, or if its decisionmaking is more or less autonomous and possesses some degree of adaptability to the environment (pp. 12–13). As this Part will illustrate, many emerging legal technologies fall within one or more of these categories. The authors distinguish three different types of AI—predictive AI, generative AI, and content moderation AI—identifying key differences among them that are likely not fully appreciated by most legal AI consumers. But these distinctions are important, as different forms of AI do different things in different ways, with varying levels of reliability and unique risks. Understanding these distinctions can play a helpful role in developing the baseline technological knowledge necessary for technological competence.

The first type of AI—predictive AI—is of great interest to lawyers because prediction is a large part of what lawyers do.⁹ From a technical standpoint, the authors explain that predictive AI is rooted in algorithms, or a “set of steps or rules used to make a decision” (p. 38). Sometimes those rules are developed manually and acted on automatically, like automatically sending COVID-19 stimulus checks to certain citizens (pp. 38–39). Other times, though, the algorithms develop the rules automatically from patterns in past data, like Netflix recommendations based on viewing history (p. 39). Then, a “model” can be created from “training data” to both develop and apply a rule through statistical techniques that have become known as “machine learning,” which is often unintelligible to humans (p. 39). Machine learning “uses past data to learn underlying patterns and make predictions about future events, often in ways that can adapt and change over time. There are no fixed rules about how the future will play out given past events,” and instead, “these rules are determined based on how the system behaved in the past” (p. 65). Overall, Narayanan and Kapoor define predictive AI as “models used for decision-making based on predictions about the future, such as who will do well at a job or who will pay back a loan” (p. 40).

Hype surrounding predictive legal AI predated the emergence of generative AI.¹⁰ But compared to generative AI,¹¹ predictive AI has “largely flown under the radar when it comes to public interrogation” (p. 37). The authors state bluntly, “We think predictive AI falls far short of the claims made by its developers” (p. 43), which include “promise[s] of accuracy, fairness, and efficiency” but with little accountability surrounding those promises when things go wrong (p. 261). Still, we see predictive AI deployed in many areas of society: calculating FICO scores, deploying personalized ads, automating hiring, and, most notably for lawyers, relying on criminal risk prediction, despite little published evidence of effectiveness in these areas (pp. 11, 66–67). Criminal risk prediction—or the use of risk factors to calculate the probability that an individual from a certain population will commit a crime¹²—is highly criticized, but also commonly practiced.¹³ The authors emphasize that predictive AI in the criminal justice system “highlights an assumption built into much of predictive AI: people with similar characteristics will behave similarly in the future” (p. 41), an assumption that has long been criticized as inaccurate and inherently racially biased.¹⁴ Looking more broadly at predictive AI in legal services, providers should take note of the shortcomings of predictive AI discussed in the book when considering adopting these increasingly common AI-driven systems to decide which clients have a winnable case, which argument is most likely to succeed, and even which lawyers to hire in a firm.¹⁵

Particularly relevant in legal services is the authors’ observation that predictive AI is especially faulty when it tries to reuse existing data for new purposes. Narayanan and Kapoor note that “even if AI can make accurate predictions based on past data, we can’t know how good the resulting decisions will be before AI is deployed on a new dataset or in a new setting” (p. 45). In the context of legal services, what is good for one client or self-represented litigant might not be good for another, so legal AI tools designed for one setting might not translate well to others.¹⁶ Further, Narayanan and Kapoor explain that when “decision subjects come from a population with different characteristics than those in the training data, the model’s decisions are likely to be wrong” (p. 51), a concern that has been echoed surrounding the effectiveness of legal document creation tools that were designed based on data for individuals with different needs.¹⁷

Predictive AI can also fail when it simply gets details about subjects wrong because there is only so much data that we can, or want to, collect about a person¹⁸ or because there are limitations as to what data may be available (p. 11). The authors acknowledge that access to more data could change this (p. 97). But the risk of bad predictions due to such limitations is even higher in legal services because lawyers often intentionally avoid “datafying”—or capturing in electronic form¹⁹—sensitive information about clients, inherently limiting the ability of data-dependent models to make predictions about specific clients’ cases.²⁰ Even when new data is desirable, collecting it is expensive and time-consuming, which undercuts AI’s cost cutting and efficiency benefits (p. 45). Moreover, the process of getting that helpful data might require invasive methods (p. 78).

As a result, predictive AI also often reflects the bias of human decisionmaking and the underlying data it uses, which is itself flawed. Since training data is created using human discretion, “decisions made using predictive AI may still be very human” (p. 42). In addition, “[t]he cost of flawed AI is not borne equally by all,” and “[t]he use of predictive AI disproportionately harms groups that have been systematically excluded and disadvantaged in the past” (p. 53). In legal services, automating the status quo (which can reinforce historical barriers to accessing justice) risks stunting the development of common law and progress toward a fairer system. As Dana Remus and Frank Levy argued in a pioneering interdisciplinary article in 2017, “reducing legal advising to legal prediction could threaten to impede the law’s development,” and while “[p]redictability and stability are of course critical rule-of-law values, . . . so too is democratic participation in lawmaking.”²¹ But even though using AI can risk reinforcing bias, it is also possible that AI could detect human bias where it might otherwise go unnoticed.²²

A second form of AI is generative AI, which burst onto the scene in 2022 with the public release of ChatGPT.²³ It was met with a mix of excitement and trepidation in the world of legal services. For example, on one hand, there was, and still is, hope that generative AI can increase efficiency, lower prices, and help self-represented litigants.²⁴ On the other hand, some high-profile misuses have made headlines²⁵ and led to knee-jerk reactions from some courts.²⁶

Generative AI is different from predictive AI in both its science and impact. Whereas Narayanan and Kapoor are very critical of predictive AI, they are more optimistic, and even excited, about generative AI (p. 30), expressing that they themselves are enthusiastic users (p. 99). Unlike some other forms of AI, generative AI is unique because it is available to everyone (p. 147). Progress has been gradual, “[b]ut the recent wave of generative-AI-based chatbots was the first time this technology has been useful to a large number of people” (p. 166; emphasis omitted). And of particular interest to lawyers, the authors observe that “[e]arly studies show the potential of generative AI for assisting . . . many . . . professionals” (p. 99), noting their belief that most knowledge workers can benefit from it (p. 146). But generative AI, too, has its limitations, and understanding those limitations is critical for legal service providers and consumers.

From a technical standpoint, generative AI is in some ways more complicated than predictive AI (p. 30). Generative AI is often described as predicting a response that the user wants,²⁷ but it is important to recognize the ways in which it functions differently from predictive AI. Definitionally speaking, “[g]enerative AI refers to AI technology that is capable of generating text, images, or other media” (p. 99). With regard to text, the authors boil generative AI’s functioning down to a key characteristic: “[A]ll modern chatbots are actually trained simply to predict the next word in a sequence of words,” and as a result, they “generate text by repeatedly producing one word at a time” (p. 132). This is possible because of an “astronomical amount of training data online” where “literally any text can be used” (p. 132). The authors marvel at the fact that, “[e]ven when we understand perfectly well how and why a chatbot works, it can remain mind-boggling that it works at all” (p. 133). Even machine learning researchers have been shocked how well ChatGPT can follow user instructions (p. 136).

But generative AI chatbots have their limits. For one, they are limited to tasks that are sufficiently similar to the tasks in their training data (p. 136). So, for example, AI became decent at playing chess because there are statistics about chess moves published online, but the same AI would be bad at a board game where such statistics or transcripts of games have not been posted online.²⁸ Applying this limitation to legal services, generative AI might be good at, for example, recreating status quo legal arguments, but not as good at creating novel ones. As Eliza Mik has noted, “[n]otwithstanding their unprecedented ability to generate text, LLMs [large language models] do not understand text,” so “[w]ithout the ability to understand the meaning of words, LLMs will remain unable to use language, to acquire knowledge and to perform complex reasoning tasks.”²⁹ This can raise serious competence and diligence questions for lawyers who heavily rely on generative AI tools, even when using services designed specifically for lawyers. That concern is likely even greater for those using free services marketed to the general public.

And like predictive AI, overreliance on generative AI could also stunt the development of the law. A similar concern has been looming over the art world. Narayanan and Kapoor describe generative AI’s essential theft of artists’ work for AI training (pp. 122–23), which not only results in potential copyright violations³⁰ but also risks artist displacement (p. 126). This displacement could mean there would be nobody to train the next generation of AI models (p. 126), potentially accelerated by a recent court decision that indicates that some courts might view AI training with legally accessed copyrighted material to constitute “fair use.”³¹ That same stunted creativity concern could extend to lawyers and the future of novel legal arguments that have historically moved the law forward.³²

Generative AI can also produce misinformation through inaccurate outputs (p. 147), colloquially labeled “hallucinations.” This term has been criticized as misleading,³³ and the authors do not use it. Instead, they describe chatbots as “bullshitters . . . [that] are trained to produce plausible text, not true statements” (p. 139). Once someone understands the nature of next-word prediction, “[t]he surprising thing is not that chatbots sometimes generate nonsense but that they answer correctly so often” (p. 140). The authors think this is “best understood as a side effect of the fact that true statements are more plausible than false ones” (p. 140). The important takeaway is that the output is not intended to be presented as true. And yet, as the authors acknowledge, chatbots have the voice of an expert while sometimes struggling to do what a child could (pp. 30, 137–38).

Inaccurate outputs have gotten the best of a growing number of lawyers. The authors discuss one such now-famous case in which a New York lawyer filed a brief with a court that contained citations to several fictitious, though realistic-looking, cases. When the court was alerted to the nonexistent cases by opposing counsel, the lawyer said that ChatGPT was the source of the citations and that ChatGPT confirmed their accuracy in a subsequent inquiry.³⁴ The authors frame this case as an example of the “problem of misinformation” created by generative AI, noting that the lawyer was presumably oblivious to disclaimers from the service about accuracy (p. 102). It was predicted at the time that similar examples would follow,³⁵ and there has indeed been a steady flow of such stories—1,330 as of April 19, 2026—with some resulting in warnings, sanctions, hefty fines, mandatory continuing legal education trainings, or suspensions.³⁶

Courts, state bar associations, and the ABA have responded. Some courts have required new disclosures about AI use,³⁷ while others have favored underscoring and enforcing current obligations concerning competence, truthfulness, and candor to the court.³⁸ Though they do not address these approaches explicitly, Narayanan and Kapoor seem to endorse the latter approach at times, emphasizing that “even in a fast-moving space like AI, most of what is needed is the enforcement of existing regulations rather than the creation of new regulations” (p. 273; emphasis omitted).

The application of existing rules is also consistent with ABA Formal Opinion 512’s analysis of these cases under the existing rules of professional conduct.³⁹ The opinion warns of inaccurate outputs, echoing some of the authors’ concerns, emphasizing, for example, that “[t]he large language models underlying [generative AI (GAI)] tools use complex algorithms to create fluent text, yet GAI tools are only as good as their data and related infrastructure.”⁴⁰ As a result, “[i]f the quality, breadth, and sources of the underlying data on which a GAI tool is trained are limited or outdated or reflect biased content, the tool might produce unreliable, incomplete, or discriminatory results.”⁴¹ It further warned that “a lawyer’s reliance on, or submission of, a GAI tool’s output—without an appropriate degree of independent verification or review of its output—could violate the duty to provide competent representation as required by Model Rule 1.1.”⁴² As Narayanan and Kapoor note, generative AI tools are beginning to incorporate “retrieval” results from authoritative sources into their generative outputs, which the authors characterize as “far from perfect, but a big improvement over the purely generative approach” (pp. 147–48). But in legal services, this still does not always yield acceptable results, a concern reflected in ABA Formal Opinion 512’s observation that generative AI tools used in legal services⁴³ combine otherwise accurate information in unexpected ways to yield false or inaccurate results.”⁴⁴

Despite AI’s potential harms, the authors emphasize that generative AI is set to become “part of our digital infrastructure” (p. 258). They encourage individuals to experiment with generative AI tools and “look[] into how your peers in your field are using them” (p. 147), though they separately acknowledge downsides to what is largely a privately-owned AI infrastructure that can lead to inequitable access for those in rural and economically disadvantaged areas (p. 259). To the extent that certain legal AI tools are increasingly seen as critical for lawyers who want to keep up,⁴⁵ unequal access to the potential benefits of these tools could prove to be a major impediment to efforts to close the access-to-justice gap.⁴⁶ The authors also stress that fixing generative AI will not fix the underlying problems that cause some institutions to turn to faulty AI in the first place,⁴⁷ a point that Part II of this Review will apply to the legal services landscape.

A third category of AI—content moderation AI, which is essentially the process of keeping unwanted content off platforms—is discussed by the authors mostly within the context of social media (ch. 6). But the principles, opportunities, and challenges in content moderation offer additional important insights for lawyers. Even though the process of developing and applying a policy concerning unwanted content is not exactly what legal service providers do, there are key parallels between content moderation efforts and the development and application of laws generally (which is exactly what legal service providers do), shedding light on the limits of AI in those processes. The development of both policies and laws is inherently political; the authors, though, focus not on “the politics of content moderation itself but rather . . . the futility of trying to automate one’s way through these political issues” (p. 183).

AI is often too literal and bad at understanding context (pp. 184, 217), something that is critically important for both content moderation and legal AI applications. Content moderation and legal services are also both plagued by cultural incompetence. With social media companies, financial motives have led to underinvestment in human content moderators outside of the United States (pp. 188–90, 217). Content moderation is often outsourced to a small number of countries separate from the primary user bases of the platforms, resulting in a lack of understanding of local cultures in content moderation policy enforcement (p. 190–91). These decisions to favor other priorities, like finances, over investment in cultural competence can also plague legal services, and the harm is magnified when technology is involved. As Sherley Cruz notes in Coding for Cultural Competence, “Without intentional consideration of end users and their needs, limits, and preferences, technology can lead to . . . barriers that will prevent access to legal services.”⁴⁸ Moreover, when the world changes, both content moderation (p. 218) and legal AI have trouble adapting. “There are many ways in which patterns learned from past data are less than perfect when applied on an ongoing basis” (p. 195). If a legal AI tool does not keep up with changes in the law, it is inadequate at best and harmful at worst.

As this Part has demonstrated, AI Snake Oil is a helpful resource in navigating the application of the Rules of Professional Conduct to legal AI applications. But the book can also be helpful in guiding reforms. Several jurisdictions have implemented regulatory sandboxes to test technology-driven legal service delivery models that might run afoul of current rules.⁴⁹ Though not referenced directly by the authors, the book highlights the importance of regulatory experimentation more generally by stressing that “[t]here isn’t one uniform way to regulate AI, and that’s not necessarily a bad thing”; the authors elaborate that “[t]hrough these different approaches, we can understand what works (and what doesn’t) and develop better principles for regulation” (p. 272). Although AI often intimidates regulators, the authors stress that “the principles of AI systems being used today are simple enough to be broadly understood” (p. 272). Moreover, they label as a myth the idea “that tech regulation is hopeless because policymakers don’t understand technology,” noting that domain expertise is not necessary because policymakers can delegate and consult with experts if necessary (p. 272). But “[u]nfortunately, there are too few of them, and the understaffing of tech experts in government is a real problem” (pp. 272–73). This is compounded by a perpetual lack of funding generally (p. 273). Expertise and funding deficiencies are both challenges for legal regulatory sandbox efforts.⁵⁰ To overcome these challenges, we may need some type of centralized coordination for legal services regulatory reform, so as not to stretch legal AI experts and sources of funding thin.⁵¹

Despite the many shortcomings of AI, the authors conclude that its “impact isn’t inevitable, nor is its trajectory predefined” (p. 258). Ultimately, there is a need to shape AI to promote the public interest (p. 258). But changing AI is not enough; institutions turning to AI must also understand why they pursue it in the first place and what the need for it reveals. The next Part encourages legal AI consumers to take such a “look in the mirror.”

II. A Look in the Mirror: Legal AI’s Demand and What It Reveals

Narayanan and Kapoor briefly address two aspects of AI in legal services that, although not the focus of their book, have, at times, dominated headlines: (1) claims that AI will replace lawyers because it “passed the bar” (p. 241) and (2) instances of lawyers submitting filings with false information generated by AI (p. 102). In a sense, there is dissonance between these narratives: One frames AI as wholly capable of practicing law, and the other frames AI as far from any such capability. While both narratives are oversimplified and misleading, they should not be ignored. Both reveal flaws in legal services institutions that are exposed under an AI spotlight. The authors’ brief responses to these headlines provide a starting point for further analysis, and the rest of this Part builds on their theories in the legal services context.

On the bar exam claim, the authors address OpenAI’s 2023 claim that its GPT-4 “exhibits human-level performance on the majority of . . . professional and academic exams” and that “GPT-4 scored in the ninetieth percentile on the bar exam” (p. 241). This claim led many to suggest that GPT-4 could do the work of lawyers. The Late Show with Stephen Colbert even responded with a sketch (albeit in jest) depicting lawyer chatbots.⁵² The authors denounce this narrative on the grounds that “a lawyer’s job is not to answer bar exam questions all day” and that “[r]eal-world utility is different from good performance on a benchmark” (p. 241). Many others responded just as critically.⁵³ When viewed as a reflection on the legal profession and its licensing requirements, the claim was just as much an indictment of the bar exam itself, even causing some legal profession commentators to look in the mirror and hope that the narrative would “finally kill the bar exam.”⁵⁴ In Narayanan and Kapoor’s words, the bar exam “notoriously overemphasize[s] subject-matter knowledge and underemphasize[s] real-world skills, which are far harder to measure using a standardized test.” (p. 241). This criticism of the exam has been widely echoed⁵⁵ and has helped lead to the development of a “NextGen” bar exam that will address some of these concerns,⁵⁶ though many problems with the bar exam and lawyer licensing more broadly remain.⁵⁷

But bigger questions persist about the fact that an increasing number of lawyers are passively doing the thing the bar exam has been criticized for incentivizing—uncritically regurgitating doctrine—sometimes leading to work product with false information produced by AI. It is easy to point fingers at the users for their passivity and misplaced trust in AI (and many have made such criticisms⁵⁸), as well as at the service itself for providing the false information. But the authors suggest there is a third actor to consider: the underlying institution, whose incentives may have drawn the user to a misguided “quick-fix” AI solution in the first place.

AI Snake Oil provides a framework for this introspection. The authors underscore that “much of the downside of AI comes down to factors outside the technology itself—like the incentives of the institutions that use AI” (p. 261). The book focuses on “broken institutions . . . desperate for a quick fix” (p. 33). From this perspective, AI is a symptom, rather than the cause, of many underlying problems. The authors argue that “[e]ven if all the AI companies that make false promises go out of business tomorrow, flawed institutions would turn to some other type of snake oil that promises a quick fix” (p. 261). Therefore, “[t]he demand for AI snake oil here isn’t primarily about AI—it’s about misguided incentives in the failing institutions that adopt them” (p. 261). The authors conclude that “[w]e can’t fix these problems by fixing AI,” and “AI snake oil [actually] does us a favor by shining a spotlight on these underlying problems” (p. 34).

In terms of the second issue facing AI users in the legal field, it is important to recognize as an initial matter that error-filled filings are a small part of the bigger picture of legal AI’s potential promise and peril. There is significant and justified optimism surrounding AI in legal services, particularly regarding the justice gap, an area with a now-robust literature spanning back over a decade.⁵⁹ But there is also risk of creating a two-tiered system of legal services.⁶⁰ Part of this fear is that large firms will thrive with carefully developed and effective AI, while other providers and self-represented litigants will be relegated to inferior off-the-shelf tools.⁶¹ AI in legal services can be inferior if it does not work at all, does not work as advertised, or is used in a way in which it was not intended to be, like using ChatGPT to research and cite legal authority. Ideally, there would be an interdisciplinary environment where all stakeholders could collaborate on innovative ways to incorporate AI into services responsibly and effectively, but such a landscape is inhibited by prohibitions on cross-industry business structures,⁶² unclear regulation of the unauthorized practice of law,⁶³ and unequal ability to leverage the necessary resources, relationships, and resilience needed for ethical use of AI.⁶⁴ Part of developing an effective legal AI environment is addressing these barriers, many of which are rooted in our institutions and their priorities and mindsets.

Another reason legal service providers may be drawn to AI stems from failures in balance and wellbeing, often manifesting in burnout and mental health challenges, causing many to see AI as a quick fix in the face of the daily challenges that come with large workloads and long hours.⁶⁵ Effective AI could improve wellbeing in theory,⁶⁶ but it could just as easily follow the path of other forms of technology, like mobile computers and communications devices, that magnified wellbeing problems by fueling round-the-clock work. Felicity Bell, Justine Rogers, and Michael Legg have noted that AI might also “inflame existing pressure points or stressors in the professional paradigms by heightening harmful elements: increasing competitive tensions in an already hyper-competitive occupation; potential[ threats to] the loss of lawyers’ monopoly on legal work (where that monopoly still exists); and [an] associated loss of prestige and sense of purpose.”⁶⁷ In this sense, AI is neither the sole cause of, nor the ultimate solution to, wellbeing failures. But it does expose them. AI also adds related ethical violations to the list of violations that those dealing with stress-related conditions like burnout and alcohol abuse are already more prone to commit.⁶⁸

The authors provide many compelling examples where “dubious AI is disproportionately adopted by institutions that are underfunded or cannot effectively perform their roles” (p. 263) or are understaffed and overburdened. These hindrances cause these organizations to “seek solutions that promise efficiency and cost cutting,” even though the dubious AI only leads to dubious results (p. 262). Their examples include AI adoption in organizational hiring (turning to bias-prone algorithms in hopes of selecting the best candidates more efficiently), journalism (turning to generative-AI-written news stories in the face of budget cuts, resulting in inaccurate information in reporting), and education (turning to faulty AI detection software that results in false positives or is easy to fool) (pp. 261–63).

Some legal services fit the desperate-from-underfunding mold. It is no secret that legal aid funding has decreased dramatically in recent decades, with the survival of the Legal Services Corporation increasingly in jeopardy.⁶⁹ Legal nonprofits, public defenders, and others have faced funding challenges for years.⁷⁰ Financial constraints have also proven to be a major—though not remotely the sole—impediment to consumers gaining access to legal services.⁷¹ The authors emphasize that “[t]hese organizations might also lack the capacity to experiment with AI and discard it if it doesn’t work out” (p. 263), a reality facing many legal services providers who lack this necessary resilience.⁷²

But finances are not always the culprit. “AI snake oil is often deployed as a way to allocate scarce resources” (p. 265), and for some attorneys, that scarce resource is time, resulting in immense pressure.⁷³ This is reflected in the Rules of Professional Conduct (RPC), which set out rules to account for the burdens lawyers face and ensure clients do not suffer as a result, including Rule 1.3’s mandates to control one’s workload⁷⁴ and to pursue matters on behalf of a client despite personal inconvenience.⁷⁵ A look at the types of lawyers who have cited AI-generated cases shows that they come from all types of locations, practice areas, and firm sizes,⁷⁶ reflecting the systemic nature of the underlying issues that cause such misguided decisions. As early anecdotes emerge that shed light on the reasons behind misguided use of generative AI in court filings, the culprits indeed include desperation, workload, and pressure.⁷⁷

In addition to financial constraints and time-related stress, profit motive can be another source of the obsession with achieving efficiency and process optimization at the expense of other considerations. As the authors observe, “[i]n all these examples, it is clear that AI isn’t the solution to the root problem that it is trying to fix. Yet, the logic of efficiency is entrenched in these institutions, and AI can seem like a silver bullet . . .” (p. 265). Relatedly, the predictive AI space often exposes a sometimes misguided “‘optimization mindset’ in which one tries to formulate a decision in computational terms in order to find the optimal solution and achieve maximum efficiency.”⁷⁸ Applied to legal services, this is not to say that the quest for efficiency and optimization is bad, but rather that AI is not a silver bullet for institutions with bigger challenges to overcome.

Lawyers should heed the authors’ advice on breaking through these tendencies:

If we discard [the optimization] mindset, a much bigger set of decision-making approaches opens up. We can aim to find strategies or policies that achieve modest efficiency gains while being simple enough to understand—both for decision-makers and decision subjects. . . . Such an approach also makes it easier to incorporate multiple objectives, some of which capture moral rather than economic goals. (p. 266)

For clients, these moral goals could include the “moral, . . . social and political factors[] that may be relevant to the client’s situation” emphasized in Model Rule 2.1⁷⁹ that are difficult to account for in AI-driven guidance. For lawyers, these other goals could and should include improved balance and wellbeing.

Failing to fix these broken institutions and misguided mindsets will reinforce a status quo where AI hype and natural, unchecked human responses continue to steer vulnerable consumers astray, magnifying the underlying problems. Part of seeing through AI hype is maintaining a basic understanding of how AI works.⁸⁰ Demystifying AI makes users “better mentally equipped to resist the tendency to defer to claims made by those who built it” (p. 105). But overcoming hype also requires AI consumers to understand themselves and what makes them susceptible to believing in AI myths. Here, too, lawyers can learn much from AI Snake Oil.

AI creators, often seeking investment to fuel growth (p. 236), generate AI hype in ways that are resistant to traditional sources of consumer protection.⁸¹ The authors reveal how AI companies use wealth in ways that make academia and the press less effective counterweights to hype (p. 32). Sometimes those institutions even contribute to the hype, including “researchers who want to publish flashy results, and journalists and public figures who make sensationalist claims to grab people’s attention” (p. 261). At the same time, academic researchers are criticized for their “cozy relationship with industry,” making academic research “of limited effectiveness as a check on industry power” (pp. 236–37).

To resist this hype, legal AI consumers must again look in the mirror to understand the cognitive biases that make someone susceptible to hype, as AI vendors “exploit cognitive biases to misinform the public” (p. 230). For example, the authors note that when we anthropomorphize AI (in other words, treat it as living), it can lead to misplaced trust (p. 231). In the short history of legal AI, examples abound of services that invoke images of lifelike “robot lawyer[s]” or services with human—usually male—names,⁸² raising justified criticism.⁸³ While anthropomorphism alone does not speak to the effectiveness of any particular service, it does show that it is no secret that consumers are drawn to services that are built on an image of humanness.⁸⁴

The authors also discuss additional cognitive biases that make people generally susceptible to hype that are also salient in the legal AI landscape. Sensationalist headlines pronouncing the unparalleled power of legal AI, without acknowledging that it may be unproven or error prone, can cause consumers and legal service providers alike to experience “priming” bias, which is “when past exposure to a concept leads to overemphasizing its importance in future decisions” (p. 256), as well as the “illusory truth effect” when “the mere repetition of inaccurate information can lead us to think it’s true” (p. 256). When such claims are not challenged, especially at first, these consumers can also experience “anchoring bias,” which “refers to the fact that individuals rely heavily on the first piece of information encountered when forming opinions or making decisions” (p. 256), and when something is corrected later, it is hard to adjust (p. 257). With these beliefs then embedded in one’s thinking, the more commonly known “confirmation bias” can come into play, exposing “our tendency to seek out information that justifies our beliefs instead of challenging them” (p. 257). The authors urge readers who work in institutions exploring AI to use this knowledge in developing more responsible selection and use of AI (p. 265), and legal AI consumers should consider themselves among these institutions.

III. A Look Beyond: Legal AI’s Underappreciated Impact, and an Ethical Response

So far, this Review has looked behind the screen to evaluate the supply of legal AI and the limits of its abilities, as well as in the mirror to understand the demand for legal AI and what institutional failures it exposes. But other impacts of AI use are less obvious. This Part examines the exploitative impacts of AI and argues that lawyers should play an active role in developing responsive best practices, some of which should become professional obligations for licensed legal service providers.

The authors’ sole exploitation focus is labor exploitation. They believe that “the labor exploitation that is at the core of the way [AI] is built and deployed today” is “[t]he most serious harm from generative AI” (p. 148). Currently, there are certain unpleasant tasks performed by humans to ensure that AI is profitable. For example, generative AI relies on “web scraping,” during which workers prevent problematic content from infiltrating chatbot training “by ensuring that . . . the bots are given examples of the kinds of things they are and aren’t allowed to say” (pp. 101, 115). This currently requires identifying and labeling undesirable content, a task performed by humans (p. 115), usually from outside of the United States, who are poorly paid for the dismal task (pp. 143–44). The authors refer to this as the “cost of improvement” that is currently seen as an indispensable component of successful commercialization of generative AI services.⁸⁵ More broadly, the authors warn that “AI will shift power away from workers and centralize it in the hands of a few companies” (p. 254), requiring a new labor movement to effect real change (p. 146). There is little visibility into these business practices (p. 145), so it is hard to appreciate the full extent of labor exploitation, but scholars and international organizations are increasingly calling for changes.⁸⁶ In her 2025 book Empire of AI: Dreams and Nightmares in Sam Altman’s OpenAI, journalist Karen Hao echoes these concerns surrounding low wages, trauma resulting from moderating disturbing content, and workers being fired for unionizing, ultimately arguing that using curated datasets that are more focused than the entire internet could minimize exposure to traumatizing content.⁸⁷

But labor is not all that is exploited. AI’s negative impact on the environment is both significant and well documented, though the authors do not explicitly link human exploitation and environmental exploitation or even address environmental exploitation beyond a passing reference. But others have.⁸⁸ Most notably, Kate Crawford extensively analyzes both forms of exploitation in her book Atlas of AI: Power, Politics, and the Planetary Costs of Artificial Intelligence.⁸⁹ This and other interdisciplinary scholarship addressing environmental and other harms from AI can provide guidance for legal service providers navigating AI’s potential benefits alongside its exploitative impacts.

To be sure, the relationship between AI and the environment is not all negative. As with legal services, there is considerable excitement for AI’s potential to advance certain environmental efforts.⁹⁰ But AI also has significant negative environmental impacts associated with its training, deployment, and use. In short, AI consumes a tremendous amount of energy and resources. Although AI Snake Oil does not address environmental impacts explicitly, its “look behind the screen” demonstrates just how much energy is needed to make AI work. With generative AI specifically, the authors emphasize the “astronomical amount of training data online” (p. 132) and generative AI’s “mind-bendingly brute-force” methods of computation that make use of it (p. 131). They emphasize that these methods are extremely “computationally expensive,” requiring trillions of arithmetic operations to generate a single token, or part of a word (p. 133).

Additional research supports these characterizations, demonstrating that AI training requires processing extremely large datasets to make ongoing adjustments for proper performance.⁹¹ Even as early as 2019, it was estimated that “training a single state-of-the-art AI model can emit as much carbon dioxide as the lifetime emissions of five cars.”⁹² In addition, the deployment of AI requires the use of cloud servers and user devices that each require their own power.⁹³ Each use of AI, in turn, also consumes energy; a ChatGPT query consumes ten times more energy than a typical Google search.⁹⁴ And it has been estimated that data centers—the computational home of AI systems with power-hungry servers, networking equipment, and cooling systems⁹⁵—“[c]onsume up to 460 billion kilowatt-hours of electricity annually—enough to power approximately 43.8 million U.S. households per year.”⁹⁶ On top of all of this, the computer chips at data centers require not only electricity, but also raw materials, critical minerals, rare elements, and nonrenewable energy sources, all leading to an increase in carbon emissions and greenhouse gases.⁹⁷ Moreover, advancements in AI come with turnover in hardware, leading to accumulation of electronic waste.⁹⁸ It is no surprise, then, that in September 2024, the UN Environment Programme declared poignantly, “AI has an environmental problem.”⁹⁹

Of course, this is not the first time lawyers’ work has intersected with environmental concerns. Lawyers impact the environment through the substance of their work in many ways,¹⁰⁰ from working for regulators, to representing polluters and environmental activists, to adjudicating environmental law disputes.¹⁰¹ And lawyers, as notoriously voracious consumers of paper, for example, have not always been the most environmentally friendly people.¹⁰² In 2016, Professor Tom Lininger proposed an expansion of Model Rule 4.4 to prohibit lawyer “conduct that entails the consumption of resources, the generation of waste, the discharge of pollution or any other degradation of the environment in a manner that is grossly disproportionate to the importance of the conduct in advancing a client’s interests or otherwise promoting the interests of justice.”¹⁰³ He also proposed expanding lawyers’ supervisory obligations to include ensuring that the conduct of other lawyers and third-party partners “minimizes the consumption of resources, the generation of waste, the discharge of pollution, or any other degradation of the environment.”¹⁰⁴ One reason such proposals have not yet been implemented is likely that lawyers have historically had other practical and ethical checks on such conduct, such as clients not wanting to pay for unnecessary printing and copying,¹⁰⁵ judges not wanting to read superfluous pages,¹⁰⁶ law offices only having so much physical storage capacity, and lawyers themselves not wanting to safeguard more physical pages than they have to.¹⁰⁷ In other words, though perhaps wasteful at times, lawyer paper usage never rose to a level labeled “unethical” because these limitations and other obligations kept their waste to some degree in check. Additionally, in some cases the waste could be attributed to compliance with other obligations like court rules requiring extensive hard-copy filings,¹⁰⁸ a practice that has received its own criticism.¹⁰⁹

The environmental harms caused by AI do not have these same limitations and checks. AI functionality has grown exponentially and almost unfathomably, not limited by printer capabilities or the size of physical storage facilities. The computing usually takes place off premises, leaving virtually no physical impact on the workplace. Lawyers’ only interactions with AI are through the screen. In fact, the things that make AI attractive actually magnify its environmental impacts through increased use and more computationally complex tasks. Although many forms of AI are expensive, clients increasingly see AI as a cost saver, leading to increased demand.¹¹⁰ There is also a compounding tension between complying with legal ethics obligations and minimizing environmental impacts. In analyzing this tension in the context of legal education, Anil Banlin has explained that “[o]ptimising AI algorithms for fairness and accuracy may require complex calculations and high computational resources, leading to increased energy consumption and environmental impact.”¹¹¹ To the extent that avoiding algorithmic bias, arguably required in jurisdictions that have adopted a version of Model Rule 8.4(g),¹¹² can be aided by more complex and nuanced programming, doing so has necessarily greater environmental impact. And in the same vein, emphasizing privacy and data protection, as required under Rule 1.6 on confidentiality,¹¹³ “may result in the duplication of data storage and processing, further exacerbating sustainability challenges.”¹¹⁴

Of course, lawyers are subject to the same laws as nonlawyers, and broader regulation of AI companies and AI users directly could curb these impacts. But there has been very little regulatory response in the United States to environmental exploitation resulting from AI.¹¹⁵ The Artificial Intelligence Environmental Impacts Act of 2024 would have directed federal agencies to study AI’s environmental impact, establish a consortium of experts to develop responsive standards, and create a voluntary reporting system for companies to disclose the environmental impacts of their systems to promote transparency and sustainability. ¹¹⁶ But the bill stalled,¹¹⁷ and is unlikely to survive in light of President Trump’s efforts to dismantle attempts to regulate AI.¹¹⁸ It would face major funding challenges even if passed.¹¹⁹ Moreover, proposals for a federal ban on state regulation of AI have been considered.¹²⁰

But even absent regulation, best practices for reducing AI’s environmental harms are emerging. For example, users can be more deliberate about which AI model they use for different tasks, choosing smaller models that use significantly less energy for smaller tasks.¹²¹ Lawyers utilizing legal AI should keep in mind that not every search or inquiry requires the most robust database or the most complex text generation. This waste-reduction consciousness would mirror calls for using virtual meetings when possible to avoid the environmental impacts of unnecessary travel.¹²² Lawyers can also favor services that use or are developing more energy-efficient algorithms and hardware architectures, which researchers are starting to explore.¹²³ Lawyers can also favor AI companies that are adopting renewable energy sources—like solar or wind—to power their data centers and AI infrastructures, thereby reducing their carbon and greenhouse gas emissions and footprint.¹²⁴ To the extent that certain sustainability efforts require action by AI companies, lawyers should pressure companies to make such changes or favor those who already have, either by direct advocacy or by voting with their consumption choices.

As AI developers and users explore these and other best practices, some should emerge as appropriate for baseline ethical obligations for the world of legal practice. Such obligations would naturally be rooted in existing ethical obligations, including competency, communication, third-party supervision, and rendering candid advice. For example, Irma Russell, John C. Dernbach, and Matt Bogoshian have argued that the duty of competence “requires every lawyer to keep abreast of current and evolving law and established knowledge concerning climate.”¹²⁵ They add that “[c]limate competence requires lawyers to have a basic understanding of how climate change is transforming, and will continue to transform, the physical world,” as well as the ways in which it “is affecting, and will likely further affect, law and law practice,” ultimately requiring lawyers “to integrate that knowledge into their law practice.”¹²⁶ Moreover, under the previously discussed technological competence obligation,¹²⁷ lawyers have a duty to stay informed about the impacts of the technologies they use, which should include the labor and environmental impacts inherent in their use. Nicolás Parra-Herrera has argued that knowledge of AI ethics should be considered as part of technological competence, arguing that the “obligation to know [about technology] means being aware of the ethical risks, including the social and environmental dimensions that GAI could trigger as it is used in legal practice.”¹²⁸ This view is consistent with ABA Formal Opinion 512’s guidance that “lawyers should become aware of the GAI tools relevant to their work so that they can make an informed decision, as a matter of professional judgment, whether to avail themselves of these tools or to conduct their work by other means.”¹²⁹ The fact that the Rules of Professional Conduct do not mention environmental considerations explicitly should not preclude this interpretation. Scholars have emphasized that technological competence requires looking beyond the Rules of Professional Conduct to other laws and best practices.¹³⁰

Clients must also be informed about the risk of external harms that can result from their lawyers’ use of AI in their cases. There are already some recognized circumstances under which a lawyer must disclose the use of AI to a client. For example, Formal Opinion 512 notes that “[w]here Model Rule 1.6 does not require disclosure and informed consent, the lawyer must separately consider whether other Model Rules, particularly Model Rule 1.4, require disclosing the use of a GAI tool in the representation.”¹³¹ Model Rule 1.4 mandates that lawyers discuss with clients the means by which client objectives are to be achieved.¹³² This should include informing clients of AI’s impacts so that they can have a say in balancing the competing interests outlined above. Clients, many of whom are increasingly concerned about the environment and the impacts of their business decisions,¹³³ might favor simpler models, even with a small risk of a less robust output, if it means lower energy consumption. Such communication would also be in line with guidance from a 2019 ABA House of Delegates Resolution addressing climate change, which, among other things, encourages lawyers “to advise their clients of the risks and opportunities that climate change provides.”¹³⁴ Even if the client does not favor sustainability in a particular instance, lawyers are not required to use the most aggressive means to achieve a client’s objectives,¹³⁵ so permitting lawyers to use an appropriate model for specific tasks, even over a client’s objection, would not undermine their obligations of competence or diligence.

Formal Opinion 512 elaborates that “[i]t is not possible to catalogue every situation in which lawyers must inform clients about their use of GAI” and that “lawyers should consider whether the specific circumstances warrant client consultation about the use of a GAI tool, including the client’s needs and expectations”¹³⁶ as well as their “interests and objectives.”¹³⁷ A client’s interests and objectives go beyond those strictly related to winning the case, with Rule 2.1 directing that, “[i]n rendering advice, a lawyer may refer not only to law but to other considerations such as moral, economic, social and political factors, that may be relevant to the client’s situation,”¹³⁸ which could very reasonably include moral considerations related to exploitative harms that can result from use of AI in that client’s case. In 2016, Lininger proposed that “environmental factors” be explicitly added to the list of considerations in Rule 2.1,¹³⁹ and the growing impact of generative AI should spur renewed consideration of that proposal.

Ideally, though, legal service consumers and providers should not have to choose between effective services and environmental harm, a tension that William Aceves has acknowledged within the context of paper waste.¹⁴⁰ Narayanan and Kapoor encourage AI adopters to favor companies with ethical practices and, if necessary, pressure vendors to change their practices (p. 149). As best practices emerge, lawyers should be obligated to engage with AI vendors accordingly. Under Rule 5.3, lawyers must supervise third party vendors to ensure that their conduct is compatible with the professional obligations of the lawyer.¹⁴¹ Formal Opinion 512 observes that existing opinions on Rule 5.3

note the importance of: reference checks and vendor credentials; understanding vendor’s security policies and protocols; familiarity with vendor’s hiring practices; using confidentiality agreements; understanding the vendor’s conflicts check system to screen for adversity among firm clients; and the availability and accessibility of a legal forum for legal relief for violations of the vendor agreement. These concepts also apply to GAI providers and tools.¹⁴²

This list is clearly nonexhaustive and represents best practices that have emerged as the risks of technology have become known.

Finally, the Preamble of the Model Rules of Professional Conduct emphasizes that lawyers are public citizens with obligations concerning justice.¹⁴³ Narayanan and Kapoor continually underscore that the stakes of harms from AI emphasize favoring the public interest instead of profits. They conclude, “If we keep going down the path of AI as almost entirely private and profit driven rather than guided by public interest, the risks are clear. But there’s still room for change” (p. 261). Lawyers can and must be a part of that change. This does not mean abandoning AI and its potential. The authors acknowledge that “[s]ome have argued that given these AI companies’ unscrupulous business practices, the only ethical course of action is to avoid using it altogether,” recognizing that “[t]hat decision is up to individuals” (p. 148). But environmental consciousness does not need to sacrifice effective services and a successful bottom line. As Lindsay Griffiths of the International Lawyers Network has noted within the context of lawyers and environmental sustainability more broadly, “Being sustainable helps to improve an organization or law firm’s reputation, attracts clients with similar goals, and helps to reduce costs.”¹⁴⁴ Moreover, at the end of the day, the authors argue that “collective action can be more fruitful than individual resistance” (p. 148). It is essential that consumers of legal AI, and members of the legal professional broadly, be part of this collective action.

Conclusion

Legal service providers stand at a crossroads as artificial intelligence continues to reshape the delivery of legal services. AI Snake Oil challenges stakeholders not only to evaluate AI tools based on their technical capabilities but also to confront the flawed institutional mindsets that drive misguided adoption in some instances. For legal service providers, this means going beyond questions of competence and efficiency to reflect critically on the social, environmental, and ethical implications of their AI adoption and use. As this Review has shown, the Model Rules of Professional Conduct already offer a framework for this engagement, but they must be enforced with renewed attention to the systemic pressures and cognitive biases that AI both exposes and exploits. Ultimately, lawyers have a responsibility not just to use AI wisely but to help steer its development in ways that serve justice, equity, and the public interest.

* Professor of Law, University of Nevada, Las Vegas William S. Boyd School of Law. Thank you to participants in the Texas A&M University School of Law Legal Ethics Schmooze for their thoughtful comments on an early draft. The author would also like to thank the UNLV Boyd School of Law Wiener-Rogers Law Library faculty, staff, and student workers for their research assistance. Thank you also to Gonzaga Law School, where the author was honored to hold the Clute-Holleran Scholar in Corporate Law position during early research for this Review.