Chevron in the Circuit Courts

Kent Barnett* and Christopher J. Walker** October, 2017

This Article presents findings from the most comprehensive empirical study to date on how the federal courts of appeals have applied Chevron deference—the doctrine under which courts defer to a federal agency’s reasonable interpretation of an ambiguous statute that it administers. Based on 1,558 agency interpretations the circuit courts reviewed from 2003 through 2013 (where they cited Chevron), we found that the circuit courts overall upheld 71% of interpretations and applied Chevron deference 77% of the time. But there was nearly a twenty-five-percentage-point difference in agency-win rates when the circuit courts applied Chevron deference than when they did not. Among many other findings, our study reveals important differences across circuits, agencies, agency formats, and subject matters as to judicial review of agency statutory interpretations.

Based on prior empirical studies of judicial deference at the Supreme Court, however, our findings suggest that there may be a Chevron Supreme and a Chevron Regular: whereas Chevron may not have much of an effect on agency outcomes at the Supreme Court, Chevron deference seems to matter in the circuit courts. That there is a Chevron Supreme and a Chevron Regular may suggest that, in Chevron, the Supreme Court has an effective tool to supervise lower courts’ review of agency statutory interpretations. To render Chevron more effective in creating uniformity throughout the circuit courts, the Supreme Court needs to send clearer signals on how courts should apply the deference standard.

Introduction

It is a bedrock principle of administrative law that a reviewing court must defer to a federal agency’s reasonable interpretation of an ambiguous statute it administers.¹ This Chevron deference doctrine is both untouchable and yet always under attack. Chevron deference has been a cornerstone of judicial review of agency action for more than thirty years, and the decision itself is one of the most cited Supreme Court decisions of all time. Indeed, as of this writing, Chevron has been cited in more than 80,000 sources available on Westlaw, including in roughly 15,000 judicial decisions and nearly 18,000 law review articles and other secondary sources.²

In these tens of thousands of sources, scholars, litigants, and judges have contested Chevron’s theoretical grounding,³ its provenance,⁴ and its impact on case outcomes.⁵ More recently, Supreme Court justices have questioned not only Chevron’s reach⁶ but also its very existence.⁷ Congressional Republicans have followed suit by introducing legislation that would abolish Chevron deference and require courts to review agency statutory and regulatory interpretations de novo.⁸

Much scholarly attention focuses on the use or absence of Chevron deference at the Supreme Court. Some scholars have focused on Chevron’s domain—that is, when Chevron applies in judicial review.⁹ Others have considered empirically how consistently the Court applies Chevron. In their leading study concerning agency deference in the Supreme Court from 1984 to 2006, Bill Eskridge and Lauren Baer found that the Court applied Chevron deference only one quarter of the time that it would have seemed to apply.¹⁰ When the Court applied the doctrine, agencies prevailed 76.2% of the time, a rate similar to those under other standards of review.¹¹

In other words, the Court’s choice to apply Chevron deference, as opposed to a less-deferential doctrine or no deference at all, does not seem to affect the outcome of the case. Chevron deference—at least at the Supreme Court—does not seem to matter. As Richard Pierce has concluded, “There is no empirical support for the widespread belief that choice of doctrine plays a major role in judicial review of agency actions.”¹² Scholars and commenters, moreover, have noticed the Court’s recent treatment of Chevron as a doctrine to ignore, disparage, or distinguish.¹³

But Chevron in the Supreme Court is not our focus. Instead, we are most concerned here with how Chevron works on the ground in the circuit courts. Prior empirical studies of Chevron in the circuit courts were limited to a particular court,¹⁴ particular agencies,¹⁵ particular subject matter,¹⁶ or a short timeframe.¹⁷ They have also largely concentrated on the rates at which agencies prevail under Chevron and the likelihood of judges’ policy preferences affecting Chevron’s application.¹⁸ Our inquiry and scope are significantly broader.

This Article presents the findings of the largest empirical study of Chevron in the circuit courts to determine how Chevron works outside the marbled enclave of One First Street. Our database of 2,272 judicial decisions, collected with broad search parameters, attempts to cull all published decisions from the circuit courts over an eleven-year period (2003–2013) that refer to the Chevron doctrine. Within the relevant 1,327 of those collected opinions, we uncovered 1,558 instances of judicial review of an agency statutory interpretation (not merely any kind of agency action). Largely following Eskridge and Baer’s methodology, we coded each agency statutory interpretation with respect to nearly forty different variables, including information about the decision (circuit, year, judges, and separate opinions); information about the agency interpretation (the agency, subject matter, final agency decisionmaker, agency procedure used, and ideological valence of agency’s interpretation); and information about the judicial outcome (outcome as to agency, ideological valence of the decision, standard of review applied, and factors that influenced the court’s decision).¹⁹ This broad set of cases permitted us to consider all instances within our parameters in which the circuit courts applied the Chevron framework. This set also permitted us to review all instances in which the circuit courts, having referred to Chevron, reviewed agency interpretations de novo or under the Skidmore deference regime (under which courts defer to an agency’s interpretation based on several factors, including the thoroughness of the agency’s interpretation and its consistency with prior pronouncements).²⁰

This treasure trove of data, albeit with methodological limitations that we discuss in Section II.B, provides a number of often-surprising insights regarding deference to agency statutory interpretations in the circuit courts. Many of these findings suggest, with some caveats, that there may be a Chevron Supreme and a Chevron Regular: whereas the choice to apply Chevron deference may not matter that much at the Supreme Court, it seems to matter in the circuit courts. Consider the following key findings from the study:

First, agency interpretations were significantly more likely to prevail under Chevron deference (77.4%) than Skidmore deference (56.0%) or, especially, de novo review (38.5%). In other words, agencies won significantly more in the circuit courts when Chevron deference applied, at least when the court expressly considered whether to apply Chevron. Indeed, there was nearly a twenty-five-percentage-point difference in agency-win rates with Chevron deference (77.4%) than without (53.6%). Because the agency-win rates in Eskridge and Baer’s study of the Supreme Court were much more similar no matter whether Chevron (76.2%), Skidmore (73.5%), or de novo review (66.0%) applied, this was one of our first indications that Chevron Supreme differs from Chevron Regular.²¹

Second, when Chevron’s well-known two-step approach applied, the circuit courts resolved the matter at step one (i.e., the step at which the courts ask whether Congress’s intent was clear) 30.0% of the time, and, of those Chevron step-one decisions, agencies prevailed 39.0% of the time. Of the 70.0% of the interpretations that moved to Chevron step two (the step at which the courts defer to reasonable agency interpretations when Congress’s intent was not clear at step one), the agency prevailed 93.8% of the time. Based on albeit-dated data from Tom Merrill, Chevron Supreme does not behave like Chevron Regular. Merrill found that the Supreme Court resolved matters in the agency’s favor 59% of the time at step one.²² (Merrill—and others after him—did not report comparable data about the Supreme Court’s step-two practice.) This difference may suggest that, given the higher likelihood of circuit-court review than Supreme Court review, agencies should give closer attention to the statutory language but that their step-two explanations are largely sufficient.²³

Third, as expected and as in the Supreme Court, formal agency interpretations prevailed at higher rates than informal ones, without regard to scope of review. But unlike in the Supreme Court, where the agency-win rate for formal adjudication (65.4%) was lower than notice-and-comment rulemaking (72.5%),²⁴ agency-win rates in formal adjudication were slightly higher (74.7%, or 81.3% when excluding immigration adjudications with idiosyncratic review procedures) than notice-and-comment rulemaking (72.8%) in the circuit courts. Formal interpretations, under a trilogy of Supreme Court decisions,²⁵ also unsurprisingly received Chevron deference at higher rates: 100.0% of the time for (albeit extremely rare) formal rulemaking, 91.9% for notice-and-comment rulemaking, and 76.7% for formal adjudication (or 85.2% if excluding immigration adjudications). Informal interpretations lagged behind at 44.8%. But, despite the Supreme Court treating legislative rulemaking and formal adjudication alike in its Chevron doctrine, our numbers revealed that the circuit courts applied the Chevron framework less frequently to formal adjudication than rulemaking. Perhaps even more surprising, when Chevron applied, interpretations in formal adjudications had a higher agency-win rate (81.7%, and 86.0% without immigration decisions) than notice-and-comment rulemaking (74.4%). These findings suggest that agencies may want to reconsider formal adjudication as a substitute to rulemaking as a means of adopting Chevron-eligible agency statutory interpretations, despite adjudication’s fall in popularity since the 1970s.²⁶

Fourth, the circuit courts varied considerably as to overall agency-win rates, application of Chevron, and agency-win rates under Chevron. For overall rates, the First Circuit was the most agency friendly with an agency-win rate of 82.8%, while the Ninth Circuit was the least agency friendly with a rate of 65.8%. As for Chevron’s application, the D.C. Circuit applied it almost as a matter of course at 88.6% of the time, while the Sixth Circuit applied it only 60.7% of the time. Once Chevron applied, though, the agency seemed to prevail as a rule in the Sixth Circuit (88.2% of the time, the highest rate), while the agency won only 72.3% of the time in the Ninth Circuit, the lowest rate. The differential between agency-win rates with and without Chevron indicates that agencies prevailed more in all circuits when Chevron applied. The most striking was the Sixth Circuit, with its nearly fifty-percentage-point difference in agency-win rates. Only the Eighth Circuit had a differential that was less than five percentage points, and the Eleventh Circuit was the only other circuit with a differential of less than ten percentage points. Although our data indicate that agencies win much more frequently when Chevron applies in all but one circuit, they also suggest that the Supreme Court may need to send clearer signals if the Court wants Chevron to apply evenly throughout the circuit courts.²⁷

Fifth, agency-win rates varied dramatically by subject matter and by the agency advancing the interpretation. For instance, the Federal Communications Commission (FCC) (82.5% overall agency-win rate), Treasury Department (78.9%), and, perhaps surprisingly, National Labor Relations Board (NLRB) (78.1%) were a few of the big winners among the agencies in the dataset. By contrast, the Equal Employment Opportunity Commission (EEOC) (42.9%), Energy Department (45.5%), and Department of Housing and Urban Development (HUD) (54.2%) were among the biggest losers in the circuit courts. The range of circuit courts applying Chevron deference also varied considerably from 100.0% for the Interstate Commerce Commission/Surface Transportation Board (ICC/STB) to 36.4% to the Federal Trade Commission (FTC). Moreover, independent agencies outperformed executive agencies as to overall agency-win rate (77.0% to 70.2%) and frequency of Chevron application (82.5% to 73.2%)—though agency-win rate when Chevron applied evened out some (79.6% to 76.8%).

Sixth, to our surprise, the circuit courts may not be as responsive to the Supreme Court’s (often conflicting or vague) signals concerning exceptions to Chevron for certain sensitive questions. The circuits applied Chevron to two sensitive questions that we coded—74.3% of regulatory-jurisdiction questions and 76.0% of state-law preemption ones—roughly at the same rate as the overall average (74.8%). Once Chevron applied, the agency-win rate for jurisdictional interpretations was lower at 70.5% than the average Chevron agency-win rate of 77.4% for all interpretations. But the 78.9% win rate for preemption interpretations was consistent with the overall average. The small population of sensitive-question interpretations, especially the preemption questions, limits the strength of inferences that one can draw from the data. But these data may suggest that at least for certain matters, the circuit courts have somewhat internalized the courts’ sensitive-question exceptions to Chevron as part of its overall analysis but not at “step zero” (i.e., the step where courts consider whether to apply Chevron’s two-step approach).²⁸

Seventh, long-standing agency interpretations prevailed under all deference regimes combined at a much higher rate (82.3%) than those that were new and replaced no earlier interpretation (65.9%), those that were inconsistent with a prior interpretation (59.8%), and those whose duration we could not discern from the decision (67.8%). The circuit courts, consistent with Supreme Court doctrine but not practice, applied Chevron consistently to long-standing and new interpretations, and—to our surprise—at an even higher rate to inconsistent interpretations. But inconsistent agency interpretations prevailed under Chevron much less frequently (65.6%) than recent (74.7%) and long-standing interpretations (87.6%). Indeed, we found that agencies’ inconsistent interpretations prevailed significantly less than other interpretations under every review standard except de novo review (although much more frequently under Chevron than other review standards). Moreover, inconsistent interpretations based on new political administrations or unclear reasons were the least likely to prevail. These findings suggest that agencies seeking to change positions should work diligently to ensure that their interpretations receive Chevron deference (by having the force of law) and that they rely on grounds such as changed circumstances or accumulated expertise.²⁹

Finally, we found that traditional contextual or theoretical grounds for deference do not have much expressed salience in the circuit courts. Courts mentioned only four of our nine coded factors in more than one in ten interpretations. The most salient factors were agency procedures, agency rulemaking authority, agency expertise, and interpretive stability. The first two’s prominence are of little surprise because they track concerns for formality and delegation in leading precedent. Moreover, expertise and interpretive stability are factors for Skidmore deference and factors that the Court has mentioned as relevant to Chevron in one leading decision. But the absence of the other factors—political accountability, public reliance, contemporaneity, uniformity in federal administrative law, and congressional acquiescence—suggests that the circuit courts have found comfort in the Court’s more rule-based line of decisions and have largely rebuffed another decision’s more open-ended, sliding-scale inquiry that could give them additional discretion.³⁰

The Article proceeds as follows. Part I briefly discusses Chevron’s birth, theoretical underpinnings, and evolution. Part II discusses prior empirical studies, our study design and methodology, and an overview of our dataset. Part III provides the 10,000-foot view of our findings, looking at agency-win rates, judicial application of Chevron deference (and its effect on agency-win rates), and the effect of agency procedures on outcomes and deference regimes. Part IV then disaggregates the findings by circuit, whereas Part V analyzes the results by agency and subject matter, including the differences between executive and independent agencies. Part VI examines what else seems to matter (or not) based on other variables in the dataset. The theoretical and normative implications of these findings, including how they support or cast doubt on existing doctrine and theory, are discussed in each of these Parts. The Article concludes by exploring the advantages of a Chevron Supreme and a Chevron Regular. Although Chevron may not have much of an effect on agency outcomes at the Supreme Court (based on prior empirical studies of the Court), it seems to matter markedly in the circuit courts. This may suggest that, in Chevron, the Supreme Court has an effective tool to supervise and rein in the lower courts in their review of agency statutory interpretations. But the Court needs to provide additional guidance to ensure that Chevron applies consistently throughout the circuits.

I. Chevron’s Ever-Changing Role

Federal courts have a long-standing practice of deferring in some manner to federal agency statutory interpretations. But the level of deference, triggers for deference, exceptions from it, and judicial discussions of deference have fluctuated significantly. Our purpose here is not to provide a complete history of judicial deference. Instead, this Part contextualizes Chevron deference as necessary to understand the salience of our findings.

A. Discursive Deference to Agencies Before Chevron

Early federal courts deferred to agency action in two key ways. First, they deferred to discretionary, as opposed to ministerial, executive decisions in mandamus actions.³¹ Second, outside of mandamus actions, they often “respected” agency interpretations of ambiguous statutory provisions when those interpretations were long-standing or contemporaneous with the statute’s enactment.³² As the administrative state grew in size and influence, debates surrounding the intensity of judicial review of agency legal interpretations became more urgent and strident.³³ Ultimately, the Supreme Court fluctuated from the New Deal until Chevron between three deference regimes.

Consistent with NLRB v. Hearst Publications, Inc., the first line of decisions called for significant deference to agency interpretations that had a “reasonable basis in law.”³⁴ In that case, for example, the Court deferred to the NLRB’s interpretation that it provided in its adjudication of “employee,” a “broad statutory term,” to include newsboys.³⁵ The Court grounded that deference on notions of congressional delegation to the agency to provide the interpretation and administrative expertise.³⁶

Skidmore v. Swift & Co.,³⁷ decided only one Term after Hearst, was the standard-bearer for the second line of decisions in which the courts applied an indefinite, multifactored inquiry to deference questions. The Skidmore Court deferred to the flexible, contextual method that the Department of Labor’s Wage and Hour Division called for in an interpretive ruling to determine whether “waiting time” was subject to overtime pay.³⁸ After noting that the agency’s interpretations were not controlling on the courts, the Court held that they “constitute[d] a body of experience and informed judgment” to guide courts.³⁹ The weight to give this agency guidance depends upon the thoroughness of the agency’s consideration, the validity of its reasoning, its consistency with previous and later agency pronouncements, and “all those factors which give it power to persuade.”⁴⁰

The final line appeared to apply de novo judicial review, without deference to the agencies. Richard Pierce identifies NLRB v. Bell Aerospace Co.⁴¹ as a key example in which the Court, ignoring the NLRB’s contrary adjudicatory interpretation, substituted its own interpretation of the term “employee” under the National Labor Relations Act.⁴² It did so without referring to notions of judicial deference.⁴³

As others have noted, the Supreme Court was not consistent, despite criticism from lower courts and despite scholarly efforts to provide a descriptive reconciliation of the competing scopes of review.⁴⁴ Nor did the Court seek to reconcile its various judicial-review regimes. But the Court’s 1984 Chevron decision, despite failing to provide doctrinal reconciliation, appeared to provide clearer, simpler guidance.

B. Chevron and Its Domain

In Chevron, the Court famously created a two-step process for judicial review of agency statutory interpretations that appeared to apply whenever “a court reviews an agency’s construction of the statute which it administers.”⁴⁵ As to the first inquiry, the court should determine if Congress has clearly provided its unambiguous intent on the issue. If Congress has not, then the court, in its second inquiry, should ask whether the agency’s construction is permissible.⁴⁶ The Court expressly stated that the agency has room to adopt more than one reasonable interpretation, meaning that an agency’s changed interpretation can receive deference.⁴⁷ The Court largely grounded Chevron deference on a theory that Congress had delegated interpretive primacy to the agency, instead of the courts.⁴⁸ But the Court also identified other theoretical support: agencies are institutionally superior to courts because of their expertise over complex statutory schemes, and executive officials are more politically accountable than judges.⁴⁹

Courts and scholars largely understood Chevron as a significant restatement or recalibration of judicial review,⁵⁰ and it quickly gained prominence in the lower courts, especially the influential D.C. Circuit.⁵¹ In so doing, questions concerning Chevron’s “domain” quickly surfaced.⁵² As we discuss in Section I.B.1, the most significant question was whether Chevron applied to all agency interpretations, or only those that were sufficiently formal.⁵³ In a trilogy of decisions from 2000 to 2002, the Court addressed when Chevron applied but sent inconsistent signals. As we discuss in Section I.B.2, additional questions arose as to Chevron’s reach even when agencies provided formal interpretations, including: Does Chevron apply to an agency’s pronouncement concerning its own jurisdiction? Or to an agency’s preemption of state law? Or to questions of deep economic or political significance?

1. The Role of Formality

In Christensen v. Harris County, the first of three key decisions concerning the role of formality, the Court held that an interpretation’s formality influenced Chevron’s applicability and indicated that informal interpretations were not eligible for Chevron deference.⁵⁴ The Court refused to apply Chevron to the Department of Labor’s statutory interpretation in an opinion letter, noting that an opinion letter is “not . . . a formal adjudication [that is, on-the-record adjudication under the APA] or notice-and-comment rulemaking.”⁵⁵ The problem with interpretations in opinion letters, policy statements, agency manuals, and other guidelines is that they “lack the force of law.”⁵⁶ Instead, they are entitled only to respect under Skidmore.⁵⁷ Justice Scalia concurred, arguing that Skidmore was an “anachronism,”⁵⁸ that Chevron should apply to all “authoritative agency positions,”⁵⁹ and that the Court had, indeed, applied Chevron to more than formal adjudication and notice-and-comment rulemaking.⁶⁰ Justice Breyer, although dissenting, supported the majority’s resuscitating Skidmore and contended that Chevron “made no relevant change” to judicial review.⁶¹ Instead, all it did was focus on congressional delegation.⁶²

One year later, the Court doubled down in United States v. Mead Corp. but left room for informal interpretations to receive Chevron deference.⁶³ The Court provided more guidance for when Congress had delegated interpretive primacy to agencies: by giving agencies the authority to act with the force of law through “relatively formal administrative procedure tending to foster . . . fairness and deliberation . . . . Thus, the overwhelming number of our cases applying Chevron deference have reviewed the fruits of notice-and-comment rulemaking or formal adjudication.”⁶⁴ Indeed, earlier in its opinion, it had stated that “[d]elegation . . . may be shown . . . by an agency’s power to engage in adjudication or notice-and-comment rulemaking.”⁶⁵ But, despite Christensen’s contrary statements, the Court acknowledged that Chevron’s applicability does not require such formality; the Court had bestowed Chevron deference upon informal agency interpretations.⁶⁶ In denying the U.S. Customs Service’s letter rulings at issue Chevron deference, the Court found no evidence of congressional intent for the rulings to have the force of law.⁶⁷ The Court remanded for the lower courts to decide whether Skidmore deference applied.⁶⁸ Conspicuously absent was any indication that other values, such as expertise, mattered for triggering Chevron deference. Justice Scalia, this time in dissent, largely repeated his views in Christensen.⁶⁹

Soon thereafter, the Court suggested in dicta that Chevron’s domain depended on more than the formality of agency action. In Barnhart v. Walton, the Court, after noting that the agency had exercised its rulemaking authority, deferred under Chevron to the Social Security Administration’s reasonable statutory interpretation in a regulation.⁷⁰ But it didn’t stop there. It held that the agency’s interpretation was “permissible” because it made “considerable sense,” was of long-standing duration (even if the regulation itself was of recent vintage), and appeared to receive congressional acquiescence in light of the relevant statute’s reenactment and amendment.⁷¹ The Court’s focus on the long-standing nature of the interpretation was surprising because it had held, in Chevron itself, that Chevron deference applies even when agencies change their interpretations.⁷² Moreover, consistent with Mead, the Court stated that formality was not necessary.⁷³ And it stated, perhaps even more surprisingly, that formality was also insufficient, which appears inconsistent with Mead’s strong suggestion that notice-and comment rules and formal adjudication always have the force of law to render agency action Chevron-eligible.⁷⁴ The Court then referred to other considerations, reminiscent of Skidmore’s.⁷⁵

Mead, Christensen, and Barnhart altogether created the “Mead Puzzle,” creating uncertainty for courts and agencies as to which factors are necessary or sufficient to trigger Chevron deference.⁷⁶ Generally, notice-and-comment rulemaking and formal adjudication have been thought sufficient for Chevron eligibility, despite Barnhart’s contrary suggestion,⁷⁷ and thus should be treated similarly. But lower courts, as Lisa Bressman has highlighted, have struggled with how to go about determining when informal interpretations are Chevron-eligible⁷⁸ and when they may engage in “Chevron avoidance”—that is, accept the agency’s view under Skidmore or de novo review without deciding whether the agency’s interpretation has the force of law.⁷⁹ At the same time, scholars, including one of us, have debated formality’s relationship with congressional delegation.⁸⁰ Some have argued that formality provides procedures that, in Mead’s words, “foster . . . fairness and deliberation”⁸¹ and provide a salience or transparency to permit congressional oversight.⁸² Others have argued that whether Congress wants to have an agency act with the force of law is a separate question from whether Congress wants searching judicial oversight of the agency’s binding actions.⁸³

2. Sensitive Questions

Even with sufficient formality, it was and continues to be unclear whether Chevron applies to certain sensitive questions: so-called jurisdictional matters, preemption, and exceptionally important policy matters. In addressing these questions, the Supreme Court has often provided more guidance on the theoretical underpinnings of Chevron deference and its triggers.

The Court in City of Arlington v. FCC held that agency decisions concerning their jurisdiction are eligible for Chevron deference, largely because of the difficulty in distinguishing jurisdictional questions from other statutory interpretations.⁸⁴ In resolving this long-simmering issue, the majority emphasized Chevron’s place in the judicial-review firmament as a stabilizing doctrine in the lower courts,⁸⁵ but oddly it left Mead’s place in doubt. Although the agency, according to the lower court, had engaged in informal adjudication with notice-and-comment opportunities,⁸⁶ the Court did not engage in a Mead inquiry into whether the action had the force of law. Instead, the majority relied on the fact that the agency had general rulemaking or adjudicatory authority and simply concluded without analysis that the agency had exercised that authority when providing its interpretation.⁸⁷ And it suggested that Barnhart’s dicta—that Chevron may not apply to formal adjudication or notice-and-comment rulemaking—was disfavored because the Court had never denied Chevron deference to interpretations in those formats.⁸⁸ Justice Breyer, concurring, abided by his contextual Barnhart approach.⁸⁹ Three dissenting justices, similarly to Justice Breyer, would have applied a searching inquiry into whether Congress delegated interpretive primacy over the specific interpretation at issue.⁹⁰

The Court, however, has not squarely addressed state-law preemption. It has applied both Chevron and Skidmore to agency preemption decisions, albeit disagreeing with the agency in both instances.⁹¹ Nina Mendelson has called for the Court to apply Skidmore deference based on agencies’ lack of expertise on federalism that influences preemption.⁹² And as a matter of congressional delegation, Skidmore seems appropriate. In their survey of legislative drafters, Abbe Gluck and Lisa Bressman found that a majority of drafters do not think that Congress delegates preemption matters to agencies.⁹³ Likewise, one of us has surveyed agency rule drafters, who mostly thought that statutory ambiguity does not signal congressional delegation of preemption matters to agencies.⁹⁴

Finally, the Court recently held in King v. Burwell that Chevron does not apply to questions of “deep ‘economic and political significance’ that [are] central to [the] statutory scheme” at issue.⁹⁵ In that case, the Court refused to defer to the IRS’s interpretation of “an Exchange established by the State.”⁹⁶ Although, in previous cases, the Court had considered an issue’s significance in deciding under Chevron step one whether Congress had clearly expressed its view,⁹⁷ this was only the second time that the Court refused to extend Chevron deference to an admittedly ambiguous statutory provision based on the significance of the matter.⁹⁸ Moreover, the earlier decision that did so—Gonzales v. Oregon—was in the context of a fundamental change to a long-standing regulatory scheme, as opposed to a significant interpretation in a new statute, as in Burwell.⁹⁹ Aside from delegation, the Court focused on the IRS’s lack of expertise on health-insurance policy,¹⁰⁰ suggesting that the Court had, once again, retreated from Mead and City of Arlington’s sole focus on formalized authority and action.¹⁰¹

Accordingly, the Supreme Court has created a complex framework for determining whether Chevron applies to agency interpretations. Congressional delegation of interpretive primacy to an agency undoubtedly has a preeminent place in the Court’s calculus. But how courts are to discern such a delegation depends on the authority of the agency to act with the force of law, the formality and procedure involved in the agency action, agency expertise, and the nature of the legal interpretation at issue. Similarly, the Court has referred to other values that influence Chevron’s applicability or judicial deference to the agency under any framework, such as the long-standing nature of the agency’s interpretation (albeit with repeated proclamations that agencies are eligible for Chevron after changing interpretations), political accountability, or congressional acquiescence. And scholars have added other values: national uniformity¹⁰² and contemporaneity of an agency’s interpretation with statutory enactments.¹⁰³ Indeed, much like Justice Breyer, Evan Criddle has argued that deference under Chevron requires the satisfaction of all leading theoretical grounds—delegation, expertise, political accountability, rationality, and uniformity; anything less calls for Skidmore’s framework.¹⁰⁴ Part VI assesses what purchase these leading values and other contextual factors, aside from the canonical delegation theory, have on the federal circuit courts.

C. Chevron for Thee, But Not for Me

Perhaps given Chevron’s complexity, the Supreme Court has not been consistent. In two empirical studies, Eskridge and Baer found that the Supreme Court applied Chevron deference only one-quarter of the time in which it would have seemed to apply.¹⁰⁵ These findings were also generally consistent with Tom Merrill’s earlier findings that the Court used Chevron “only about one-third of the [time].”¹⁰⁶

The Court’s questionable loyalty to Chevron suggests that the doctrine is not meant to discipline Supreme Court decisionmaking. Instead, the doctrine may better serve to control lower courts and provide nationwide uniformity.¹⁰⁷ After all, the Court itself recently referred to Chevron as serving a “stabilizing purpose” to prevent “[t]hirteen Courts of Appeals [from] applying a totality-of-the-circumstances test [that] would render the binding effect of agency rules unpredictable.”¹⁰⁸ And perhaps most importantly for lower courts, it did so in the context of rejecting the dissent’s more provision-specific inquiry and, à la Mead, looking only at force-of-law authority.¹⁰⁹

II. Empirical Study of Judicial Deference

Before exploring our findings in Parts III through VI, we provide in this Part a brief overview of key empirical studies of Chevron in the Supreme Court and the circuit courts, our methodology, and an overview of the composition of our dataset.

A. Prior Empirical Studies of Chevron

Our purpose here is to identify the basic methodology of the prior studies that were relevant to our methodological choices and results. We discuss our or others’ specific findings or methods in more detail as relevant to our discussion in subsequent Parts.

1. Key Studies Concerning the Supreme Court

The most comprehensive study of deference in the Supreme Court comes from Eskridge and Baer. Their study provided the model for our Codebook and many of our variable fields, and it provides much of the data to compare deference in the Supreme Court and courts of appeals. In their study, they evaluated all Supreme Court decisions reviewing an agency’s statutory interpretation between the issuance of Chevron in 1984 and Hamdan v. Rumsfield in 2006.¹¹⁰ They coded 1,014 interpretations for 156 variables¹¹¹ to describe which deference regimes the Supreme Court applied and how it did so.¹¹² As most relevant here, they found that agencies prevailed 68.3% of the time, without regard to deference regimes.¹¹³ The Court applied Chevron in only 8.3% of the decisions, and agencies prevailed 76.2% of the time under Chevron.¹¹⁴ Skidmore, for its part, applied to 6.7% of the decisions, and agencies prevailed 73.5% of the time,¹¹⁵ a rate that was significantly higher than the 60.4% win rate in Kristin Hickman and Matthew Kreuger’s earlier study that considered Skidmore in the circuit courts.¹¹⁶ Eskridge and Baer also considered various “ad hoc” variables to which the courts have referred in their deference decisions (some of which were also factors in Barnhart)¹¹⁷: the interpretation’s longevity, the subject matter, whether the interpretation concerned certain sensitive issues (such as preemption and jurisdiction), and factors that the Court referred to in its decision (such as congressional acquiescence, agency procedures, rulemaking authority, etc.).¹¹⁸

One other study is relevant to our findings here. To determine the impact of the so-called Chevron revolution in the years shortly before and after Chevron, Tom Merrill considered all decisions in the Supreme Court from 1984 to 1990 in which at least one justice identified a question of whether the Court should defer to an agency’s statutory interpretation.¹¹⁹ He determined that, similar to Eskridge and Baer’s findings, the Court applied Chevron in only about one-third of the applicable decisions,¹²⁰ and, surprisingly, the agencies had a lower win rate after Chevron (both for the aggregate of the decisions reviewed and for only those decisions in which the Chevron framework applied), despite Chevron’s reputation for being agency friendly.¹²¹ Finally, he found that the Court referred less often to traditional deference factors (such as the interpretation’s long-standing nature and contemporaneity, congressional ratification, etc.) after Chevron.¹²²

2. Key Studies Concerning Circuit Courts

More studies concern the courts of appeals, each with its own limitations on matters of judicial deference because of each project’s focus. The largest study, by Peter Schuck and Donald Elliott, considered all administrative actions on direct review in the courts of appeals in various time periods (1965, 1974–1975, 1984–1985, and 1988) to obtain a descriptive account of how judicial review of agency action works in the lower courts—both before and after Chevron.¹²³ Although we refer to some of their findings in our discussion below, their data, despite concerning nearly 2,500 published and unpublished decisions on direct review,¹²⁴ are of limited comparative use here. As prior scholars have noted, Schuck and Elliott neither limited their data to judicial review of agency statutory interpretations (and, relatedly, whether courts validated agency interpretations) nor indicated how many interpretations were upheld or reversed.¹²⁵ Moreover, their more-than-a-quarter-century-old study occurred a decade before the Court’s reaffirmation of the Skidmore and Chevron dichotomy, after which one would expect that courts might decide decisions under the two regimes differently. Finally, they primarily focused on whether Chevron affected agency-remand rates, as compared to our focus on acceptance rates of agency interpretations specifically.¹²⁶

Two other studies concerning the circuit courts largely focus on the role of judicial ideology, and thus they are of limited relevance of our findings here. In one study, Tom Miles and Cass Sunstein considered all published circuit-court decisions from 1990 to 2004 that applied Chevron to legal interpretations by the EPA and the NLRB.¹²⁷ Because their purpose was to investigate whether Chevron limited political ideology from affecting judicial review, they focused on “two important agencies known for producing politically contentious decisions.”¹²⁸ A second study by Frank Cross and Emerson Tiller considered the “whistleblower effect” in Chevron decisions in the circuit courts—that is, the effect of having one judge of a different ideology on a panel who can alert others to the majority’s failure to adhere to Chevron.¹²⁹ To do so, they considered all decisions from the D.C. Circuit from 1991 to 1995 that cited Chevron.¹³⁰ But, similar to Miles and Sunstein’s study, their focus on one particular question and decisions from only one court limits the study’s comparative value to the issues that we address here.

Finally, Orin Kerr’s study had broader data than the studies above, providing us more comparative findings. Kerr sought to evaluate how well three leading models (contextual, political, and interpretive) describe Chevron in the circuit courts.¹³¹ To that end, he reviewed 253 agency statutory interpretations in published circuit-court decisions from 1995 and 1996 that applied the Chevron framework.¹³² Unlike Schuck and Elliott, who considered all administrative actions and remand rates, he counted interpretations and considered how often agencies prevailed on each interpretation.¹³³ But similar to Shuck and Elliott, he considered only direct review of agency actions.¹³⁴ His data concerning the “overview of Chevron” and the contextual model (similar to the traditional or Barnhart factors that Eskridge and Baer as well as Merrill considered) provide the most relevant comparisons to our findings here,¹³⁵ despite his shorter timeframe and smaller population of interpretations.¹³⁶

B. Our Study Design and Methodology

For our study, we completed several related searches on Westlaw to attempt to capture all published decisions over an eleven-year period in which the circuit courts referred to Chevron. In contrast to prior studies’ focus on certain courts, agencies, or issues, we considered all circuits and attempted to include interpretations from all federal agencies. Likewise, in contrast to short time frames in most other studies,¹³⁷ we selected an eleven-year period from January 1, 2003, until December 31, 2013. We began our study with decisions from 2003 to ensure that courts and parties had sufficient time to become accustomed to (1) instructions in the Supreme Court’s 2001 United States v. Mead decision that clarified Chevron’s applicability and Skidmore’s renaissance,¹³⁸ and (2) certain factors for Chevron eligibility that the Supreme Court mentioned in dicta in 2002’s Barnhart v. Walton.¹³⁹ Such a large dataset allowed us to draw more meaningful conclusions concerning the circuit courts collectively, each circuit individually, and each agency with a significant number of reviewed interpretations.

Searching for all instances of judicial review of agency statutory interpretations, as Eskridge and Baer did, was not feasible for the appellate courts. Instead, we narrowed our parameters by focusing on references to Chevron, similar to nearly all of the studies concerning circuit courts.¹⁴⁰ Limiting ourselves to decisions that cited Chevron would give us the best data on Chevron itself and, because of Chevron’s prominence, extremely useful data on decisions in which judicial deference was in dispute. Although our dataset does not permit us to paint as complete a picture as to non-Chevron regimes as the Eskridge and Baer or Hickman and Krueger studies, we anticipated obtaining a sufficient number of applications to inform understandings of those doctrines.

We engaged in a broad, yet practicable, search for decisions. Simply searching for “Chevron” returned too many false positives, since Chevron Corporation is a party in numerous decisions and “Chevron” is part of the case style of numerous other, unrelated decisions. To identify the decisions concerning Chevron deference, we prepared searches that include “Chevron” with relevant terms: agency, ALJ, order, formal adjudication, rule, and 553 (the APA section that concerns notice-and-comment rulemaking).¹⁴¹ To render it more likely that we captured all Chevron deference decisions even if the court did not use the word “agency” in its opinion, we ran similar, yet even broader, searches for approximately twenty-five federal agencies that we thought would be most likely to be involved in statutory interpretation.¹⁴² Because Mead indicated that Chevron is most applicable to formal adjudication and notice-and-comment rulemaking,¹⁴³ we used terms especially designed to capture decisions concerning these methods of agency action. But our terms (e.g., rule, order, and interpretation) would also have captured less formal methods of agency action, such as interpretive rules or informal adjudications, to which Chevron can apply.¹⁴⁴ Unlike Schuck and Elliott and some of Kerr’s data, we collected decisions concerning direct and collateral review. We then combined all of the collected decisions into one database, removed duplicates, and removed a handful of obviously irrelevant authorities (such as unpublished decisions or treatises that were included in the results for unknown reasons). We ultimately had a database of 2,272 decisions.¹⁴⁵

Our research assistants initially reviewed the decisions, and we then completed a secondary review of every decision to increase uniformity and validity. In our secondary review, we divided the cases up randomly for one of us to review, and we flagged cases for a third-level review where the other then weighed in. One of us then conducted a more systemic review of the cases in preparing the dataset for analysis in the IBM SPSS statistics software. For all decisions with at least one instance of an agency’s statutory interpretation of a statute that it administers, we coded each instance of interpretation within one case as its own entry (as Kerr, Re, and Hickman and Krueger did, but Eskridge and Baer did not), meaning that one decision could have more than one entry in our dataset.¹⁴⁶ We had a total of 1,558 separate instances of statutory interpretation from 1,327 judicial opinions.

We coded the decisions on a spreadsheet for thirty-seven¹⁴⁷ variables identified in our Codebook, which provided guidance to our reviewers in coding. In broad strokes, aside from relevance,¹⁴⁸ we coded the decisions for the following:

identifying information, such as the relevant circuit, judges, and additional opinions concerning statutory interpretation;
the nature of the agency’s interpretation, such as the agency, the subject matter, the agency’s format and final decisionmaker,¹⁴⁹ the political valence of the interpretation largely based on Eskridge and Baer’s definitions,¹⁵⁰ and the long-standing nature of the agency’s interpretation or novelty (and reasons for any new interpretation);
whether the agency’s interpretation concerned certain sensitive topics, such as its own jurisdiction, state-law preemption, or foreign affairs; and
the nature of the judicial decision, such as the result for the agency, political valence of the court’s interpretation based on Eskridge and Baer’s definitions, the applied deference regime, how the court applied Chevron if it was the applicable regime, and traditional factors that Eskridge and Baer identified in their study.

As we discuss our findings in Parts III through VI, we will explain additional methodological matters as relevant to our findings’ implications. Because of the wealth of information provided by the raw numbers alone, in this Article we have chosen to present our findings descriptively, saving more sophisticated statistical analysis of the data for subsequent work. But before turning to our findings, it is important to note a number of significant methodological limitations inherent in our study design.

First, like most of the prior studies, we reviewed only published decisions based on the view that the courts were likely to designate decisions as published in which they reviewed a federal agency’s statutory interpretation and that they were likely to mark decisions as unpublished when they referred to their past review of agency interpretations as circuit precedent. As one of us has documented in an empirical analysis of constitutional litigation, it may well be the case that courts engage in strategic behavior by not publishing certain decisions.¹⁵¹ Although it is theoretically possible that a court could strategically not publish a decision that strikes down an agency statutory interpretation or refuses to apply Chevron deference, it is more likely that unpublished decisions both apply Chevron deference and uphold agency statutory interpretations. Accordingly, if anything, our findings likely underestimate the overall effect of Chevron deference in the circuit courts.

Second, we reviewed only circuit courts. To be sure, district courts also apply Chevron.¹⁵² But we reasoned that many, if not most, review comes to the courts of appeals and significant district-court decisions would likely be appealed—though one could argue that the federal government may not appeal certain losses for fear of creating binding precedent. Similarly, our study can only evaluate the effect of Chevron deference with respect to agency statutory interpretations that actually make it to the circuit courts. Our prior experience litigating such cases suggests that regulated entities and individuals will often not waste resources to bring judicial challenges to agency statutory interpretations precisely because of the deferential standards of review. In other words, our findings may well underestimate the overall effect of Chevron deference on agency interpretive practices.¹⁵³

Third, we culled only those decisions in which courts invoked Chevron by name. Our database did not include instances in which they referred to a Chevron-like doctrine by another name, including the name of a circuit or Supreme Court precedent that functioned similarly,¹⁵⁴ or failed to refer to Chevron based on inadvertence of strategic behavior.¹⁵⁵ Nor does our dataset include cases where Chevron was not mentioned at all. This approach limits our ability to compare agency-win rates under Chevron and other deference standards, though the findings of other studies on Skidmore deference and de novo review are consistent with our findings. For instance, one could imagine instances in which a reviewing court intentionally does not cite Chevron deference when setting aside an agency statutory interpretation. Perhaps more likely, though, are situations where the federal agency strategically decides not to invoke Chevron deference for fear of limiting Chevron’s domain in future cases of similar procedural posture (less-formal procedures) or substantive position (major constitutional or policy questions). Or, conversely, in easy interpretation cases, the agency may forgo more formal procedures and thus not request any deference.¹⁵⁶

To further address this methodological limitation, we coded separately every published circuit-court decision that cited Skidmore during that same eleven-year period (2003–2013). The total number of such cases was 168, of which only 55 were deemed relevant. Because this Article focuses on how circuit courts cite and use Chevron, we have decided not to include these Skidmore-only cases in our description of the findings. Instead, we separately note, where helpful for comparison purposes, the findings for the Skidmore-only cases.

Fourth, although we largely based our coding on Eskridge and Baer’s model, our comparative findings are each based on different periods of time. Their study considers decisions from approximately 1985 until 2006, while ours considers decisions from 2003 until 2013. Likewise, one should remember that other studies of Chevron in the circuit courts, to which we often compare our data, consider different time periods, or sometimes limited circuits or subject matters.

Fifth, consistent coding is inherently difficult because of the large number of decisions and the judgments required in the face of unclear judicial language. To mitigate these concerns, we included several procedural checks in our study design, such as continual communication with reviewers during the review process, a secondary review by only the two of us (with significant communication during the secondary review), and a tertiary systemic review in the IBM SPSS statistics software to check for inconstancies across cases. A number of our coding variables facilitated this systemic review since they required consistent answers. We also cross-checked numerous coding fields to improve consistency after our secondary review.¹⁵⁷

C. Overview of Our Dataset

Before presenting our findings in Parts III through VI, it is helpful to sketch out the composition of the dataset. As noted in Section II.B, our original set of 2,272 circuit-court decisions published from 2003 through 2013 resulted in 1,558 instances (from 1,327 decisions) in which a circuit court reviewed an agency statutory interpretation. The interpretations are evenly spread throughout the eleven-year time period.¹⁵⁸ But they are not as evenly distributed by circuit, agency, subject matter, or format of agency procedure.

For example, nearly one in five interpretations came from the D.C. Circuit (19.7%), followed by the Ninth Circuit (16.9%), Second Circuit (11.0%), Third Circuit (8.5%), and Federal Circuit (7.9%).¹⁵⁹ Although scholars and practitioners likely expect the D.C. Circuit’s preeminence based on its role as “a de facto, quasi-specialized administrative law court,”¹⁶⁰ Schuck and Elliott found that, as late as the 1980s, the Federal Circuit reviewed the most agency decisions.¹⁶¹ As for subject matter, 30.6% of interpretations concerned immigration, perhaps explaining in part the Ninth Circuit’s disproportionate share of the interpretations in the dataset.¹⁶² The environment (13.9%) and entitlement programs (8.9%) were the next most predominant subject matters. If the subject matters of employment, labor/collective bargaining, and pensions are combined, they represented 10.5%. The dataset’s breakdown by agency was similar.¹⁶³

As for agency format, of the 1,558 interpretations, roughly a third resulted from notice-and-comment rulemaking (36.5%) and another third from formal adjudication (36.1%). With the latter, we included immigration adjudications and similar adjudications that perhaps are not APA-defined formal adjudications but nevertheless have been recognized by courts as being sufficiently formal to be accorded Chevron deference. Only four (0.3%) agency interpretations arose in formal rulemaking, whereas the remaining interpretations (24.8%) involved some sort of informal interpretation.¹⁶⁴

Finally, nearly two-thirds (63.0%) of the agency interpretations were “conservative” under the Eskridge–Baer model,¹⁶⁵ with 29.2% “liberal” and the remainder (7.8%) neutral, mixed, or otherwise too difficult to categorize. By contrast, only half (51.3%) of the court decisions on the agency statutory interpretation were “conservative,” with 40.9% “liberal” and the remainder (7.8%) neutral, mixed, or otherwise too difficult to categorize. In other words, the circuit courts tended to decide statutory interpretation issues more liberally than agencies.¹⁶⁶

III. General Findings on Chevron in the Circuit Courts

We begin by considering three categories of findings: (1) agency-win rates under all standards of review and the frequency of standards of review in our dataset, (2) how circuit courts applied Chevron’s two steps when Chevron applied, and (3) agency-win rates under differing interpretive formats.

A. Agency-Win Rates and Deference Differences

Consistent with prior studies, federal agencies in this study prevailed most of the time—in 71.4% interpretations—when we considered all interpretations together (that is, under any scope of review). None of the prior studies tracks perfectly with ours, but some provide limited comparison. For instance, Schuck and Elliott found that, in 1984 and 1985, the agency prevailed 76.7% of the time based on the overall outcome for the agency (meaning the result for the agency, whether or not the agency prevailed on the statutory-interpretation issue alone) in their review of circuit-court decisions of any agency action on direct review.¹⁶⁷ Likewise, Eskridge and Baer found that the agency prevailed 68.3% of the time in their review of Supreme Court decisions from 1984 until 2006 in which an agency’s interpretation of a statute was at issue.¹⁶⁸ And Tom Merrill’s review of similar Supreme Court decisions from 1984 until 1990 found that the agency prevailed 70.0% of the time.¹⁶⁹

The overall win rate differed somewhat depending on whether the agency statutory interpretation under review was “conservative” or “liberal” per the Eskridge–Baer model: “conservative” agency statutory interpretations were upheld 69.3% of the time (982 total “conservative” interpretations), whereas “liberal” interpretations were upheld 74.5% of the time (455 total “liberal” interpretations). In the remaining 121 interpretations where the agency interpretation was neutral, mixed, or otherwise too difficult to categorize, the agency won 76.0% of the time.¹⁷⁰

Similarly, in 74.8% of interpretations the circuit courts applied the Chevron deference framework.¹⁷¹ By contrast, they applied the Skidmore standard to 10.8% of the interpretations and refused to apply any deference (de novo review) to 7.5% of them. In the remaining interpretations (107 interpretations, or 6.9% of total interpretations), the courts declined to choose a deference standard, usually holding that the answer would have been the same under any standard. When discussing these deference-regime findings, care should be taken, especially in comparing the findings from this study with those of prior studies. This study looked only at decisions in which courts cited Chevron deference, so it is no doubt far from a complete picture of the Skidmore and no-deference precedent in the circuit courts. That said, our large number of Skidmore and no-deference decisions provides a meaningful understanding of judicial review of agency action more generally, even if not a complete picture. It is also probably reasonable to conclude that our study captures the vast majority of published circuit-court decisions during the time period where the agency requested Chevron deference (as one would assume that courts would typically address the deference question if a party raised it).

As detailed in Figure 1, the agency prevailed at a higher rate than the overall agency-win rate (77.4% to 71.4%) when the court determined that Chevron applied. Conversely, the win rate dropped considerably when the court did not apply the Chevron standard: 66.4% when the court refused to decide which standard applies; 56.0% under the Skidmore standard; and 38.5% when the court applied de novo review.¹⁷²

***BARNETT & WALKER FIGURE 1 HERE***

Again, comparison between deference regimes based on the decisions reviewed should be done carefully, since the dataset only includes decisions in which circuit courts expressly mentioned Chevron deference. It would not include decisions in which the court only mentioned Skidmore or reviewed interpretations de novo without mentioning Chevron—perhaps decisions in which one may expect higher agency-win rates whose inclusion would alter the results that we found. But at least in instances in which the court recognizes Chevron expressly in its opinion, the application of the Chevron framework seems to make a meaningful difference as to whether agencies prevail on the interpretive question. Indeed, there was nearly a twenty-four-percentage-point difference in win rates when the circuit courts applied Chevron deference (77.4%) than when they refused to apply it (53.6%). The agency was twice as likely (77.4% to 38.5%) to prevail if the court applied Chevron deference as opposed to reviewing the interpretation de novo and nearly three-fourths more likely (77.4% to 56.0%) to prevail under Chevron than Skidmore. In other words, agencies won more in the circuit courts when Chevron deference applied, at least when the court expressly considered whether to apply Chevron deference.

These findings challenge certain conclusions based on earlier studies. Evaluating affirmance rates in the Supreme Court and circuit courts from earlier studies, Richard Pierce found that, as relevant here, the affirmance ranges for de novo, Skidmore, and Chevron review overlap: 66% for de novo review, 55.1% to 70.9% for Skidmore, and 64% to 81.3% for Chevron.¹⁷³ He concluded that “a court’s choice of which doctrine to apply in reviewing an agency action is not an important determinant of outcomes in the Supreme Court or the circuit courts.”¹⁷⁴ Contrary to his conclusion concerning the circuit courts, our findings suggest that agency-win rates are meaningfully different under different deference regimes.

Our findings concerning Chevron (77.4% agency-win rate) are within the range of prior affirmance rates for circuit courts—from 64.0% to 81.3%.¹⁷⁵ But importantly, our findings indicate that the affirmance rate is significantly towards the upper end of the range, and they may have the most validity because our data were the only to consider all agencies and all circuit courts over more than a decade.¹⁷⁶ Interestingly, our agency-win rate is almost identical to the one that Eskridge and Baer found for the Supreme Court (76.2%).¹⁷⁷

Likewise, our finding that circuit courts agreed with agencies 56.0% of the time when Skidmore applied is consistent with three of four earlier studies finding relatively lower affirmance rates. Those three earlier studies found agency affirmance rates of 55.1% in 1965, 60.6% in 1975, and 60.4% from 2001 to 2005.¹⁷⁸ The consistency of findings within a range of approximately five percentage points suggests that one study of decisions from 1984 with the highest affirmance rate of 70.9% in the circuit courts is an outlier.¹⁷⁹ Although our Skidmore data come only from decisions that also mentioned Chevron and not all decisions that applied Skidmore, we are comforted that our results are very similar to results from earlier studies that did include all Skidmore decisions.

Finally, the affirmance rates for the circuit courts’ de novo review only further support this view. Although the de novo affirmance rate was 66.0% in the Supreme Court (with no data available for the circuit courts),¹⁸⁰ our data revealed that the circuit courts affirmed agencies’ interpretation only 38.5% of the time, at least when the court had cited Chevron. Our data, accordingly, suggest that the range of affirming agency interpretation is about forty percentage points from de novo to Chevron review and about twenty percentage points from Skidmore to Chevron. Contrary to Pierce’s conclusions based on earlier studies, agency-win rates do appear to differ significantly under different deference regimes.

Of course, these data cannot demonstrate a causal relationship between deference regimes and agency-win rates. For instance, courts could be strategically choosing deference regimes that more easily allow them to reach an outcome that matches their policy preferences. But one shouldn’t overstate this concern. First, Mead constrains judicial discretion by focusing heavily on force-of-law authority. Second, there is probably no perfect way to test for strategic behavior because we would have to know the “correct” result and compare that to the result that the court reached. That said, there are some ways to see if courts are seeking to get around Mead. One easy way would be to invoke Barnhart’s ad hoc factors or theoretical grounds for deference, but, as described in Section VI.C, they rarely do so. Indeed, the courts applied Chevron at almost identical rates to “conservative” and “liberal” agency interpretations.¹⁸¹ Future work that considers ideology may provide more insight.

In sum, using agency-win rates as an admittedly less-than-perfect heuristic to assess the meaningfulness of deference regimes, as others before us have done, we see that deference regimes appear to matter.

B. How Chevron Is Applied

Of the 1,558 total interpretations reviewed, the circuit courts applied the Chevron framework in 1,166 of them (74.8%). Of those 1,166 interpretations, the agency prevailed 902 times (77.4%). The more interesting questions, however, may concern how the circuit courts applied the two-step framework. In other words, how many decisions were decided at step one? How many were decided at step two? And, perhaps most importantly, what were the agency-win rates at each step? Figure 2 depicts the overall win/loss numbers at both steps, with the percentages reflecting the portion of the set of interpretations in which the circuit courts applied the Chevron framework.

***BARNETT &WALKER FIGURE 2 HERE***

Consistent with prior studies, the vast majority of agency interpretations (817 interpretations, or 70.0%) made it to step two.¹⁸² And an even greater percentage of interpretations that made it to step two (766 interpretations, or 93.8%) were upheld. Indeed, we found that the agency won slightly more under step two (whether the court describes its analysis as one of “reasonableness” in one step or two) than in an earlier study. In comparison to our agency-win rate of 93.8% under step two, Kerr found that agencies in 1995 and 1996 won at step two or in a one-step “reasonableness” inquiry a combined total of 84.7% (156 out of 184 interpretations) of the time.¹⁸³ To be sure, it is not true that Chevron, at least as an empirical matter, has collapsed into just one step of statutory ambiguity.¹⁸⁴ In particular, fifty-one agency statutory interpretations in our dataset—6.2% of those cases that made it to step two—were deemed unreasonable even though the court found the statute to be ambiguous as to the question at issue.

What happens at step one is perhaps even more noteworthy. Courts decided 30.0% of interpretations at Chevron’s step one. This finding is consistent with Kerr’s earlier finding that circuit courts resolved 27.2% of Chevron deference interpretations at step one.¹⁸⁵ But step-one resolution did not mean that the agency lost. Our data indicated that the agencies still prevailed 39.0% of the time, meaning that the agency’s interpretation was the only possible one under the statute. Our finding, once again, is similar to the 42.0% win rate Kerr found.¹⁸⁶ This number is roughly the same as the agency-win rate when the circuits reviewed interpretations de novo (38.5%).

Nevertheless, limited data on the Supreme Court differ. Tom Merrill found that the Court resolved matters in agencies’ favor 59.0% of the time at step one over seven Terms,¹⁸⁷ significantly more often than the circuit courts in our study. To put these numbers in perspective, Figure 1 is reproduced here as Figure 3, but now with Chevron step one and step two broken out into distinct standards.

These findings concerning Chevron step one—where the Court finds that the statute has only one clear meaning—are important for at least two reasons. First, a step-one ruling in favor of the agency cements the agency’s current interpretation in place, such that subsequent presidential administrations will not be able to change positions. Nor may the agency change positions based on changed circumstances short of statutory amendment. Second, these findings suggest that the circuit courts may well be taking the Brand X Court’s lesson that they clarify the nature of their holding to ensure that the agency knows whether it has discretion (or not) to change its statutory interpretation in the future.¹⁸⁸

***BARNETT & WALKER FIGURE 3 HERE***

C. Rulemaking Versus Adjudication

As detailed in Section I.B.1, a trilogy of Supreme Court decisions has suggested that the formality of the agency procedure may affect the level of deference accorded to the agency statutory interpretation. Moreover, those decisions suggest that all legislative rulemaking and formal adjudication should be treated similarly. The scholarship and empirical studies on point are plentiful. Our dataset sheds substantial empirical light on the role of formality in Chevron in the circuit courts.

1. Agency Procedure and Overall Agency-Win Rates

Roughly a third (36.5%) of the 1,558 interpretations in the dataset resulted from notice-and-comment rulemaking and another third from formal adjudication (36.1%).¹⁸⁹ Only 4 (0.3%) agency interpretations arose in formal rulemaking (rulemaking required by statute to be “on the record after opportunity for an agency hearing”¹⁹⁰)—a finding that reinforces the well-settled understanding that formal rulemaking “has become almost extinct” since 1973.¹⁹¹ The remaining interpretations (24.8%) involved some sort of informal adjudication. Due to difficulty in coding as rulemaking or adjudication, the 37 FERC interpretations (2.4%) were treated as a separate category in the dataset and not addressed in the following discussion.¹⁹²

Perhaps in light of the Supreme Court’s focus on procedural formality, it should be no surprise that agencies win nearly three-fourths of the time when their interpretation is the product of notice-and-comment rulemaking (72.8%) or formal adjudication (74.7%). By contrast, the win rate falls to 65.0% when an agency uses a less formal means. (The win rate for formal rulemaking is only 50.0%, but with only 4 interpretations in the dataset, one should read little, if anything, into that finding.) Figure 4 depicts these findings.¹⁹³

Among the 562 interpretations from formal adjudication, however, there are 386 immigration interpretations, whose agency-win rate is lower (70.2%). If those interpretations are removed, the agency-win rate in formal adjudication rises to 84.7%—more than ten percentage points greater than the rate for informal rulemaking.¹⁹⁴

***BARNETT & WALKER FIGURE 4 HERE***

Our results for rulemakings and informal interpretations are similar to those that Eskridge and Baer found for the Supreme Court. They found that legislative rules and executive orders prevailed 72.5% of the time under all deference regimes combined,¹⁹⁵ which is almost the exact same as our finding of 72.8% in the circuit courts. Similarly, our agency-win rate for informal interpretations of 65.0% in the circuit courts is similar to theirs of 68.1% in the Supreme Court.¹⁹⁶

But findings concerning formal adjudication significantly differ. In contrast to our finding that agencies prevailed slightly more often in formal adjudication (74.7% including immigration cases and 84.7% without them) than in informal rulemaking (72.8%), Eskridge and Baer found that agency interpretations in formal adjudication prevailed in the Supreme Court only 65.4% of the time, slightly less than the win rate of 72.5% for agency interpretations from rulemakings.¹⁹⁷ Likewise, our findings indicated that, depending on whether one includes immigration interpretations, agency interpretations from formal adjudication prevailed slightly or significantly more often than the overall agency-win rate for all formats (71.4%) in the circuit courts. Eskridge and Baer found the opposite, with the agency-win rate from formal adjudication in the Supreme Court slightly below the average win rate (68.8%).¹⁹⁸ Although the nearly ten-percentage-point difference between our formal adjudication agency-win rates (our 74.7% to their 65.4%) is meaningful by itself, it is perhaps more appropriate to compare our 84.7% finding that excluded immigration decisions with their 65.4% finding because Eskridge and Baer indicated (without percentages or absolute numbers) that the two largest groups of formal adjudications that they considered were from the NLRB and the FLRA.¹⁹⁹ If these are the more appropriate comparative groups, then the difference in agency-win rates in formal adjudication increases by nearly twenty percentage points between the circuit courts and the Supreme Court.

2. Agency Procedure and Chevron

Turning to whether the courts applied Chevron deference, we found that the data, depicted in Figure 5, are consistent with what one would expect from the Supreme Court precedent in one respect but not another. The findings are consistent with Mead in that formal interpretations receive Chevron deference at higher rates than informal interpretations, but they are inconsistent with expectations that rulemaking and formal adjudication are treated the same.

As detailed in Figure 5, the circuit courts applied the Chevron framework in 91.9% of notice-and-comment rulemakings and in all 4 formal rulemakings in the dataset. For formal adjudication, however, courts applied the Chevron deference framework to only 76.7% of interpretations. Not surprisingly based on Mead and Christensen’s preference for formal interpretations, the rate dropped significantly below 50% for informal interpretations (44.8%). (Nonetheless, although the Court had indicated in those cases that Chevron’s application would be rare for informal interpretations, the circuit courts apply Chevron nearly half the time.) Thus, formal interpretations (all rulemakings and formal adjudication) obtained the Chevron framework more frequently than informal interpretations. But notice-and-comment rulemaking obtained the Chevron framework fifteen percentage points more frequently than formal adjudication, despite their doctrinal parity under Mead.

Again, however, if the 386 immigration adjudications were removed from the formal adjudication category,²⁰⁰ the frequency of applying Chevron deference to formal adjudications would rise nearly ten percentage points to 85.2% and bring the formal formats into closer parity. The difference between immigration decisions and other agencies’ formal adjudication likely arises in part because many immigration decisions are affirmed by only one Board of Immigration Appeals member (instead of the entire board), a procedure for which most circuit courts refuse to give Chevron deference.²⁰¹ By excluding immigration interpretations for which the BIA has an idiosyncratic review process, the Chevron-application differential between informal rulemaking and formal adjudication significantly shrinks to fewer than seven percentage points.

***BARNETT & WALKER FIGURE 5 HERE***

That Chevron deference applies almost as often to agency statutory interpretations promulgated in formal adjudication (when excluding immigration proceedings) as in notice-and-comment rulemaking may have important implications for administrative law. The Court has repeatedly held, most notably in SEC v. Chenery, that agencies have extremely broad discretion to choose whether to engage in rulemaking or adjudication.²⁰² In deciding whether to use rulemaking or adjudication, the agency may consider various factors, such as the benefits of case-by-case development, the novelty of the issue, or time constraints in fashioning an interpretation.²⁰³ But agencies may also ponder whether they will pay a price in using formal adjudication instead of rulemaking. Our findings indicate that they may pay only a slight price—with a slightly lower rate of obtaining Chevron—for choosing formal adjudication (at least without idiosyncratic procedures).

But this small price seems worth it when one considers the better agency-win rates for formal adjudication once the circuit courts apply Chevron. As detailed in Figure 6, formal-adjudication win rates increased to 81.7% when Chevron applied, compared to a 74.7% overall win rate for formal adjudications and a 51.9% formal-adjudication win rate when Chevron did not apply. Indeed, if the 281 immigration adjudications to which Chevron applied were excluded, the win rate would rise to 86.0% (compared to 81.3% under all standards of review combined). Notably, Kerr found in his study of circuit courts in 1995 and 1996 that agency interpretations from adjudication (of all formality stripes) prevailed 72% of the time under Chevron, a rate of almost ten or fourteen percentage points below our findings.²⁰⁴ Conversely, our win rates for notice-and-comment rulemaking under Chevron and under all scopes of review were about the same: 74.4% under Chevron and 72.8% overall (though only 54.3% when Chevron did not apply). This win rate is similar to the 74% that Kerr found for prevailing rulemakings on direct review in his study.²⁰⁵

The difference in agency-win rates between formal adjudication (81.7% with all subject matter, or 86.0% without immigration) and notice-and-comment rulemaking (74.4%), seemingly absent in Kerr’s earlier findings, may cause agencies to consider adopting Chevron-eligible agency statutory interpretations in formal adjudication as opposed to the more time- and resource-intensive notice-and-comment rulemaking.²⁰⁶ As a caveat, our data do not take into account whether the agency is less or more aggressive in its interpretations depending on whether the agency uses formal adjudication or notice-and-comment rulemaking, although we have no reason to think that agency behavior differs. Because of the higher agency-win rate with formal adjudication, formal adjudication may well be the better option for agencies in more cases than agencies may first surmise.

***BARNETT & WALKER FIGURE 6 HERE***

Agency adjudication has had a rough go of it the past few decades. Rulemaking has increased in popularity as adjudication has come under fire for its comparative downsides²⁰⁷: it is less efficient because it does not address numerous issues at once, is less appropriate for determining “legislative” facts with input from numerous interpreted persons,²⁰⁸ provides case-by-case decisions that provide less prospective notice,²⁰⁹ relies on enforcement actions for compliance that may not be necessary for existing rules,²¹⁰ targets one regulated party as a test case and creates a retroactive norm,²¹¹ provides agencies less agenda control depending on how matters are docketed,²¹² and provides a less audible “fire-alarm” to enable congressional oversight.²¹³ Both formal and informal adjudication face existential attacks based on their fairness to regulated parties.²¹⁴

Nonetheless, adjudication has numerous benefits. Aside from those mentioned in Chenery II,²¹⁵ adjudication permits agencies to escape review by the White House’s Office of Information and Regulatory Affairs and thus obtain more independence,²¹⁶ conserves more onerous rulemaking resources by using adjudication, allows agencies to act in an incremental way with a light regulatory touch,²¹⁷ provides broad participation by interested parties,²¹⁸ permits retroactive standard setting when necessary,²¹⁹ avoids onerous congressionally imposed constraints on rulemaking,²²⁰ and has more flexibility than it is often given credit for.²²¹ And our data suggest that there is one more to add: better agency-win rates under Chevron.²²²

Although we do not enter the rulemaking–adjudication debate here, our findings that agencies prevailed more frequently under Chevron in adjudication than rulemaking may matter to agencies. One of us previously surveyed 128 agency rule drafters.²²³ Among more than twenty interpretive tools included in the survey, Chevron deference was reported by most agency rule drafters (90.0%) as being used when interpreting statutes and drafting regulations.²²⁴ The vast majority of agency rule drafters surveyed thinks about judicial review when interpreting statutes and views their chances of prevailing in court as better under Chevron. “Indeed, two in five rule drafters agreed or strongly agreed—and another two in five somewhat agreed—that a federal agency is more aggressive in its interpretive efforts if it is confident that Chevron deference (as opposed to Skidmore deference or de novo review) applies.”²²⁵ To be sure, one must be cautious in drawing strong inferences from these data because the “somewhat agree” (as opposed to agree or strongly agree) responses predominated and some volunteered comments discounted the effect of judicial review.²²⁶ Moreover, these drafters were not addressing whether the presence of agency-win rates would alter agency formats. But they are candid indications that agencies think about standards of review and that those standards may affect their statutory interpretations.

Finally, it is worth noting the differences in agency-win rates for informal interpretations. The overall agency-win rate for informal interpretations was 65.0%. But when the circuit courts applied Chevron, the win rate rose to 78.6%—near the rate for formal adjudication (81.7%) and slightly better than notice-and-comment rulemaking (74.4%). And, when the courts refused to apply Chevron deference to informal interpretations, the win rate dropped to 54.0%, which again is similar to the win rate without Chevron for notice-and-comment rulemaking (54.3%) and formal adjudication (51.9%). Although our findings here do not demonstrate what relationship the review standards and the agency-win rates have, these findings, along with our findings concerning the application of Chevron to informal interpretations, suggest that agencies should seek Chevron deference for every interpretation, regardless of its formality. Although Christensen and Mead both suggested that Chevron’s application to informal interpretations would be rare, our findings indicate that courts apply the Chevron framework, if not as a matter of course, almost half the time (44.8%) in decisions in which the Court referred to Chevron. This is especially true, as we shall see, when the litigation is in the D.C. Circuit.²²⁷ And when courts applied the framework, the agency-win rate was extremely high, higher than that for even notice-and-comment rulemaking. Again, however, our data do not allow us to account for whether the agency is less or more aggressive in its interpretive efforts depending on the level of formality involved in the regulatory effort.

IV. Findings on Circuit Disparities

As reported in Section III.A, our findings suggest that standards of review matter for agency statutory interpretations. Recall that the overall agency-win rate in the 1,558 interpretations—regardless of the deference standard applied—was 71.4%. Recall, too, that the agency-win rate increased to 77.4% for the 1,166 interpretations subject to the Chevron framework, which was significantly higher than the rate under Skidmore (56.0%) and de novo review (38.5%). These findings provide some support that Chevron deference matters in the federal circuit courts. In other words, there may well be a Chevron Supreme and a Chevron Regular. But disaggregating the data by circuit, as depicted in Figure 7, complicates this story.

***BARNETT & WALKER FIGURE 7 HERE***

As detailed in Figure 7, the overall agency-win rates varied significantly by circuit. The most deferential circuit was the First Circuit (82.8%), followed by the Tenth (78.5%) and Eleventh (75.5%) Circuits. The two circuits that specialize in administrative law—the D.C. (72.6%) and Federal (73.2%) Circuits—are right around the mean (71.4%) and median (72.2%). The least deferential was the Ninth Circuit (65.8%), followed by the Fifth (67.8%), Sixth (69.0%), and Third (69.9%) Circuits.²²⁸ These results may not be too surprising based on one’s intuitions about the circuits and their reputations vis-à-vis the federal government.

Perhaps some of the agency-win rates differ in the circuits based on subject matter, but subject matter or other effects require careful inquiry in future work. For instance, we intuitively—and it turns out correctly—thought that the Ninth Circuit’s large number of immigration cases likely affected the agency-win rate. Indeed, agencies prevailed in immigration cases 55.9% of the time in the Ninth Circuit, ten percentage points fewer than in all cases within the Ninth Circuit (65.8%). Put differently, when immigration cases are excluded, the agency-win rate in the Ninth Circuit rises to 73.8%, much more in line with the median and mean circuit. By contrast, the agency-win rates in immigration cases were significantly higher in other circuits that also had large number of immigration cases: 82.4% in the Fifth Circuit, 73.2% in the Second Circuit, and 70.9% in the Third Circuit. Notably, the Fifth Circuit’s overall agency-win rate (67.8%) was significantly lower than its agency-win rate in only immigration matters (82.4%). In the Second and Third Circuits, the overall and immigration-specific agency-win rates were nearly identical. Other factors, such as political valence and panel effects, appear to have more influence. The key point is that readers should keep in mind that more sophisticated analysis is necessary to understand why various circuit disparities exist.

Assessing the circuits based on the frequency at which they applied the Chevron framework paints a somewhat different picture, as depicted in Figure 8. As to the frequency of Chevron’s application, five circuits were well above the average (74.8%) and median circuit (73.2%). The D.C. Circuit led the way by applying the Chevron standard to 88.6% of interpretations, followed by the First (87.9%), Eighth (85.7%), Federal (84.6%), and Fourth (80.6%) Circuits. The Sixth Circuit, by contrast, applied Chevron the least frequently, only 60.7% of the time. Five other circuits were below 70%.

***BARNETT & WALKER FIGURE 8 HERE***

And inside these Chevron-application statistics is another fascinating finding. The “Mead Puzzle” arises from the difficulty lower courts have had in determining whether informal interpretations have the force of law and thus are entitled to Chevron deference.²²⁹ All but two circuits refused to apply Chevron to informal interpretations (at least when the court referred to Chevron) more than 50% of the time.²³⁰ And the median circuit rate was 36.8%.

But the D.C. Circuit, a Chevron early adopter, applied the Chevron framework to informal interpretations 80.7% of the time, nearly twenty-five percentage points more often than the next circuit (the Eighth Circuit, at 57.1%), more than forty percentage points more than the median circuit, and approximately sixty-five percentage points more than the circuit least likely to apply Chevron in these cases (the Second Circuit, at 16.2%). Accordingly, the circuit that reviewed the most agency interpretations in our dataset does not appear to have found the “Mead Puzzle” enigmatic. This finding bolsters our early conclusion that agencies should seek Chevron deference even for informal interpretations;²³¹ not doing so in the D.C. Circuit borders on malpractice.

But to appreciate the circuit-by-circuit effect of Chevron deference (regardless of an interpretation’s formality), one needs to compare the agency’s win rate overall with its win rate when courts applied the Chevron framework. The average win-rate difference for the dataset is six percentage points, with an overall win rate of 71.4% compared to a win rate of 77.4% when the court applied the Chevron deference framework.

Several circuits were dramatic outliers with respect to win-rate differential. The greatest difference came from the Sixth Circuit, where the overall win rate was 69.0%, whereas the win rate when Chevron applied was 88.2%—nearly twenty percentage points higher. Agency-win rates in the Second (72.5% to 83.2%) and Seventh (72.0% to 83.7%) Circuits were also more than ten percentage points higher when Chevron applied. In those circuits, it was harder to obtain Chevron deference, but, once obtained, the agency’s chances of winning improved considerably. Conversely, the win-rate differential was within three percentage points in six of the thirteen circuits: the First (82.8% to 84.3%), Eighth (75.5% to 76.2%), Tenth (78.5% to 81.3%), Eleventh (70.4% to 73.1%), D.C. (72.6% to 75.4%), and Federal (73.2% to 76.0%) Circuits. In other words, in those circuits, whether Chevron applies does not seem to meaningfully affect agency-win rates.

Of course, that may not be the correct inference from these data. For many of these circuits where there was little difference in win rate, that is because the court applied Chevron deference at such a high rate that the Chevron win rate and overall win rate were basically the same. Such a win rate could be the result of those circuits having imbued Chevron’s deference principles into judicial decisionmaking. Figure 9 attempts to tease out those nuances by comparing the win rate under Chevron versus the win rate when Chevron does not apply. The circuits are ordered left to right in Figure 9 starting with the circuits with the greatest difference with and without Chevron. The overall win rate when Chevron applied was 77.4%, as noted above. When Chevron did not apply, however, the win rate plummeted nearly twenty-five percentage points to 53.6%.

For the six circuits whose differential between win rates overall and win rates under Chevron were within three percentage points (First, Eighth, Tenth, Eleventh, D.C., and Federal Circuits), the numbers are quite different when comparing win rates with and without Chevron’s application. The differential in the D.C. Circuit, for instance, was over twenty percentage points (75.4% to 51.4%). Of all thirteen circuits, the Eighth Circuit (76.2% to 71.4%) was the only outlier with a difference of less than five percentage points, with the only other under ten percentage points being the Eleventh Circuit—just barely (73.1% to 63.2%). And, before leaving Figure 9, we note that the largest differences were striking: 48.8 percentage points in the Sixth Circuit, 36.5 points in the Fourth Circuit, 33.7 points in the Seventh Circuit, and 31.5 points for the Second Circuit.

***BARNETT & WALKER FIGURE 9 HERE***

To compare circuits, it is perhaps helpful to create a composite score of the three indicators of deference in our dataset: overall agency-win rate; frequency of Chevron framework; and win rate when Chevron applied. Table 1 takes the average of these three percentages and turns that into a composite score on a ten-point scale—with 10.00 being a perfectly deferential score where the agency always wins and the court always applies Chevron deference and 0.00 being a perfectly nondeferential score where the agency never wins and the court never applies Chevron. The circuit rankings for each of the three deference indicators are provided in parentheses.

Utilizing these composite scores, the First Circuit (8.38 out of 10.00) emerges as the most deferential circuit, followed by the Eighth (7.91), D.C. (7.89), Federal (7.79), and Fourth (7.74) Circuits. On the other end of the spectrum, the Ninth and Fifth Circuits (6.85) tie as the least deferential circuits, followed by the Third (7.21) and Eleventh (7.22) Circuits.

Prior studies have not considered similar circuit effects. But these effects can be meaningful for agencies and litigating parties. For instance, if an agency seeks to bring an enforcement action to test one of its statutory interpretations, the Ninth Circuit may not be the best place to do so. Moreover, if the agency is worried about receiving Chevron deference, the Eighth Circuit is a promising venue because agency-win rates are similar with or without Chevron. Or, if the agency is confident that it will receive Chevron deference, the Sixth Circuit appears promising because agencies have the highest win rates under Chevron in that circuit. Of course, these strategic decisions should involve more considerations, including the ideological valence of the advanced interpretation, the subject matter, the agency, and the regulated parties at issue. We investigate some of these findings in the next Part.

***BARNETT & WALKER TABLE 1 HERE***

V. Findings on Agency and Subject-Matter Differences

Just as the circuit disparities discussed in Part IV complicate the story regarding Chevron in the circuit courts, so do the differences uncovered in Chevron’s application by subject matter and agency. This Part turns to those findings. Section V.A explores differences based on subject matter, whereas Section V.B looks at agency-by-agency disparities. Section V.C focuses on the differences between executive and independent agencies. Our purpose here is to provide an overview of the data and brief discussions of the most noticeable findings. We leave for future work (whether ours or others’) to dive into the findings for individual subject matters and agencies.

A. Subject-Matter Differences

Differences in agency-deference rates based on subject matter are particularly interesting in light of the extensive focus of late in the literature and the real world on administrative law exceptionalism—“the misperception that a particular regulatory field is so different from the rest of the regulatory state that general administrative law principles do not apply.”²³²

To better appreciate differences based on subject matter, Table 2 presents the composite scores for all subject matters where there were at least ten agency interpretations in the dataset. This composite score is based on the same methodology used as that for ranking circuits in Part IV, with the three indicators of deference in our dataset weighted equally: overall win rate; frequency of Chevron framework; and win rate when Chevron applied. Table 2 takes the average of these three percentages and turns that into a composite score on a ten-point scale—with 10.00 being a perfectly deferential score where the agency always wins and the court always applies Chevron deference and 0.00 being a perfectly nondeferential score where the agency never wins and the court never applies the Chevron standard. The rankings for each indicator are provided in parentheses.

***BARNETT & WALKER TABLE 2 HERE***

As Table 2 indicates, the subject matters for which courts defer most often to agency interpretations included telecommunications (8.67), Indian affairs (8.33), federal government (8.18), pensions (8.17), education (8.15), health and safety (8.14), and entitlement programs (8.03). Conversely, the subject matters for which courts defer the least were civil rights (5.99), followed by housing (6.04), prisons (6.64), tax (6.74), and employment (6.96).

The findings depicted in Table 2 merit article-length treatment and further exploration. But this Section merely highlights a few noteworthy findings. For instance, it perhaps should come as no surprise that tax ranks nineteen out of twenty-two in light of entrenched tax exceptionalism. At 56.3%, moreover, tax was second to last in the rate at which circuit courts applied Chevron deference. Indeed, it was not until 2011 that the Supreme Court announced that certain IRS interpretations are entitled to Chevron deference²³³—a position that the Supreme Court may have qualified last year in King v. Burwell, at least with respect to questions of deep political or economic significance.²³⁴ Similarly, it is perhaps no surprise to see immigration in the latter half of the rankings in light of the current discussion regarding immigration exceptionalism.²³⁵

The range of circuit-court deference by subject matter is also worth underscoring. For instance, the overall agency-win rate ranged from 86.7% for Indian affairs to 50.0% for civil rights.²³⁶ The rate of circuit courts applying Chevron deference ranged from 100.0% for federal government matters to 52.0% for housing. Similarly, when circuit courts applied the Chevron deference framework, agency-win rates ranged from 92.3% for agency interpretations involving pensions (as opposed to an overall win rate of just 76.5% for pensions) to 57.1% for prisons. (Strangely, agencies were more successful when Chevron deference did not apply in the prison context, with an overall 68.4% win rate.) These disparities based on subject matter complicate the findings discussed in Part III regarding how Chevron matters on the ground in the circuit courts.

The data reveal some differences and similarities between the treatment of various subject matters in the Supreme Court and the circuit courts. As for the key differences, Eskridge and Baer reported that interpretations concerning energy had the highest overall agency-win rate of 93.3%,²³⁷ but in the circuit courts the agency prevailed 60.0% of the time, rendering it one of the lowest-prevailing agencies. On the flip side, Indian affairs had the second lowest overall agency-win rate of 51.6% in the Supreme Court,²³⁸ but it had the highest rate of 86.7% in the circuit courts. At the same time, some subject matters performed about the same in the Supreme Court and the circuit courts: environmental (68.4% and 70.8%, respectively), immigration (67.7% and 67.9%), business regulation (77.1% and 79.2%), and transportation (78.6% and 79.1%).²³⁹

Contrary to Eskridge and Baer’s conclusion as to the Supreme Court, we cannot conclude with much confidence that the circuit courts defer based on notions of their perceived institutional advantage or disadvantage. Eskridge and Baer divided subject matter with at least ten decisions in the Supreme Court into six categories (foreign affairs, technical and economic regulation, procedural rules, socioeconomic regulation, criminal law, and federal governance) and indicated which subject matters within each category performed better and worse than the overall average.²⁴⁰ They determined that the Supreme Court was generally more deferential to foreign affairs (despite being less deferential to immigration, one of only two subject matters in the foreign affairs category, than the overall average), technical and economic matters, and procedural rules; the Supreme Court was generally less deferential to subject matter in criminal, socioeconomic, and federal governmental matters. Eskridge and Baer concluded that the justices were less likely to defer to matters that were not viewed as technical and matters that the Court thought that it could just as easily answer.²⁴¹

Our dataset does not tell the same story with as much certainty when we organize our data similarly to theirs. While the circuit courts’ treatment of foreign affairs and technical matters are similar to the Supreme Court’s (but less deferential in each category), the circuit courts do not appear to defer meaningfully more or less to social-economic regulations, and, in fact, they defer much more to issues concerning the federal government than the Supreme Court. Moreover, although the circuit courts defer less to criminal matters than the overall average, the agency-win rates in this category (70.0% for criminal law and 68.4% for prisons) are very close to the average (71.4%) and higher than the agency-win rate in the Supreme Court (62.3% for criminal law, the only category there).²⁴²

B. Agency-Deference Rankings

Although there is an obvious overlap between analyzing deference based on subject matter and agency, disaggregating the deference rankings by agency provides some helpful additional granularity. Table 3 presents the composite scores for all agencies with at least ten interpretations in the dataset. This composite score is based on the same methodology used as that for ranking circuits in Part IV and subject matters in Section V.A, with the three indicators of deference in our dataset weighted equally: overall win rate, frequency of Chevron framework, and win rate when Chevron applied. Table 3 takes the average of these three percentages and turns that into a composite score on a ten-point scale. The rankings for each of the three deference indicators are provided in parentheses.

Similar to subject matters, Table 3 illustrates the disparate win rates by agency. The ICC/STB was the agency to which the circuit courts most deferred (9.38), followed by the FCC (8.67), Treasury Department (8.37), NLRB (8.26), Commerce Department (8.18), Defense Department/Armed Services (8.13), FDA (8.08), and Education Department (8.06). On the other end, the least-deferred-to agency was the EEOC (5.08), followed by HUD (5.19), Energy Department (6.21), FTC (6.74), Justice Department (6.77), IRS (6.78), and Bureau of Prisons (6.79).

***BARNETT & WALKER TABLE 3 HERE***

We underscore that these agency composite deference scores have the potential to mask some of the underlying deference differentials. For instance, the FTC is the fourth-worst agency on the composite score despite having the second- highest overall win rate (90.9%). That is because circuit courts applied the Chevron doctrine to only 36.4% of the FTC’s statutory interpretations; indeed, the FTC’s win rate fell to 75.0% when the circuit courts applied Chevron.²⁴³ (The agency-win rate similarly fell for the ITC [72.7% to 70.0%] and Bureau of Prisons [73.7% to 61.5%] when Chevron was applied.)

It is likewise interesting to evaluate the stark disparities between agencies dealing with similar subject matters. For instance, the Energy Department (6.21) was the third worst among the twenty-eight agencies, whereas the EPA (7.49) ranked thirteenth overall. Although the frequency of Chevron being applied was somewhat similar (89.3% for the EPA to 90.9% for Energy), there was a difference of more than twenty percentage points in overall agency-win rates (67.9% to 45.5%) and in agency-win rates when Chevron applied (67.6% to 50.0%). Meanwhile a third energy-related agency, FERC, always received Chevron deference, but only prevailed 60.5% of the time—for a composite score (7.37) ranking of sixteen.

Likewise, the Treasury Department (8.37) ranked third overall on the composite scale, whereas the IRS (6.78), an agency within Treasury,²⁴⁴ ranked twenty-third out of the twenty-eight agencies. This divergence further illustrates the tax-exceptionalism phenomenon discussed in Section V.A. By contrast, the Department of Health and Human Services (HHS) (7.89) and the FDA (8.08), an agency within HHS, were basically ranked the same.

Consider, too, the stark differences in agencies dealing with labor and employment: the NLRB (8.26) surprisingly ranked fourth overall, whereas the EEOC (5.08) came in last place of twenty-eight agencies, and the Labor Department (7.14) was nineteenth. In terms of overall agency-win rates in the circuit courts, the NRLB prevailed 78.1% of the time, compared to 70.4% for the Labor Department and 42.9% for the EEOC. These circuit-court findings differ substantially from findings on the Supreme Court where, in one study, agency-win rates post-Chevron were virtually the same for the EEOC (51.9%) and NLRB (52.4%) and somewhat better for the Labor Department (66.7%).²⁴⁵

Again, it is worth noting the wide range of circuit-court deference by agency, which is similar to the range for subject matter. For instance, the overall agency-win rate ranged from 100.0% for the ICC/STB to 45.5% for the Energy Department and 42.9% for the EEOC.²⁴⁶ The rate of circuit courts applying Chevron ranged from 100.0% for FERC to 36.4% for the FTC and 41.7% for HUD. Similarly, when circuit courts applied Chevron, agency-win rates ranged from 100% for the ICC/STB and 93.3% for the Treasury Department to 50.0% for the Energy Department. Like the disparities based on subject matter, these agency-by-agency differences complicate the findings discussed in Part III regarding how Chevron matters in the circuit courts.

C. Executive Versus Independent Agencies

One may also wonder how circuit courts treat executive and independent agencies differently. After all, the Chevron Court itself emphasized how political accountability may be a justification for deferring to agency statutory interpretations.²⁴⁷ Figure 10 separates out the key findings as to executive and independent agencies.

Of the 1,558 interpretations in our dataset, 1,284 interpretations (82.4%) were made by executive agencies, whereas the remaining 274 interpretations were made by independent agencies.²⁴⁸ Perhaps surprisingly, the composite deference score for independent agencies (7.97) was higher than for executive agencies (7.34). Indeed, independent agencies were more successful, to varying degrees, as to all three indicators of deference: the overall agency-win rate (77.0% to 70.2%); the rate of circuit courts applying Chevron deference (82.5% to 73.2%); and the agency-win rate with Chevron deference (79.6% to 76.8%).

***BARNETT & WALKER FIGURE 10 HERE***

Again, one should be cautious inferring causation here. Especially in light of the nearly ten-percentage-point difference in Chevron deference being applied, one may be tempted to declare dead the political-accountability theory for Chevron deference. Indeed, higher agency-win rates and significantly more Chevron applications are seemingly contrary to one scholar’s view that independent agencies should receive less deference because they lack the same political accountability as executive agencies.²⁴⁹ But there may well be other explanations. Independent agencies may be more cautious in seeking Chevron deference, and they may also be less aggressive in their interpretive efforts due to their independence from the President. The stark difference in agency-win rates (64.6% for independent agencies to 52.0% for executive agencies) when the circuit courts refused to apply the Chevron framework may support the theory that independent agencies are less aggressive.

VI. Additional Findings: What Else Matters?

In this final Part, we consider some additional findings that are especially relevant to ascertain whether the circuit courts have internalized certain, often vague, nudges from the Supreme Court, especially when the Court’s practice is to the contrary. We begin by looking at how the circuit courts approach two sensitive subjects in Section IV.A, move in Section IV.B to whether stable interpretations fare better than inconsistent ones (despite similar doctrinal treatment), and conclude in Section IV.C by evaluating the salience of certain traditional deference factors in the courts of appeals.

A. Sensitive Matters

As discussed in Section I.B.2, certain sensitive subjects—such as regulatory jurisdiction, state-law preemption, and significant political or economic questions—have created wrinkles, at one time or another, in the Supreme Court’s deference jurisprudence. Because the Court did not clearly identify significant questions as relevant to all Chevron step-zero inquiries until 2015,²⁵⁰ very recently and well after our selected timeframe, we cannot say what impact that decision has in the circuit courts. But our data can provide insight as to regulatory jurisdiction and state-law preemption. Figure 11 compares the overall agency-win rate with the win rates for jurisdictional and preemption interpretations (with the frequency of Chevron’s application for each also depicted).

***BARNETT & WALKER FIGURE 11 HERE***

The Court clarified in May 2013 in City of Arlington v. FCC that Chevron applied to regulatory-jurisdiction questions largely because of the difficulty of distinguishing run-of-the-mill interpretation questions from so-called jurisdictional ones.²⁵¹ Before that ruling, however, Eskridge and Baer had found that the Court applied Chevron to regulatory-jurisdiction questions only 34.4% of the time.²⁵² The circuit courts, however, appeared to do a better job of anticipating City of Arlington. Interpretations concerning regulatory jurisdiction made up 105 out of our 1,558 interpretations (6.7%).²⁵³ Of those 105, the circuit courts applied Chevron deference to 78 of them (74.3%).²⁵⁴ Notably, this Chevron-application rate to regulatory-jurisdiction interpretations (74.3%) was basically the same for all interpretations (74.8%).

Although we did not directly code for major questions, we can get a sense of how the courts responded to Oregon v. Gonzales’s exception that declines to apply Chevron to changed agency positions as to major questions.²⁵⁵ To do so, we can parse the regulatory-jurisdiction interpretations further by considering the frequency to which the circuit courts applied Chevron to agency interpretations concerning their jurisdictional or regulatory authority that replaced a prior, inconsistent interpretation (what we refer to as “evolving interpretations”). Of the 19 evolving interpretations that concerned regulatory jurisdiction, the circuit courts applied the Chevron framework 17 times (89.4%). This application rate was significantly higher than the average Chevron-application rates for all regulatory-jurisdiction interpretations (74.3%) and all interpretations, regardless of type, combined (74.8%). Despite these small numbers and the limited inferences that we can draw from them, this finding suggests courts have not internalized Gonzales’s step-zero exception.

Nevertheless, agency-win rates suggest that the circuit courts may be slightly uncomfortable deferring to agencies on these seemingly more significant matters. Under any deference regime, agencies prevailed on regulatory-jurisdiction matters only 63.8% of the time (67 of 105 interpretations). That win rate is somewhat lower than the overall agency-win rate of 71.4%. Similarly, despite receiving Chevron deference at basically the same rate as normal, agencies’ regulatory-jurisdiction interpretations prevailed 70.5% of the time, a lower win rate than that of 77.4% for all Chevron applications. (That said, Chevron still mattered, as agencies prevailed on regulatory-interpretations 70.5% of time with Chevron, and only 44.4% of time without it.) In the 17 of 19 instances when Chevron applied to an agency’s evolving regulatory-jurisdiction interpretation (similar to the Gonzales issue), the agency-win rate was 63.2% (12 wins). These slightly lower rates perhaps arise from the general significance of agency decisions or aggressive agency interpretations to expand their dominion.

As for state-law preemption, the doctrinal and scholarly dispute concerning the suitability of Chevron deference to state-law preemption may not be significantly meaningful to agencies. We uncovered only 25 interpretations concerning preemption in our dataset. Of those, the agency prevailed 80.0% of the time (20 of 25 cases). The agencies always prevailed when the court applied no deference or did not indicate whether deference applied, although there were only three of these decisions. The courts applied Chevron to 76.0% of the interpretations (19 of 25), and agencies prevailed 78.9% of the time under Chevron, meaning that the Chevron-application and agency-win rates were approximately the same as our database averages for both variables in all interpretations (74.8% application rate and 77.4% agency-win rate). This win rate under Chevron of 78.9% is near the agency-win rate for preemption questions when the circuit courts did not apply Chevron (83.3%).

Despite the scholarly call for Skidmore deference to apply to state-law preemption (from one of us and others)²⁵⁶ and the finding (from a study by the other of us) that a majority of 128 agency rule drafters surveyed indicated that Congress does not delegate preemption matters to agencies,²⁵⁷ the Skidmore-application rate is 12.0% (3 of 25), roughly the same rate for our entire database (10.8%). Indeed, only one of those applications involved an agency rulemaking, where Chevron would be more likely to apply under Mead.²⁵⁸ Based on this small number of Skidmore decisions, the agency-win rate is more than ten percentage points greater than the database average for Skidmore (66.7% to 56.0%). The agency-win differential between Chevron and Skidmore deference, therefore, decreases from more than twenty percentage points for all relevant interpretations in our database to about twelve points for preemption-related interpretations. Again, however, we are dealing with small numbers.

Regulatory jurisdiction and state-law preemption together provide findings concerning two sensitive matters. These findings suggest that the circuit courts have not internalized the Supreme Court’s often vague and conflicting signals over limiting Chevron’s application to certain matters because they applied Chevron at higher rates to these matters than to all matters combined. But the lower agency-win rates under Chevron for regulatory jurisdiction suggest that agencies may account for judicial unease as part of their overall judicial review. All of this said, our findings do not allow us to make any definite conclusions based on the nature of the Court’s unclear directives, the relatively small number of decisions that arise in the circuit courts on these matters, the limited questions that we coded, and the inherent limitations in our coding methodology that cannot account for ad hoc concerns in the opinions or concerns that the courts did not express.

B. Interpretive Continuity

Interpretive continuity has a complex role in deference doctrines and judicial interpretation generally.²⁵⁹ Interpretive continuity is relevant to whether agencies receive Skidmore deference,²⁶⁰ but Chevron itself stated that such continuity is not germane to Chevron deference.²⁶¹ Nevertheless, both before and after Chevron, the Court identified its presence at times as a factor to consider when reviewing an agency’s interpretation.²⁶² Eskridge and Baer found that, despite the Court’s tendency not to apply Chevron where it would appear to apply,²⁶³ “the overwhelming majority of the cases in which the Court invokes Chevron (70.6%) involve a long-standing or fairly stable interpretation. Indeed[,] this category dwarfs applications of Chevron where the agency interpretation is recent (27.1%) or evolving (2.4%),”²⁶⁴ suggesting that the Court does not follow its own pronouncements as to Chevron’s applicability. Long-standing interpretations had an overall success rate under any deference regime of 73.2%, while recent and evolving interpretations had lower win rates of 66.9% and 60.5%, respectively, in the Supreme Court.²⁶⁵ We sought to determine how long-standing and newer interpretations fared in the circuit courts.

Based on information that we could glean from the opinion itself, we coded the duration of interpretations as long-standing, evolving (meaning that one interpretation replaced a prior one), recent (meaning that a new interpretation did not replace a prior one), and unclear. Our coding was similar to Eskridge and Baer’s, except that we added an “unclear” category. We coded interpretations where the court made some reference to the stability or date of the agency interpretation, while we coded those for which we could not discern the longevity from the decision as “unclear.” We had a fairly even sample of interpretations of long-standing, recent, and unclear vintage. Approximately one-third of our interpretations were long-standing (34.5%), one-third were of unknown duration (35.0%), and one-third were either recent or evolving (30.5%).

Our data indicate that long-standing interpretations prevailed more frequently than other interpretations. Of all long-standing interpretations regardless of deference regime, agencies prevailed 82.3% of the time—far ahead of ones that were evolving (59.8%), recent (65.9%), or of unknown duration (67.8%). As compared to Eskridge and Baer’s findings, the long-standing interpretations fared even better in the circuit courts (about nine percentage points better), while the recent and evolving interpretations fared about the same (both only one percentage point worse).²⁶⁶ That said, if we combine all interpretations under any deference regime for long-standing interpretations and those whose duration is unclear (1,086 interpretations), as it appears that Eskridge and Baer did, the agency-win rate (813 wins out of 1,084 interpretations) falls to 75.0%, almost the same as theirs for long-standing interpretations. Figure 12 summarizes these comparisons, with the unknown category in our study broken out separately.

***BARNETT & WALKER FIGURE 12 HERE***

When it came to applying the Chevron framework, however, circuit courts were not more likely to apply Chevron to long-standing interpretations than other interpretations. Courts applied Chevron to 76.2% of long-standing interpretations (410 out of 538) and roughly the same frequency to recent interpretations (76.1%, or 194 of 255 interpretations). The surprise came with evolving interpretations. Circuit courts applied Chevron even more frequently to them (86.3%, or 189 of 219 interpretations). When the interpretation was unclear, courts applied Chevron 68.3% of the time (373 of 546 interpretations). When we parsed the data further to see whether courts applied Chevron at different rates for long-standing versus new or evolving interpretations that were presumptively Chevron-eligible (meaning those from formal rulemaking or adjudication or notice-and-comment rulemaking), the disparity disappeared. Circuit courts applied the Chevron framework to 88.1% of long-standing formal interpretations and 87.7% of evolving or recent formal interpretations (92.0% and 83.3%, respectively).²⁶⁷

Once Chevron applied, long-standing agency interpretations triumphed again, especially over evolving ones. Long-standing interpretations prevailed 87.6% of the time. Interpretations of recent or unclear vintage were affirmed at lower rates of 74.7% and 73.5%, respectively. Evolving interpretations, the interpretations most likely to have Chevron apply, had the lowest agency-win rate of 65.6%.²⁶⁸ Despite having the lowest agency-win rate under Chevron, this win rate for evolving interpretations was actually its highest by a significant margin under any of the deference regimes. Evolving interpretations have the lowest win rate under every deference regime except de novo review, often by wide margins. For instance, they had a 0.0% win rate in the three instances when the courts identified no deference regime, in comparison to a win rate of 72.7% for long-standing interpretations. Likewise, evolving agency interpretations prevailed only 30.8% of the time under de novo review (13 instances), with only recent interpretations doing more poorly with a win rate of 16.7% (24 instances). Under Skidmore, evolving interpretations prevailed only 21.4% of the time (14 instances), while recent ones prevailed 46.2% of the time (26 instances) and long-standing ones 67.6% of the time (71 instances). The findings are fully presented in Figure 13.

Based on our coding, we can further mine the data on recent and evolving interpretations. When courts reviewed evolving or recent interpretations under Chevron, certain of those interpretations did significantly better than others. Of the 383 recent or evolving interpretations to which courts applied Chevron, they arose in response to new or amended statutes (98), agencies facing new issues (95 interpretations), changed facts or judicial decisions (91), the agency’s practical experience (67), new presidential administrations (8), reevaluated litigating positions (3), or in response to a judicial decision (1)—with the remainder for unclear reasons (20). Agency interpretations in the four largest categories all prevailed under Chevron at relatively consistent and high rates: from a high of 73.1% and 72.5% for practical experience and changed circumstances, respectively, to 70.4% and 69.5% for new statutory provisions and new issues, respectively. A sharp drop occurred when the reasons weren’t clear (60.0%, based on 20 interpretations) or the changed interpretation came from a new administration (50.0%, based on 8 interpretations).

***BARNETT & WALKER FIGURE 13 HERE***

What to make of this continuity data?

First, the findings suggest that the circuit courts have followed Chevron’s command that Chevron applies with equal force to all agency positions, whether they are changed, new, or long-standing. The circuit courts’ Chevron-application rate was similar for recent and long-standing interpretations, and the rate even increased for evolving interpretations, perhaps because the government went out of its way to point out Chevron’s command on the duration issue.²⁶⁹ Moreover, when we filtered the data further to compare long-standing with new or evolving interpretations that were presumptively Chevron-eligible, the courts applied Chevron at almost the same rate (88.1% and 87.7%, respectively).

Second, once Chevron applied, interpretive duration seems to matter, although the nature of that relationship is unclear. Long-standing interpretations prevailed 87.6% of the time, approximately thirteen and fourteen percentage points more often than new interpretations and those of unclear duration, respectively, and twenty-two percentage points more often than evolving interpretations. Accounting for an interpretation’s longevity in the deference process, despite seeming contrary to Chevron itself, would be consistent with courts thinking of deference on a sliding scale, as Justice Breyer has long advocated, perhaps most successfully in Barnhart. And it would be consistent with the Court’s recent invocation of interpretive duration when it blessed a Patent and Trademark Office rule under Chevron step two.²⁷⁰ But it may also be that long-standing interpretations are more likely to be better thought-out and less aggressive than more recent, especially changed, ones.

The noticeable lack of agency success when a new administration simply changes the interpretation might suggest that circuit courts have not fully embraced the political-accountability theory that undergirds Chevron. Chevron recognized that the political branches had more accountability than unelected judges and were in a better position to make policy choices inherent in interpretive issues.²⁷¹ Indeed, the Chevron Court deferred to the Reagan Administration’s interpretation, despite the fact that the Carter Administration had interpreted the term at issue differently.²⁷² Or it could show judicial discomfort with APA arbitrary-and-capricious review, which some decisions have folded into Chevron step two (as opposed to treating it as a distinct step).²⁷³ In Motor Vehicles Manufacturers Ass’n v. State Farm Mutual Automobile Insurance Co., Justice Rehnquist’s partial dissent, joined by three other justices, blessed an agency’s reasonable reappraisal of costs and benefits in light of a new administration,²⁷⁴ but the majority’s silence on this point and preference for technocratic analysis has been understood to mean that changes based on political forces are improper.²⁷⁵ Ultimately, however, the small number of interpretations limits the inferences that one can draw from them. Indeed, that only 9 of the 474 total recent or evolving interpretations expressly implicated change in administration may reflect agencies’ strategic decisionmaking to avoid justifying a new or different interpretation on political grounds. And, notably, these data do not tell us when courts are expressly referring to an interpretation’s duration as part of their analysis. We discuss the invocation of factors, including duration, in Section VI.C.

Third, Skidmore seems to be working much as expected. Interpretive consistency is a germane factor under the doctrine that favors an agency’s position. Long-standing interpretations prevailed more frequently (67.6%) than others, indeed at a rate above the average rate for all Skidmore decisions (56.0%). The other interpretations’ agency-win rates were below the average, as one would expect: 21.4% for evolving ones, 46.2% for new ones, and 54.4% for ones of unclear duration. It makes sense that if consistency were the concern, new decisions would not evidence inconsistency (because there is no prior interpretation with which to be inconsistent) and thus should prevail more frequently than evolving ones that do, even if at a lesser rate than long-standing, consistent ones.

Finally, agencies seeking to issue evolving interpretations should be mindful of how they do so. Although agency-win rates were at their nadir for those interpretations under nearly every deference regime, agencies seem to be able to significantly improve their win rates by providing the interpretations with the force of law to render it more likely that they obtain Chevron deference, under which evolving interpretations prevailed 65.6% of the time. When agencies use less-formal means, courts are much less likely to apply the Chevron framework—only 59.0% for informal evolving interpretations but 92.0% for formal ones. With Chevron, the agency-win rate was 65.6%, but it plummeted to 30.8% with de novo review. And they plummeted forty-four percentage points from the 65.6% win rate under Chevron to the 21.4% win rate under Skidmore. Moreover, even with Chevron deference, agencies should carefully consider the reasons for the change. Changes based on differing political administrations or unclear changes suffered significantly lower win rates. When changing interpretations, agencies will likely place themselves on better footing by clearly pointing to changed facts and their experience to support the change.

C. Traditional Deference Factors or Theoretical Grounds

Before Chevron, the courts evaluated various factors in an ad hoc manner to determine whether to defer to agency interpretations. These factors included whether the matter fell within the agency’s expertise, its careful consideration over a long period of time, congressional delegation, its contemporaneity with the statute’s enactment, or vague notions of congressional ratification.²⁷⁶ Although Chevron and Mead had suggested that some of these factors were more important than others (delegation) or no longer relevant (consistency) when deciding whether Chevron applied, the Court’s dicta in Barnhart referred to more than delegation and force-of-law authority. It invoked some of these traditional factors—agency expertise, congressional acquiescence, and the agency’s careful consideration over a long period of time—and some additional ones concerning the nature of the legal question and the complexity of the statute.²⁷⁷

To get a sense of these factors’ relevance in the circuit courts, we followed Eskridge and Baer’s coding, where they added three theoretical factors and combined some of the contextual factors: agency expertise, accountability, national standard, long-standing interpretation, contemporaneity, public reliance, rulemaking authority, agency procedures, and congressional acquiescence.²⁷⁸ Like Eskridge and Baer, we coded each variable if the circuit court expressly referred to one of them in its opinion, whether specifically in the step-zero context or as part of its analysis of the interpretation itself. Similarly, we coded these factors whether courts noted their presence or absence; the findings reported in this Section do not disaggregate them. We found that only four of these factors had even an arguably regular place in circuit courts’ deference discourse under any regime. Figure 14 depicts these findings.

***BARNETT & WALKER FIGURE 14 HERE***

The most-invoked factors were not surprising: agency procedures utilized (25.7% of the time), rulemaking authority (18.3%), agency expertise (18.4%), and interpretive stability (10.7%). One would have expected, if anything, the first two to figure more prominently because they are the two factors that relate most closely to Mead’s delegation inquiry and concern for formality.

Expertise’s limited prominence in the dataset was also contrary to expectations. It is one of the relevant factors for Skidmore deference, and it is likely to come up as part of an inquiry into whether Congress intended to delegate certain issues. But once again, if there is any surprise here, it is that expertise played such a small role in our Skidmore interpretations. Courts referred to expertise in 42.9% of the 168 Skidmore decisions, 15.3% of the 1,166 Chevron decisions, and 24.8% of the 117 de novo decisions. Despite its serving as the theoretical basis for the Skidmore doctrine and its relevance to the agency’s reasoning and consideration,²⁷⁹ it was invoked less than half the time for interpretations to which the Skidmore framework applied.

And so the story goes for interpretive stability. The courts referred to the duration of an interpretation in only 10.7% of all their discussions and only 8.9% of interpretations where Chevron applied. These numbers are smaller than expected, considering that courts agreed with long-standing agency interpretations at higher rates regardless of deference regime as well as under Chevron. Courts were, as with agency expertise, more likely to refer to this factor when they applied Skidmore, the regime under which consistency is a factor. They referred to it in 23.8% of all 168 Skidmore decisions. But because it is a Skidmore factor, one would have expected it, as well, to be referred to more frequently than only about a quarter of the time. The circuit courts’ ambivalence in expressing its thoughts on the long-standing nature of the agency statutory interpretation—no matter its actual impact on decisionmaking—ultimately confirms one leading scholar’s view that the federal courts have not thought out interpretive durability’s place in judicial review.²⁸⁰

The five remaining factors were obscure in circuit-court decisions. Courts invoked political accountability in 0.5% of all interpretations, public reliance in 0.7%, contemporaneity in 1.9%, national standards in 2.2%, and congressional acquiescence in 3.1%.

These results provide some (albeit limited) insights on the place of Mead, Barnhart, and the remaining contextual factors. Mead’s focus on delegation and formality, unsurprisingly, has a firm grasp on the circuit courts. Two of the most significant factors that courts invoked were agency procedures and rulemaking authority, both of which focus on the ability of the agencies to speak with the force of law and use of that authority. Relatedly, given the high rates at which formalized agency interpretations received Chevron deference,²⁸¹ it appears that courts and parties considered formality even if they did not usually mention it. Given Mead’s relatively straightforward view as to formal interpretations, these factors’ prominence is not surprising.

The salience of Barnhart’s dicta, in contrast, is less certain. Of the three Barnhart factors that we coded (expertise, longevity, and congressional acquiescence), courts invoked the first two more frequently than other contextual factors, but still at relatively low rates of 18.4% and 10.7% and more frequently in the context of Skidmore review in which they are doctrinal factors. But similar to our inferences from our data on Chevron applications to formal interpretations above, our data on interpretations’ duration—where consistent agency interpretations prevailed at higher rates under all deference regimes combined, despite not receiving Chevron deference at increased levels²⁸²—suggest that the factor may be doing silent work in the circuit courts’ decisionmaking after a step-zero inquiry. Unlike with long-standing interpretations, we do not have another variable that might illuminate whether agency expertise informs circuit-court decisionmaking even when the courts do not refer to it. As to the third factor, circuit courts referred to congressional acquiescence only 3.1% of the time, significantly less than the other two factors, suggesting perhaps that it has little salience in judicial decisionmaking. Yet, as with the other variables, we cannot rule out the chance that courts consider congressional acquiescence without mentioning it.

The remaining ad hoc contextual factors or theoretical concerns appear to have little purchase on the circuit courts, which referred to any one of them only, at most, approximately 2% of the time. Their low salience suggests that certain traditional factors have faded from judicial memory. Most prominently, contemporaneity (along with long-standing consistency), a traditional factor of long provenance, has essentially lost its hold on circuit courts.²⁸³ This finding was not surprising given the Supreme Court’s consistent view that “neither antiquity nor contemporaneity with [a] statute is a condition of [a regulation’s] validity.”²⁸⁴ Perhaps, though, like other factors, courts accept contemporaneous interpretations more frequently and thus the factor is doing more work behind the scenes than expressed invocations suggest.²⁸⁵

Although we coded for courts’ express references to contemporaneity, we did not code specifically for contemporaneous interpretations by themselves. But if we use our variable of recent interpretations arising from a new or amended statute as contemporaneous (117 interpretations), contemporaneous interpretations prevailed under any deference regime 63.2% of the time and under Chevron 69.5% of the time. Notably, both of these numbers were lower than the overall agency-win rate under any regime (71.4%) and the average Chevron agency-win rate (77.4%), suggesting that contemporaneity does not have the same pull on courts as the win rates suggest that stability and formality do. That said, our variable for recent interpretations arising from a new or amended statute wouldn’t include all contemporaneous interpretations, such as those that are long-standing (and thus not new) but issued contemporaneously with a statute’s enactment or amendment. And it may include interpretations that, while new, did not occur until many years after a statute’s enactment or amendment because, after all, rulemaking or adjudication takes time. Because our variable doesn’t track contemporaneity perfectly, our conclusions are necessarily limited.

Whatever the normative value of the Barnhart and other contextual factors in judicial deference, their largescale absence from deference discussions in the circuit courts suggests that courts prefer the relatively more rule-like certainty of Mead than the ad hoc approaches before Chevron or offered by Barnhart. This is so despite the fact that the ad hoc approaches would provide circuit courts more discretion and allow them to better hide strategic decisionmaking to allow courts to align policy preferences with their interpretations. Like Odysseus tied to the mast, circuit courts seem to have found some benefits in having others limit their agency.

Conclusion

Let us briefly return to where we began with our findings in Part III—the big picture. We have discussed particular findings and their implications in each Part. But what broader insights about Chevron Regular and Chevron Supreme can we glean from stepping back and considering our findings as a whole?

We have demonstrated empirically that, contrary to how they fare in the Supreme Court,²⁸⁶ agencies usually prevail more under Chevron than other standards of review in the circuit courts (at least when those courts refer to Chevron).²⁸⁷ This finding is meaningful for agencies and litigating parties because circuit courts review far more agency statutory interpretations than the Supreme Court. Although we cannot say in our discussion here how the deference standards affect judicial decisionmaking, we can say outcomes do vary. Because they do, one leading scholar’s call, based on findings from past empirical studies, for practitioners, teachers, courts, and scholars to deemphasize review standards appears premature.²⁸⁸ They seem to matter, even if no one, including us (based on methodological limitations), can yet say exactly how.

If Chevron matters, we should consider whether it is functioning properly. The Supreme Court indicated that Chevron exists to provide agencies a congressionally delegated space to regulate, where courts keep agencies in their space without imposing their own policy judgments.²⁸⁹ The doctrine largely appears to fail at achieving these aims in the Supreme Court based on its rare invocation²⁹⁰ and failure to constrict the justices’ perceived preferences.²⁹¹ Prior studies of the circuit courts have also found that Chevron does not appear to meaningfully constrict judges from deciding in accord with their perceived political preferences²⁹²—at least when a judge on a panel with different political preferences isn’t on the panel.²⁹³ Although we leave our ideology data and more sophisticated statistical modeling for future work, our initial, descriptive findings suggest, based on a larger dataset than in prior studies, that Chevron has some kind of disciplining effect in the aggregate on circuit courts because agency-win rates are so disparate between when Chevron applies and when it does not, even when the agency statutory interpretations use the same formal interpretive methods.²⁹⁴

More specifically, our thirty-nine-percentage-point difference between agency-win rates under Chevron and de novo review suggests that courts distinguish looking for the best answer from permitting a reasonable one.²⁹⁵ If they are able and willing to do so, then the Supreme Court’s recently invoked “stabilizing purpose”—to render outcomes from thirteen circuit courts more predictable²⁹⁶ and thereby further the uniformity goals that Peter Strauss highlighted decades ago²⁹⁷—becomes more compelling, regardless of the delegation theory’s normative force.²⁹⁸ Indeed, as federal dockets have swelled, Chevron may be one more device that federal courts have used to avoid what they perceive as low-value or low-interest cases.²⁹⁹

But, at the same time, our data indicate that the Supreme Court needs to provide better guidance to lower courts if it seeks to create a stabilizing doctrine. The circuit-by-circuit disparity in the circuit courts’ invocation of Chevron and agency-win rates reveals that Chevron may not be operating uniformly among the circuits.³⁰⁰ To ameliorate uniformity, the Court should provide clearer guidance to numerous issues, which other scholars have noted: What are the “traditional tools of statutory construction”³⁰¹ to which Chevron referred for step one that courts should use?³⁰² Should the long-standing nature of agency interpretations matter?³⁰³ What role exactly should legislative history or a purposivist inquiry have?³⁰⁴ Is there an “order of battle” in which the circuit courts proceed through certain steps or interpretive canons to interpret statutes?³⁰⁵ Is step two different from arbitrary-and-capricious review and, if so, how?³⁰⁶ And perhaps more prominently, what role do agency expertise, formality, and the significance of the question have when determining when Congress has delegated authority to agencies?³⁰⁷ If Chevron is a means of controlling the lower courts, the case for providing more guidance becomes urgent.

And our findings, albeit to a limited degree, suggest that lower courts will view more rule-based guidance as a comforting swaddling blanket rather than handcuffs. Circuit courts rarely invoked various values—including those mentioned in Barnhart—that they could have used to gain additional discretion in deciding whether to invoke Chevron or ultimately side with the agency.³⁰⁸ And they appeared to largely ignore troubling step-zero questions concerning sensitive matters, perhaps having difficulty discerning the Supreme Court’s vague or inconsistent signals as to these matters.³⁰⁹ If Chevron can function as a welcomed supervisory doctrine, the differences between Chevron Supreme—functioning as a malleable, discretionary canon of construction³¹⁰—and Chevron Regular—functioning as precedent—become less troubling.

Exceptional questions, rare theoretical grounds, and Chevron’s inconsistent use can permit the Supreme Court to keep the delegation theory in check at the margins without, as our data suggest, creating confusion and, as we plan to consider in future work, promoting ideological decisionmaking in the circuit courts. Indeed, two scholars have recently argued that distinctions between Chevron Supreme and Chevron Regular, at least as to major questions, are normatively justified.³¹¹ Their argument follows another scholar’s call for the degree of deference to agency interpretations to vary based on the deciding court’s place in the federal judicial hierarchy, with more deference in lower courts and less deference in superior courts.³¹² But even if differences in deference among courts defy normative justification as to all interpretive matters or exceptional questions, our data suggest that any problematic characteristics of Chevron Supreme do not necessarily trickle down to the lower courts. Ultimately, Chevron Supreme, with its comparatively broader discretion, will shift power from the circuit courts to the Supreme Court and agencies but leave Chevron Regular in place to create more certainty in the lower courts and, thus, greater national uniformity in federal administrative law.³¹³

This is not our last word on what our data say about Chevron, and we hope that it furthers numerous other conversations concerning deference to agency statutory interpretations—whether about its normative place, its operation, or its meaningfulness.

* Associate Professor, University of Georgia School of Law.

** Associate Professor, The Ohio State University Moritz College of Law. For helpful comments on prior drafts, many thanks to Michael Asimow, Nick Bagley, Emily Bremer, Aaron-Andrew Bruhl, Bill Eskridge, David Hausman, Kristin Hickman, Brian Kalt, Ron Levin, Gillian Metzger, Aaron Nielson, Jennifer Nou, Jim Oleske, Dick Pierce, Daphna Renan, Usha Rodrigues, Guy Rub, Michael Sant’Ambrogio, Jed Stiglitz, Peter Strauss, and Adrian Vermeule and to the participants at the American Association of Law Schools Annual Meeting, the American Bar Association’s Annual Administrative Law Conference, and the First Annual Administrative Law New Scholarship Roundtable. We are extremely grateful to our research assistants: Morgan Allyn, Megan Bracher, Mathew Doney, Sidney Eberhart, Lauren Farrar, JD Howard, Gregg Jacobson, Mariam Keramati, Patrick Leed, David McGee, James Mee, Andrew Mikac, Justin Nelson, Meghna Rao, Serge Rumyantsev, Jonathan Stuart, and Molly Werhan. Finally, we appreciate the Michigan Law Review editors’ careful attention to our Article.