Chevron in the Circuit Courts

This Article presents findings from the most comprehensive empirical study to date on how the federal courts of appeals have applied Chevron deference—the doctrine under which courts defer to a federal agency’s reasonable interpretation of an ambiguous statute that it administers. Based on 1,558 agency interpretations the circuit courts reviewed from 2003 through 2013 (where they cited Chevron), we found that the circuit courts overall upheld 71% of interpretations and applied Chevron deference 77% of the time. But there was nearly a twenty-five-percentage-point difference in agency-win rates when the circuit courts applied Chevron deference than when they did not. Among many other findings, our study reveals important differences across circuits, agencies, agency formats, and subject matters as to judicial review of agency statutory interpretations.

Based on prior empirical studies of judicial deference at the Supreme Court, however, our findings suggest that there may be a Chevron Supreme and a Chevron Regular: whereas Chevron may not have much of an effect on agency outcomes at the Supreme Court, Chevron deference seems to matter in the circuit courts. That there is a Chevron Supreme and a Chevron Regular may suggest that, in Chevron, the Supreme Court has an effective tool to supervise lower courts’ review of agency statutory interpretations. To render Chevron more effective in creating uniformity throughout the circuit courts, the Supreme Court needs to send clearer signals on how courts should apply the deference standard.


It is a bedrock principle of administrative law that a reviewing court must defer to a federal agency’s reasonable interpretation of an ambiguous statute it administers.1See Chevron U.S.A. Inc. v. Nat. Res. Def. Council, Inc., 467 U.S. 837 (1984). This Chevron deference doctrine is both untouchable and yet always under attack. Chevron deference has been a cornerstone of judicial review of agency action for more than thirty years, and the decision itself is one of the most cited Supreme Court decisions of all time. Indeed, as of this writing, Chevron has been cited in more than 80,000 sources available on Westlaw, including in roughly 15,000 judicial decisions and nearly 18,000 law review articles and other secondary sources.2See, e.g., Aaron L. Nielson, Visualizing Change in Administrative Law, 49 Ga. L. Rev. 757, 786 (2015) (citing 1 Richard Pierce, Jr., Administrative Law Treatise § 3.2 (5th ed. 2010)); Peter M. Shane & Christopher J. Walker, Foreword, Symposium on Chevron at 30: Looking Back and Looking Forward, 83 Fordham L. Rev. 475, 475 (2014).

In these tens of thousands of sources, scholars, litigants, and judges have contested Chevron’s theoretical grounding,3See, e.g., Stephen Breyer, Judicial Review of Questions of Law and Policy, 38 Admin. L. Rev. 363, 370 (1986) (referring to Chevron’s delegation theory as a “legal fiction”); Evan J. Criddle, Chevron’s Consensus, 88 B.U. L. Rev. 1271, 1273 (2008) (arguing that Chevron’s methodology is pluralistic because all leading theories of the administrative state support deference); Ronald J. Krotoszynski, Jr., Why Deference?: Implied Delegations, Agency Expertise, and the Misplaced Legacy ofSkidmore, 54 Admin. L. Rev. 735, 737 (2002) (arguing that expertise should inform judicial deference, not delegation of interpretive primacy); Antonin Scalia, Judicial Deference to Administrative Interpretations of Law, 1989 Duke L.J. 511, 517 (arguing that Chevron’s delegation theory is a fiction providing a background rule for legislative drafting). its provenance,4See, e.g., John F. Duffy, Administrative Common Law in Judicial Review, 77 Tex. L. Rev. 113, 193–99 (1998) (arguing that Chevron is inconsistent with the Administrative Procedure Act (APA)); Cynthia R. Farina, Statutory Interpretation and the Balance of Power in the Administrative State, 89 Colum. L. Rev. 452, 476 (1989) (“It is surely a far more remarkable step than Chevron acknowledged to number among Congress’s constitutional prerogatives the power to compel courts to accept and enforce another entity’s view of legal meaning whenever the law is ambiguous.”); Thomas W. Merrill & Kristin E. Hickman, Chevron’s Domain, 89 Geo. L.J. 833, 868 (2001) (noting that if Chevron is federal common law, it does not coexist well with the judicial-review provisions of the APA). and its impact on case outcomes.5See, e.g., Jack M. Beermann, End the FailedChevron Experiment Now: HowChevron Has Failed and Why It Can and Should Be Overruled, 42 Conn. L. Rev. 779, 829 (2010); Richard J. Pierce, Jr., What Do the Studies of Judicial Review of Agency Actions Mean?, 63 Admin. L. Rev. 77, 85 (2011) (presenting ranges of agency affirmance rates under judicial-review doctrines and finding that differences are “barely detectable,” except for deference to agencies’ interpretations of their own regulations). More recently, Supreme Court justices have questioned not only Chevron’s reach6See Kent Barnett, Why Bias Challenges to Administrative Adjudication Should Succeed, 81 Mo. L. Rev. 1023, 1036–37 (2016) (noting the Court’s ignoring of Chevron when it would have appeared to apply in Texas Department of Housing & Community Affairs v. Inclusive Communities Project, Inc., 135 S. Ct. 2507 (2015), and the Court’s refusal to apply Chevron to matters of “deep ‘economic and political significance’” in King v. Burwell, 135 S. Ct. 2480, 2489 (2015)). Three justices have argued that Chevron should not apply to agencies’ interpretations concerning the scope of the agency’s authority. City of Arlington v. FCC, 133 S. Ct. 1863, 1877–86 (2013) (Roberts, C.J., dissenting). but also its very existence.7E.g., Michigan v. EPA, 135 S. Ct. 2699, 2712 (2015) (Thomas, J., concurring) (arguing that Chevron deference presented “serious separation-of-powers questions” because it either, if interpreting a statute, contravened the original understanding that the Article III judicial power requires courts to “say what the law is” or, if making policy, improperly delegated legislative power (quoting Marbury v. Madison, 5 U.S. (1 Cranch) 137, 177 (1803))). Congressional Republicans have followed suit by introducing legislation that would abolish Chevron deference and require courts to review agency statutory and regulatory interpretations de novo.8See Separation of Powers Restoration Act of 2016, S. 2724, 114th Cong. § 2 (2016) (amending the APA to require courts to review “de novo all relevant questions of law, including the interpretation of constitutional and statutory provisions and rules”); Separation of Powers Restoration Act of 2016, H.R. 4768, 114th Cong. § 2 (2016) (same); see also Christopher J. Walker, Courts Regulating the Regulators, RegBlog (Apr. 25, 2016), [] (explaining how pending legislation is an outgrowth of recent Supreme Court criticism of Chevron deference).

Much scholarly attention focuses on the use or absence of Chevron deference at the Supreme Court. Some scholars have focused on Chevron’s domain—that is, when Chevron applies in judicial review.9See, e.g., Merrill & Hickman, supra note 4. Others have considered empirically how consistently the Court applies Chevron. In their leading study concerning agency deference in the Supreme Court from 1984 to 2006, Bill Eskridge and Lauren Baer found that the Court applied Chevron deference only one quarter of the time that it would have seemed to apply.10William N. Eskridge, Jr. & Lauren E. Baer, The Continuum of Deference: Supreme Court Treatment of Agency Statutory Interpretations fromChevron to Hamdan, 96 Geo. L.J. 1083, 1124–25 (2008); see also Thomas W. Merrill, Judicial Deference to Executive Precedent, 101 Yale L.J. 969, 982 (1992) (“[I]t is clear that Chevron is often ignored by the Supreme Court. . . . [T]he two-step framework has been used . . . only about one-third of the [time].”).  When the Court applied the doctrine, agencies prevailed 76.2% of the time, a rate similar to those under other standards of review.11Eskridge & Baer, supra note 10, at 1142. A later study by Thomas Miles and Cass Sunstein that considered decisions from 1989 until 2005 in which the Supreme Court invoked Chevron found that agencies prevailed only 67% of the time. Thomas J. Miles & Cass R. Sunstein, Do Judges Make Regulatory Policy? An Empirical Investigation of Chevron, 73 U. Chi. L. Rev. 823, 849 (2006). “Since the period studied by Miles and Sunstein overlaps almost completely with the last fifteen years of the period studied by Eskridge and Baer, the lower affirmance rate found by Miles and Sunstein implies a decline in the Supreme Court’s rate of affirmance in Chevron cases after 1990.” Pierce, supra note 5, at 83–84.

In other words, the Court’s choice to apply Chevron deference, as opposed to a less-deferential doctrine or no deference at all, does not seem to affect the outcome of the case. Chevron deference—at least at the Supreme Court—does not seem to matter. As Richard Pierce has concluded, “There is no empirical support for the widespread belief that choice of doctrine plays a major role in judicial review of agency actions.”12Pierce, supra note 5, at 93; see also Stuart Minor Benjamin & Arti K. Rai, Administrative Power in the Era of Patent Stare Decisis, 65 Duke L.J. 1563, 1598 (2016). Scholars and commenters, moreover, have noticed the Court’s recent treatment of Chevron as a doctrine to ignore, disparage, or distinguish.13See, e.g., Michael Herz, Chevron Is Dead; Long Live Chevron, 115 Colum. L. Rev. 1867, 1869 (2015) (“Some early commenters see these decisions as at least potentially marking a watershed moment, a fundamental shift from a regime of meaningful deference to a reassertion of judicial supremacy.”).

But Chevron in the Supreme Court is not our focus. Instead, we are most concerned here with how Chevron works on the ground in the circuit courts. Prior empirical studies of Chevron in the circuit courts were limited to a particular court,14See, e.g., Frank B. Cross & Emerson H. Tiller, Judicial Partisanship and Obedience to Legal Doctrine: Whistleblowing on the Federal Courts of Appeals, 107 Yale L.J. 2155, 2168 (1998) (considering more than 200 D.C. Circuit decisions from 1991 to 1995). particular agencies,15See, e.g., Miles & Sunstein, supra note 11, at 825 (considering decisions from 1990 to 2004 concerning the NLRB or the EPA). particular subject matter,16See, e.g., Jason J. Czarnezki, An Empirical Investigation of Judicial Decisionmaking, Statutory Interpretation, and the Chevron Doctrine in Environmental Law, 79 U. Colo. L. Rev. 767, 784–85 (2008) (considering environmental-law cases in the circuit courts for a three-year period). or a short timeframe.17See, e.g., Orin S. Kerr, Shedding Light on Chevron: An Empirical Study of theChevron Doctrine in the U.S. Courts of Appeals, 15 Yale J. on Reg. 1, 4 (1998) (considering more than 200 cases from 1995 to 1996); Peter H. Schuck & E. Donald Elliott, To the Chevron Station: An Empirical Study of Federal Administrative Law, 1990 Duke L.J. 984, 989–90 (considering nearly 2,500 decisions, including those without published opinions, in six- or two-month periods during certain years to ascertain Chevron’s effect on judicial review). Although the Schuck and Elliott study is important and comprehensive, its more-than-thirty-year-old data fail to address contemporary appellate practice after Skidmore and the Court’s inconsistent use of its deference doctrines. Moreover, the decisions, too, included all administrative decisions, not just those concerning statutory interpretation, and the study did not identify the number of agency interpretations upheld or reversed because it considered only remand rates. See Linda R. Cohen & Matthew L. Spitzer, Solving theChevron Puzzle, 57 Law & Contemp. Probs. 65, 91, 103 (1994). They have also largely concentrated on the rates at which agencies prevail under Chevron and the likelihood of judges’ policy preferences affecting Chevron’s application.18See, e.g., Cohen & Spitzer, supra note 17, at 103–06; Cross & Tiller, supra note 14, at 2169–76 (considering panel composition on agency-win rates); Miles & Sunstein, supra note 11, at 825–26. Our inquiry and scope are significantly broader.

This Article presents the findings of the largest empirical study of Chevron in the circuit courts to determine how Chevron works outside the marbled enclave of One First Street. Our database of 2,272 judicial decisions, collected with broad search parameters, attempts to cull all published decisions from the circuit courts over an eleven-year period (2003–2013) that refer to the Chevron doctrine. Within the relevant 1,327 of those collected opinions, we uncovered 1,558 instances of judicial review of an agency statutory interpretation (not merely any kind of agency action). Largely following Eskridge and Baer’s methodology, we coded each agency statutory interpretation with respect to nearly forty different variables, including information about the decision (circuit, year, judges, and separate opinions); information about the agency interpretation (the agency, subject matter, final agency decisionmaker, agency procedure used, and ideological valence of agency’s interpretation); and information about the judicial outcome (outcome as to agency, ideological valence of the decision, standard of review applied, and factors that influenced the court’s decision).19Our Codebook, which lists all of the variables with descriptions, is available at ______. This broad set of cases permitted us to consider all instances within our parameters in which the circuit courts applied the Chevron framework. This set also permitted us to review all instances in which the circuit courts, having referred to Chevron, reviewed agency interpretations de novo or under the Skidmore deference regime (under which courts defer to an agency’s interpretation based on several factors, including the thoroughness of the agency’s interpretation and its consistency with prior pronouncements).20For a discussion of Skidmore deference, in which courts retain interpretive primacy and evaluate the weight to give an agency’s interpretation, see infra Section I.A.

This treasure trove of data, albeit with methodological limitations that we discuss in Section II.B, provides a number of often-surprising insights regarding deference to agency statutory interpretations in the circuit courts. Many of these findings suggest, with some caveats, that there may be a Chevron Supreme and a Chevron Regular: whereas the choice to apply Chevron deference may not matter that much at the Supreme Court, it seems to matter in the circuit courts. Consider the following key findings from the study:

First, agency interpretations were significantly more likely to prevail under Chevron deference (77.4%) than Skidmore deference (56.0%) or, especially, de novo review (38.5%). In other words, agencies won significantly more in the circuit courts when Chevron deference applied, at least when the court expressly considered whether to apply Chevron. Indeed, there was nearly a twenty-five-percentage-point difference in agency-win rates with Chevron deference (77.4%) than without (53.6%). Because the agency-win rates in Eskridge and Baer’s study of the Supreme Court were much more similar no matter whether Chevron (76.2%), Skidmore (73.5%), or de novo review (66.0%) applied, this was one of our first indications that Chevron Supreme differs from Chevron Regular.21See infra Section III.A.

Second, when Chevron’s well-known two-step approach applied, the circuit courts resolved the matter at step one (i.e., the step at which the courts ask whether Congress’s intent was clear) 30.0% of the time, and, of those Chevron step-one decisions, agencies prevailed 39.0% of the time. Of the 70.0% of the interpretations that moved to Chevron step two (the step at which the courts defer to reasonable agency interpretations when Congress’s intent was not clear at step one), the agency prevailed 93.8% of the time. Based on albeit-dated data from Tom Merrill, Chevron Supreme does not behave like Chevron Regular. Merrill found that the Supreme Court resolved matters in the agency’s favor 59% of the time at step one.22Merrill, supra note 10, at 981 tbl.1. (Merrill—and others after him—did not report comparable data about the Supreme Court’s step-two practice.) This difference may suggest that, given the higher likelihood of circuit-court review than Supreme Court review, agencies should give closer attention to the statutory language but that their step-two explanations are largely sufficient.23See infra Section III.B.

Third, as expected and as in the Supreme Court, formal agency interpretations prevailed at higher rates than informal ones, without regard to scope of review. But unlike in the Supreme Court, where the agency-win rate for formal adjudication (65.4%) was lower than notice-and-comment rulemaking (72.5%),24Eskridge & Baer, supra note 10, at 1148 tbl.17. agency-win rates in formal adjudication were slightly higher (74.7%, or 81.3% when excluding immigration adjudications with idiosyncratic review procedures) than notice-and-comment rulemaking (72.8%) in the circuit courts. Formal interpretations, under a trilogy of Supreme Court decisions,25See infra Section I.B.1. also unsurprisingly received Chevron deference at higher rates: 100.0% of the time for (albeit extremely rare) formal rulemaking, 91.9% for notice-and-comment rulemaking, and 76.7% for formal adjudication (or 85.2% if excluding immigration adjudications). Informal interpretations lagged behind at 44.8%. But, despite the Supreme Court treating legislative rulemaking and formal adjudication alike in its Chevron doctrine, our numbers revealed that the circuit courts applied the Chevron framework less frequently to formal adjudication than rulemaking. Perhaps even more surprising, when Chevron applied, interpretations in formal adjudications had a higher agency-win rate (81.7%, and 86.0% without immigration decisions) than notice-and-comment rulemaking (74.4%). These findings suggest that agencies may want to reconsider formal adjudication as a substitute to rulemaking as a means of adopting Chevron-eligible agency statutory interpretations, despite adjudication’s fall in popularity since the 1970s.26See infra Section III.C.

Fourth, the circuit courts varied considerably as to overall agency-win rates, application of Chevron, and agency-win rates under Chevron. For overall rates, the First Circuit was the most agency friendly with an agency-win rate of 82.8%, while the Ninth Circuit was the least agency friendly with a rate of 65.8%. As for Chevron’s application, the D.C. Circuit applied it almost as a matter of course at 88.6% of the time, while the Sixth Circuit applied it only 60.7% of the time. Once Chevron applied, though, the agency seemed to prevail as a rule in the Sixth Circuit (88.2% of the time, the highest rate), while the agency won only 72.3% of the time in the Ninth Circuit, the lowest rate. The differential between agency-win rates with and without Chevron indicates that agencies prevailed more in all circuits when Chevron applied. The most striking was the Sixth Circuit, with its nearly fifty-percentage-point difference in agency-win rates. Only the Eighth Circuit had a differential that was less than five percentage points, and the Eleventh Circuit was the only other circuit with a differential of less than ten percentage points. Although our data indicate that agencies win much more frequently when Chevron applies in all but one circuit, they also suggest that the Supreme Court may need to send clearer signals if the Court wants Chevron to apply evenly throughout the circuit courts.27See infra Part IV.

Fifth, agency-win rates varied dramatically by subject matter and by the agency advancing the interpretation. For instance, the Federal Communications Commission (FCC) (82.5% overall agency-win rate), Treasury Department (78.9%), and, perhaps surprisingly, National Labor Relations Board (NLRB) (78.1%) were a few of the big winners among the agencies in the dataset. By contrast, the Equal Employment Opportunity Commission (EEOC) (42.9%), Energy Department (45.5%), and Department of Housing and Urban Development (HUD) (54.2%) were among the biggest losers in the circuit courts. The range of circuit courts applying Chevron deference also varied considerably from 100.0% for the Interstate Commerce Commission/Surface Transportation Board (ICC/STB) to 36.4% to the Federal Trade Commission (FTC). Moreover, independent agencies outperformed executive agencies as to overall agency-win rate (77.0% to 70.2%) and frequency of Chevron application (82.5% to 73.2%)—though agency-win rate when Chevron applied evened out some (79.6% to 76.8%).

Sixth, to our surprise, the circuit courts may not be as responsive to the Supreme Court’s (often conflicting or vague) signals concerning exceptions to Chevron for certain sensitive questions. The circuits applied Chevron to two sensitive questions that we coded—74.3% of regulatory-jurisdiction questions and 76.0% of state-law preemption ones—roughly at the same rate as the overall average (74.8%). Once Chevron applied, the agency-win rate for jurisdictional interpretations was lower at 70.5% than the average Chevron agency-win rate of 77.4% for all interpretations. But the 78.9% win rate for preemption interpretations was consistent with the overall average. The small population of sensitive-question interpretations, especially the preemption questions, limits the strength of inferences that one can draw from the data. But these data may suggest that at least for certain matters, the circuit courts have somewhat internalized the courts’ sensitive-question exceptions to Chevron as part of its overall analysis but not at “step zero” (i.e., the step where courts consider whether to apply Chevron’s two-step approach).28See infra Section VI.A.

Seventh, long-standing agency interpretations prevailed under all deference regimes combined at a much higher rate (82.3%) than those that were new and replaced no earlier interpretation (65.9%), those that were inconsistent with a prior interpretation (59.8%), and those whose duration we could not discern from the decision (67.8%). The circuit courts, consistent with Supreme Court doctrine but not practice, applied Chevron consistently to long-standing and new interpretations, and—to our surprise—at an even higher rate to inconsistent interpretations. But inconsistent agency interpretations prevailed under Chevron much less frequently (65.6%) than recent (74.7%) and long-standing interpretations (87.6%). Indeed, we found that agencies’ inconsistent interpretations prevailed significantly less than other interpretations under every review standard except de novo review (although much more frequently under Chevron than other review standards). Moreover, inconsistent interpretations based on new political administrations or unclear reasons were the least likely to prevail. These findings suggest that agencies seeking to change positions should work diligently to ensure that their interpretations receive Chevron deference (by having the force of law) and that they rely on grounds such as changed circumstances or accumulated expertise.29See infra Section VI.B.

Finally, we found that traditional contextual or theoretical grounds for deference do not have much expressed salience in the circuit courts. Courts mentioned only four of our nine coded factors in more than one in ten interpretations. The most salient factors were agency procedures, agency rulemaking authority, agency expertise, and interpretive stability. The first two’s prominence are of little surprise because they track concerns for formality and delegation in leading precedent. Moreover, expertise and interpretive stability are factors for Skidmore deference and factors that the Court has mentioned as relevant to Chevron in one leading decision. But the absence of the other factors—political accountability, public reliance, contemporaneity, uniformity in federal administrative law, and congressional acquiescence—suggests that the circuit courts have found comfort in the Court’s more rule-based line of decisions and have largely rebuffed another decision’s more open-ended, sliding-scale inquiry that could give them additional discretion.30See infra Section VI.C.

The Article proceeds as follows. Part I briefly discusses Chevron’s birth, theoretical underpinnings, and evolution. Part II discusses prior empirical studies, our study design and methodology, and an overview of our dataset. Part III provides the 10,000-foot view of our findings, looking at agency-win rates, judicial application of Chevron deference (and its effect on agency-win rates), and the effect of agency procedures on outcomes and deference regimes. Part IV then disaggregates the findings by circuit, whereas Part V analyzes the results by agency and subject matter, including the differences between executive and independent agencies. Part VI examines what else seems to matter (or not) based on other variables in the dataset. The theoretical and normative implications of these findings, including how they support or cast doubt on existing doctrine and theory, are discussed in each of these Parts. The Article concludes by exploring the advantages of a Chevron Supreme and a Chevron Regular. Although Chevron may not have much of an effect on agency outcomes at the Supreme Court (based on prior empirical studies of the Court), it seems to matter markedly in the circuit courts. This may suggest that, in Chevron, the Supreme Court has an effective tool to supervise and rein in the lower courts in their review of agency statutory interpretations. But the Court needs to provide additional guidance to ensure that Chevron applies consistently throughout the circuits.

I.        Chevron’s Ever-Changing Role

Federal courts have a long-standing practice of deferring in some manner to federal agency statutory interpretations. But the level of deference, triggers for deference, exceptions from it, and judicial discussions of deference have fluctuated significantly. Our purpose here is not to provide a complete history of judicial deference. Instead, this Part contextualizes Chevron deference as necessary to understand the salience of our findings.

A.     Discursive Deference to Agencies Before Chevron

Early federal courts deferred to agency action in two key ways. First, they deferred to discretionary, as opposed to ministerial, executive decisions in mandamus actions.31See, e.g., Kendall v. United States ex rel. Stokes, 37 U.S. (12 Pet.) 524, 613–14 (1838); Marbury v. Madison, 5 U.S. (1 Cranch) 137, 170 (1803). Second, outside of mandamus actions, they often “respected” agency interpretations of ambiguous statutory provisions when those interpretations were long-standing or contemporaneous with the statute’s enactment.32Aditya Bamzai, The Origins of Judicial Deference to Executive Interpretation, 126 Yale L.J. 908, 941 (2017). As the administrative state grew in size and influence, debates surrounding the intensity of judicial review of agency legal interpretations became more urgent and strident.33See id. at 966–82 (discussing judicial and academic debates concerning judicial review and challenges to the traditional approaches). Ultimately, the Supreme Court fluctuated from the New Deal until Chevron between three deference regimes.

Consistent with NLRB v. Hearst Publications, Inc., the first line of decisions called for significant deference to agency interpretations that had a “reasonable basis in law.”34322 U.S. 111, 131 (1944), overruled in part by Nationwide Mut. Ins. Co. v. Darden, 503 U.S. 318 (1992). In that case, for example, the Court deferred to the NLRB’s interpretation that it provided in its adjudication of “employee,” a “broad statutory term,” to include newsboys.35Hearst Publ’ns, 322 U.S. at 131–32. The Court grounded that deference on notions of congressional delegation to the agency to provide the interpretation and administrative expertise.36Id. at 130. See generally Kristin E. Hickman & Matthew D. Krueger, In Search of the Modern Skidmore Standard, 107 Colum. L. Rev. 1235, 1239 (2007) (collecting decisions).

Skidmore v. Swift & Co.,37323 U.S. 134 (1944). decided only one Term after Hearst, was the standard-bearer for the second line of decisions in which the courts applied an indefinite, multifactored inquiry to deference questions. The Skidmore Court deferred to the flexible, contextual method that the Department of Labor’s Wage and Hour Division called for in an interpretive ruling to determine whether “waiting time” was subject to overtime pay.38See Skidmore, 323 U.S. at 140. After noting that the agency’s interpretations were not controlling on the courts, the Court held that they “constitute[d] a body of experience and informed judgment” to guide courts.39Id. The weight to give this agency guidance depends upon the thoroughness of the agency’s consideration, the validity of its reasoning, its consistency with previous and later agency pronouncements, and “all those factors which give it power to persuade.”40Id. Since Skidmore, courts have relied on numerous other factors, including agency expertise, the contemporaneity and longevity of the agency’s interpretation, formality, congressional acquiescence, and alignment between the agency’s interpretation and congressional preferences. See Hickman & Krueger, supra note 36, at 1258–59.

The final line appeared to apply de novo judicial review, without deference to the agencies. Richard Pierce identifies NLRB v. Bell Aerospace Co.41416 U.S. 267 (1974). as a key example in which the Court, ignoring the NLRB’s contrary adjudicatory interpretation, substituted its own interpretation of the term “employee” under the National Labor Relations Act.42Pierce, supra note 2, at 157. It did so without referring to notions of judicial deference.43See Bell Aerospace Co., 416 U.S. 267.

As others have noted, the Supreme Court was not consistent, despite criticism from lower courts and despite scholarly efforts to provide a descriptive reconciliation of the competing scopes of review.44See, e.g., Pierce, supra note 2, § 3.1, at 156; Colin S. Diver, Statutory Interpretation in the Administrative State, 133 U. Pa. L. Rev. 549, 562 n.95 (1985) (arguing that the Court relied on ten different factors pre-Chevron). Nor did the Court seek to reconcile its various judicial-review regimes. But the Court’s 1984 Chevron decision, despite failing to provide doctrinal reconciliation, appeared to provide clearer, simpler guidance.

B.     Chevron and Its Domain

In Chevron, the Court famously created a two-step process for judicial review of agency statutory interpretations that appeared to apply whenever “a court reviews an agency’s construction of the statute which it administers.”45Chevron U.S.A. Inc. v. Nat. Res. Def. Council, Inc., 467 U.S. 837, 842–43 (1984). As to the first inquiry, the court should determine if Congress has clearly provided its unambiguous intent on the issue. If Congress has not, then the court, in its second inquiry, should ask whether the agency’s construction is permissible.46Id. The Court expressly stated that the agency has room to adopt more than one reasonable interpretation, meaning that an agency’s changed interpretation can receive deference.47See id. at 863. The Court largely grounded Chevron deference on a theory that Congress had delegated interpretive primacy to the agency, instead of the courts.48Id. at 843–44. But the Court also identified other theoretical support: agencies are institutionally superior to courts because of their expertise over complex statutory schemes, and executive officials are more politically accountable than judges.49See id. at 865–66.

Courts and scholars largely understood Chevron as a significant restatement or recalibration of judicial review,50See, e.g., Pierce, supra note 2, § 3.4 (discussing benefits of Chevron’s reconceptualization of judicial review); Keith Werhan, Principles of Administrative Law 372 (2d ed. 2014) (“Chevron fundamentally altered scope-of-review doctrine governing an agency’s interpretations of its enabling act.”). and it quickly gained prominence in the lower courts, especially the influential D.C. Circuit.51Gary Lawson, Federal Administrative Law 540–41 (6th ed. 2013); Merrill & Hickman, supra note 4, at 838; Patricia M. Wald, The D.C. Circuit: Here and Now, 55 Geo. Wash. L. Rev. 718, 726–28 (1987). In so doing, questions concerning Chevron’s “domain” quickly surfaced.52Merrill & Hickman, supra note 4, at 848–52 (posing fourteen questions concerning Chevron’s domain). As we discuss in Section I.B.1, the most significant question was whether Chevron applied to all agency interpretations, or only those that were sufficiently formal.53See id. at 849. In a trilogy of decisions from 2000 to 2002, the Court addressed when Chevron applied but sent inconsistent signals. As we discuss in Section I.B.2, additional questions arose as to Chevron’s reach even when agencies provided formal interpretations, including: Does Chevron apply to an agency’s pronouncement concerning its own jurisdiction? Or to an agency’s preemption of state law? Or to questions of deep economic or political significance?

1.       The Role of Formality

In Christensen v. Harris County, the first of three key decisions concerning the role of formality, the Court held that an interpretation’s formality influenced Chevron’s applicability and indicated that informal interpretations were not eligible for Chevron deference.54529 U.S. 576 (2000). The Court refused to apply Chevron to the Department of Labor’s statutory interpretation in an opinion letter, noting that an opinion letter is “not . . . a formal adjudication [that is, on-the-record adjudication under the APA] or notice-and-comment rulemaking.”55Christensen, 529 U.S. at 587. The problem with interpretations in opinion letters, policy statements, agency manuals, and other guidelines is that they “lack the force of law.”56Id. Instead, they are entitled only to respect under Skidmore.57Id. Justice Scalia concurred, arguing that Skidmore was an “anachronism,”58Id. at 589 (Scalia, J., concurring in part and concurring in the judgment). that Chevron should apply to all “authoritative agency positions,”59Id. at 590. and that the Court had, indeed, applied Chevron to more than formal adjudication and notice-and-comment rulemaking.60Id. at 590–91. Justice Breyer, although dissenting, supported the majority’s resuscitating Skidmore and contended that Chevron “made no relevant change” to judicial review.61Id. at 596 (Breyer, J., dissenting). Instead, all it did was focus on congressional delegation.62Id.

One year later, the Court doubled down in United States v. Mead Corp. but left room for informal interpretations to receive Chevron deference.63533 U.S. 218 (2001). The Court provided more guidance for when Congress had delegated interpretive primacy to agencies: by giving agencies the authority to act with the force of law through “relatively formal administrative procedure tending to foster . . . fairness and deliberation . . . . Thus, the overwhelming number of our cases applying Chevron deference have reviewed the fruits of notice-and-comment rulemaking or formal adjudication.”64Mead, 533 U.S. at 230. Indeed, earlier in its opinion, it had stated that “[d]elegation . . . may be shown . . . by an agency’s power to engage in adjudication or notice-and-comment rulemaking.”65Id. at 227. But, despite Christensen’s contrary statements, the Court acknowledged that Chevron’s applicability does not require such formality; the Court had bestowed Chevron deference upon informal agency interpretations.66Id. at 231. In denying the U.S. Customs Service’s letter rulings at issue Chevron deference, the Court found no evidence of congressional intent for the rulings to have the force of law.67Id. at 231–35. The Court remanded for the lower courts to decide whether Skidmore deference applied.68Id. at 238–39. Conspicuously absent was any indication that other values, such as expertise, mattered for triggering Chevron deference. Justice Scalia, this time in dissent, largely repeated his views in Christensen.69See id. at 239–61 (Scalia, J., dissenting).

Soon thereafter, the Court suggested in dicta that Chevron’s domain depended on more than the formality of agency action. In Barnhart v. Walton, the Court, after noting that the agency had exercised its rulemaking authority, deferred under Chevron to the Social Security Administration’s reasonable statutory interpretation in a regulation.70535 U.S. 212, 217, 222 (2002). But it didn’t stop there. It held that the agency’s interpretation was “permissible” because it made “considerable sense,” was of long-standing duration (even if the regulation itself was of recent vintage), and appeared to receive congressional acquiescence in light of the relevant statute’s reenactment and amendment.71Barnhart, 535 U.S. at 219–20. The Court’s focus on the long-standing nature of the interpretation was surprising because it had held, in Chevron itself, that Chevron deference applies even when agencies change their interpretations.72See Chevron U.S.A. Inc. v. Nat. Res. Def. Council, Inc., 467 U.S. 837, 863–64 (1984); accord Nat’l Cable & Telecomms. Ass’n v. Brand X Internet Servs., 545 U.S. 967, 981–82 (2005). Moreover, consistent with Mead, the Court stated that formality was not necessary.73See Barnhart, 535 U.S. at 222 (noting that “the presence or absence of notice-and-comment rulemaking [was not] dispositive” in Mead). And it stated, perhaps even more surprisingly, that formality was also insufficient, which appears inconsistent with Mead’s strong suggestion that notice-and comment rules and formal adjudication always have the force of law to render agency action Chevron-eligible.74See id. But see Lawson, supra note 51, at 590 (“Mead establishes that an agency interpretation that is promulgated as law and has an adequate procedural pedigree (informal notice-and-comment rulemaking qualifies for this purpose) will definitely get Chevron deference . . . .”). The Court then referred to other considerations, reminiscent of Skidmore’s.75See Barnhart, 535 U.S. at 222 (including “the interstitial nature of the legal question, the related expertise of the Agency, the importance of the question to administration of the statute, the complexity of that administration, and the careful consideration the Agency has given the question over a long period of time”).

Mead, Christensen, and Barnhart altogether created the “Mead Puzzle,” creating uncertainty for courts and agencies as to which factors are necessary or sufficient to trigger Chevron deference.76See Kent Barnett, CodifyingChevmore, 90 N.Y.U. L. Rev. 1, 56–58 (2015). Generally, notice-and-comment rulemaking and formal adjudication have been thought sufficient for Chevron eligibility, despite Barnhart’s contrary suggestion,77See Lisa Schultz Bressman, How Mead Has Muddled Judicial Review of Agency Action, 58 Vand. L. Rev. 1443, 1449–50 (2005) (discussing views). and thus should be treated similarly. But lower courts, as Lisa Bressman has highlighted, have struggled with how to go about determining when informal interpretations are Chevron-eligible78See id. at 1458–64. and when they may engage in “Chevron avoidance”—that is, accept the agency’s view under Skidmore or de novo review without deciding whether the agency’s interpretation has the force of law.79See id. at 1464–69. At the same time, scholars, including one of us, have debated formality’s relationship with congressional delegation.80Barnett, supra note 76, at 20–22 (discussing scholarly views); Lisa Schultz Bressman, Reclaiming the Legal Fiction of Congressional Delegation, 97 Va. L. Rev. 2009, 2043–44 (2011). Some have argued that formality provides procedures that, in Mead’s words, “foster . . . fairness and deliberation”81United States v. Mead Corp., 533 U.S. 218, 230 (2001); see also Bressman, supra note 77, at 1449 (contending that “notice-and-comment rulemaking best ensures the transparency, deliberation, and consistency that produce fair and reasonable laws”); Merrill & Hickman, supra note 4, at 884–85 (arguing for Chevron to apply only to notice-and-comment rulemaking and formal adjudication based on public-participation values). and provide a salience or transparency to permit congressional oversight.82See, e.g., Bressman, supra note 80, at 2044–45. Others have argued that whether Congress wants to have an agency act with the force of law is a separate question from whether Congress wants searching judicial oversight of the agency’s binding actions.83See Barnett, supra note 76, at 20–21, 38–48; David J. Barron & Elena Kagan, Chevron’s Nondelegation Doctrine, 2001 Sup. Ct. Rev. 201, 218. Merrill and Hickman also argued that formality itself was not determinative; instead, it was whether the agency had congressionally delegated authority “to resolve controversies in a legally binding fashion.” Merrill & Hickman, supra note 4, at 902.

2.       Sensitive Questions

Even with sufficient formality, it was and continues to be unclear whether Chevron applies to certain sensitive questions: so-called jurisdictional matters, preemption, and exceptionally important policy matters. In addressing these questions, the Supreme Court has often provided more guidance on the theoretical underpinnings of Chevron deference and its triggers.

The Court in City of Arlington v. FCC held that agency decisions concerning their jurisdiction are eligible for Chevron deference, largely because of the difficulty in distinguishing jurisdictional questions from other statutory interpretations.84See 133 S. Ct. 1863, 1868–73 (2013). In resolving this long-simmering issue, the majority emphasized Chevron’s place in the judicial-review firmament as a stabilizing doctrine in the lower courts,85See City of Arlington, 133 S. Ct. at 1873–74. but oddly it left Mead’s place in doubt. Although the agency, according to the lower court, had engaged in informal adjudication with notice-and-comment opportunities,86See City of Arlington v. FCC, 668 F.3d 229, 241, 246–47 (5th Cir. 2012), aff’d, 133 S. Ct. 1863 (2013). the Court did not engage in a Mead inquiry into whether the action had the force of law. Instead, the majority relied on the fact that the agency had general rulemaking or adjudicatory authority and simply concluded without analysis that the agency had exercised that authority when providing its interpretation.87See City of Arlington, 133 S. Ct. at 1874. And it suggested that Barnhart’s dicta—that Chevron may not apply to formal adjudication or notice-and-comment rulemaking—was disfavored because the Court had never denied Chevron deference to interpretations in those formats.88See id. Justice Breyer, concurring, abided by his contextual Barnhart approach.89See id. at 1875 (Breyer, J., concurring in part and concurring in the judgment) (citing additional factors from Barnhart v. Walton, 535 U.S. 212, 222 (2002)). Three dissenting justices, similarly to Justice Breyer, would have applied a searching inquiry into whether Congress delegated interpretive primacy over the specific interpretation at issue.90See id. at 1881, 1883 (Roberts, C.J., dissenting).

The Court, however, has not squarely addressed state-law preemption. It has applied both Chevron and Skidmore to agency preemption decisions, albeit disagreeing with the agency in both instances.91Compare Cuomo v. Clearing House Ass’n, 557 U.S. 519, 525 (2009) (applying the Chevron framework to the Office of the Comptroller of the Currency’s preemption of state visitorial powers in notice-and-comment regulation), with Wyeth v. Levine, 555 U.S. 555, 576–77 (2009) (applying Skidmore framework to agency’s preemption decision in a notice-and-comment rule’s preamble). Nina Mendelson has called for the Court to apply Skidmore deference based on agencies’ lack of expertise on federalism that influences preemption.92See Nina A. Mendelson, Chevron and Preemption, 102 Mich. L. Rev. 737, 742, 779 (2004). And as a matter of congressional delegation, Skidmore seems appropriate. In their survey of legislative drafters, Abbe Gluck and Lisa Bressman found that a majority of drafters do not think that Congress delegates preemption matters to agencies.93Abbe R. Gluck & Lisa Schultz Bressman, Statutory Interpretation from the Inside—An Empirical Study of Congressional Drafting, Delegation, and the Canons: Part I, 65 Stan. L. Rev. 901, 1004 (2013). Likewise, one of us has surveyed agency rule drafters, who mostly thought that statutory ambiguity does not signal congressional delegation of preemption matters to agencies.94See Christopher J. Walker, Chevron Inside the Regulatory State: An Empirical Assessment, 83 Fordham L. Rev. 703, 721 (2014) (“[R]egarding preemption of state law, fewer than half (46 percent) agreed that Congress intends to delegate preemption questions by ambiguity.”).

Finally, the Court recently held in King v. Burwell that Chevron does not apply to questions of “deep ‘economic and political significance’ that [are] central to [the] statutory scheme” at issue.95135 S. Ct. 2480, 2489 (2015) (quoting FDA v. Brown & Williamson Tobacco Corp., 529 U.S. 120, 160 (2000)). In that case, the Court refused to defer to the IRS’s interpretation of “an Exchange established by the State.”96King, 135 S. Ct. at 2488–89. Although, in previous cases, the Court had considered an issue’s significance in deciding under Chevron step one whether Congress had clearly expressed its view,97See FDA v. Brown & Williamson Tobacco Corp., 529 U.S. 120, 159–60 (2000) (explaining that Congress has consistently precluded the FDA from regulating the tobacco industry, which “constitute[s] a significant portion of the American economy”). this was only the second time that the Court refused to extend Chevron deference to an admittedly ambiguous statutory provision based on the significance of the matter.98See King, 135 S. Ct. at 2492 (“[W]e cannot conclude that the phrase ‘an Exchange established by the State under [42 U.S.C. § 18031]’ is unambiguous.” (second alteration in original)); see also Gonzales v. Oregon, 546 U.S. 243, 266–67 (2006). Justice Breyer joined the majority opinion in King. What may be surprising is that not only did he find that Chevron should not apply to these exceptional significant questions, but earlier he had also indicated that it should not apply, cryptically, to “an unusually basic legal question.” Nat’l Cable & Telecomms. Ass’n v. Brand X Internet Servs., 545 U.S. 967, 1004 (2005) (Breyer, J., concurring). Moreover, the earlier decision that did so—Gonzales v. Oregon—was in the context of a fundamental change to a long-standing regulatory scheme, as opposed to a significant interpretation in a new statute, as in Burwell.99See Gonzales, 546 U.S. at 250–54 (discussing statute’s history and prior interpretation by the Department of Justice). Aside from delegation, the Court focused on the IRS’s lack of expertise on health-insurance policy,100King, 135 S. Ct. at 2489. suggesting that the Court had, once again, retreated from Mead and City of Arlington’s sole focus on formalized authority and action.101But see Stephanie Hoffer & Christopher J. Walker, Is the Chief Justice a Tax Lawyer?, 2015 Pepp. L. Rev. 33, 46 (“We do not know yet if the Court (or the lower courts) will extend this sweeping change in administrative law to other regulatory contexts” or whether King v. Burwell’s “new major questions doctrine may well be good for tax only.”).

Accordingly, the Supreme Court has created a complex framework for determining whether Chevron applies to agency interpretations. Congressional delegation of interpretive primacy to an agency undoubtedly has a preeminent place in the Court’s calculus. But how courts are to discern such a delegation depends on the authority of the agency to act with the force of law, the formality and procedure involved in the agency action, agency expertise, and the nature of the legal interpretation at issue. Similarly, the Court has referred to other values that influence Chevron’s applicability or judicial deference to the agency under any framework, such as the long-standing nature of the agency’s interpretation (albeit with repeated proclamations that agencies are eligible for Chevron after changing interpretations), political accountability, or congressional acquiescence. And scholars have added other values: national uniformity102Peter L. Strauss, One Hundred Fifty Cases per Year: Some Implications of the Supreme Court’s Limited Resources for Judicial Review of Agency Action, 87 Colum. L. Rev. 1093, 1112 (1987) (considering the values of national uniformity that Chevron furthers). and contemporaneity of an agency’s interpretation with statutory enactments.103Bamzai, supra note 32, at 941 (referring to respect that judiciary gave to consistent and contemporaneous executive interpretations as a canon of construction). Indeed, much like Justice Breyer, Evan Criddle has argued that deference under Chevron requires the satisfaction of all leading theoretical grounds—delegation, expertise, political accountability, rationality, and uniformity; anything less calls for Skidmore’s framework.104Criddle, supra note 3, at 1275. Part VI assesses what purchase these leading values and other contextual factors, aside from the canonical delegation theory, have on the federal circuit courts.

C.     Chevron for Thee, But Not for Me

Perhaps given Chevron’s complexity, the Supreme Court has not been consistent. In two empirical studies, Eskridge and Baer found that the Supreme Court applied Chevron deference only one-quarter of the time in which it would have seemed to apply.105Eskridge & Baer, supra note 10, at 1124–25. These findings were also generally consistent with Tom Merrill’s earlier findings that the Court used Chevron “only about one-third of the [time].”106Merrill, supra note 10, at 982.

The Court’s questionable loyalty to Chevron suggests that the doctrine is not meant to discipline Supreme Court decisionmaking. Instead, the doctrine may better serve to control lower courts and provide nationwide uniformity.107See, e.g., Christopher J. Walker, Chevron Deference and Patent Exceptionalism, 65 Duke L.J. Online 149, 156–58, 161–62 (2016), [] (making this argument in the context of the Federal Circuit and agency interpretations of substantive patent law). After all, the Court itself recently referred to Chevron as serving a “stabilizing purpose” to prevent “[t]hirteen Courts of Appeals [from] applying a totality-of-the-circumstances test [that] would render the binding effect of agency rules unpredictable.”108City of Arlington v. FCC, 133 S. Ct. 1863, 1874 (2013). And perhaps most importantly for lower courts, it did so in the context of rejecting the dissent’s more provision-specific inquiry and, à la Mead, looking only at force-of-law authority.109See id.

II.      Empirical Study of Judicial Deference

Before exploring our findings in Parts III through VI, we provide in this Part a brief overview of key empirical studies of Chevron in the Supreme Court and the circuit courts, our methodology, and an overview of the composition of our dataset.

A.     Prior Empirical Studies of Chevron

Our purpose here is to identify the basic methodology of the prior studies that were relevant to our methodological choices and results. We discuss our or others’ specific findings or methods in more detail as relevant to our discussion in subsequent Parts.

1.       Key Studies Concerning the Supreme Court

The most comprehensive study of deference in the Supreme Court comes from Eskridge and Baer. Their study provided the model for our Codebook and many of our variable fields, and it provides much of the data to compare deference in the Supreme Court and courts of appeals. In their study, they evaluated all Supreme Court decisions reviewing an agency’s statutory interpretation between the issuance of Chevron in 1984 and Hamdan v. Rumsfield in 2006.110Eskridge & Baer, supra note 10, at 1094 (referencing Hamdan v. Rumsfield, 548 U.S. 557 (2006)). They coded 1,014 interpretations for 156 variables111Id. at 1090. to describe which deference regimes the Supreme Court applied and how it did so.112See id. at 1099 tbl.1. As most relevant here, they found that agencies prevailed 68.3% of the time, without regard to deference regimes.113Id. at 1100. The Court applied Chevron in only 8.3% of the decisions, and agencies prevailed 76.2% of the time under Chevron.114Id. at 1099 tbl.1. Skidmore, for its part, applied to 6.7% of the decisions, and agencies prevailed 73.5% of the time,115Id. a rate that was significantly higher than the 60.4% win rate in Kristin Hickman and Matthew Kreuger’s earlier study that considered Skidmore in the circuit courts.116Hickman & Krueger, supra note 36, at 1275. Eskridge and Baer also considered various “ad hoc” variables to which the courts have referred in their deference decisions (some of which were also factors in Barnhart)117Eskridge & Baer, supra note 10, at 1157.: the interpretation’s longevity, the subject matter, whether the interpretation concerned certain sensitive issues (such as preemption and jurisdiction), and factors that the Court referred to in its decision (such as congressional acquiescence, agency procedures, rulemaking authority, etc.).118See id. app. at 1203–16.

One other study is relevant to our findings here. To determine the impact of the so-called Chevron revolution in the years shortly before and after Chevron, Tom Merrill considered all decisions in the Supreme Court from 1984 to 1990 in which at least one justice identified a question of whether the Court should defer to an agency’s statutory interpretation.119Merrill, supra note 10, at 980–81, 981 tbl.1. He determined that, similar to Eskridge and Baer’s findings, the Court applied Chevron in only about one-third of the applicable decisions,120See id. at 982. and, surprisingly, the agencies had a lower win rate after Chevron (both for the aggregate of the decisions reviewed and for only those decisions in which the Chevron framework applied), despite Chevron’s reputation for being agency friendly.121See id. at 981 tbl.1, 982 tbl.2, 984. Finally, he found that the Court referred less often to traditional deference factors (such as the interpretation’s long-standing nature and contemporaneity, congressional ratification, etc.) after Chevron.122Id. at 984–85, 1018–22. Another study by Thomas Miles and Cass Sunstein concluded, after reviewing all Supreme Court decisions from 1989 to 2005 that applied the Chevron framework, that Chevron did not constrain judges from validating agency interpretations in accordance with the justices’ ideological preferences. Miles & Sunstein, supra note 11, at 825–26. Because Miles and Sunstein focused almost exclusively on Chevron’s effect on judicial ideology, their study has only minimal relevance for our findings here. To be sure, our dataset also includes coding for all of these ideological variables as to the agency’s decision, the composition of the judicial panel, and the court’s decision. Those findings will be explored in subsequent work.

2.       Key Studies Concerning Circuit Courts

More studies concern the courts of appeals, each with its own limitations on matters of judicial deference because of each project’s focus. The largest study, by Peter Schuck and Donald Elliott, considered all administrative actions on direct review in the courts of appeals in various time periods (1965, 1974–1975, 1984–1985, and 1988) to obtain a descriptive account of how judicial review of agency action works in the lower courts—both before and after Chevron.123Schuck & Elliott, supra note 17, at 989–90. Although we refer to some of their findings in our discussion below, their data, despite concerning nearly 2,500 published and unpublished decisions on direct review,124See id. at 989 n.13, 990 n.14. are of limited comparative use here. As prior scholars have noted, Schuck and Elliott neither limited their data to judicial review of agency statutory interpretations (and, relatedly, whether courts validated agency interpretations) nor indicated how many interpretations were upheld or reversed.125See supra note 17. Moreover, their more-than-a-quarter-century-old study occurred a decade before the Court’s reaffirmation of the Skidmore and Chevron dichotomy, after which one would expect that courts might decide decisions under the two regimes differently. Finally, they primarily focused on whether Chevron affected agency-remand rates, as compared to our focus on acceptance rates of agency interpretations specifically.126A later study by Linda Cohen and Matthew Spitzer found “a slight increase in the appellate affirm rate in the mid-1980s,” although “less dramatic . . . than . . . found by Schuck and Elliott.” Cohen & Spitzer, supra note 17, at 103.

Two other studies concerning the circuit courts largely focus on the role of judicial ideology, and thus they are of limited relevance of our findings here. In one study, Tom Miles and Cass Sunstein considered all published circuit-court decisions from 1990 to 2004 that applied Chevron to legal interpretations by the EPA and the NLRB.127Miles & Sunstein, supra note 11, at 825, 848. Because their purpose was to investigate whether Chevron limited political ideology from affecting judicial review, they focused on “two important agencies known for producing politically contentious decisions.”128Id. at 848. A second study by Frank Cross and Emerson Tiller considered the “whistleblower effect” in Chevron decisions in the circuit courts—that is, the effect of having one judge of a different ideology on a panel who can alert others to the majority’s failure to adhere to Chevron.129Cross & Tiller, supra note 14, at 2159–60. To do so, they considered all decisions from the D.C. Circuit from 1991 to 1995 that cited Chevron.130See id. at 2168. But, similar to Miles and Sunstein’s study, their focus on one particular question and decisions from only one court limits the study’s comparative value to the issues that we address here.

Finally, Orin Kerr’s study had broader data than the studies above, providing us more comparative findings. Kerr sought to evaluate how well three leading models (contextual, political, and interpretive) describe Chevron in the circuit courts.131Kerr, supra note 17, at 3–4. To that end, he reviewed 253 agency statutory interpretations in published circuit-court decisions from 1995 and 1996 that applied the Chevron framework.132Id. at 18–20, 30. Unlike Schuck and Elliott, who considered all administrative actions and remand rates, he counted interpretations and considered how often agencies prevailed on each interpretation.133Id. But similar to Shuck and Elliott, he considered only direct review of agency actions.134Id. at 19–20. His data concerning the “overview of Chevron” and the contextual model (similar to the traditional or Barnhart factors that Eskridge and Baer as well as Merrill considered) provide the most relevant comparisons to our findings here,135See id. at 30–35, 31 fig.A, 32 charts 1 & 2, 33 chart 3, 35 fig.B. despite his shorter timeframe and smaller population of interpretations.136Two final studies are relevant, but their different methods of reporting data render it difficult to compare our data to theirs. First, Jason Czarnezki evaluated how the circuit courts employed Chevron in environmental decisions from 2003 to 2005. Czarnezki, supra note 16, at 769. Like us, he searched for decisions that cited Chevron. See id. at 784 n.96. But because of his significant focus on judicial ideology, he reported his findings in terms of individual judges’ votes. See id. at 790–803. Second, Richard Re considered whether the circuit courts applied the Chevron framework as the traditional two steps or as one “reasonableness” step. Richard M. Re, Should Chevron Have Two Steps?, 89 Ind. L.J. 605, 637–39 (2014). He reviewed all published circuit-court decisions that cited Chevron in 2011. Id. But, as with Czarnezki’s study, Re reported his data in a different manner than us, rendering comparison of our data with his difficult or impossible. Nonetheless, we compare findings or consider their insights when possible.

B.     Our Study Design and Methodology

For our study, we completed several related searches on Westlaw to attempt to capture all published decisions over an eleven-year period in which the circuit courts referred to Chevron. In contrast to prior studies’ focus on certain courts, agencies, or issues, we considered all circuits and attempted to include interpretations from all federal agencies. Likewise, in contrast to short time frames in most other studies,137See supra note 17 and accompanying text. we selected an eleven-year period from January 1, 2003, until December 31, 2013. We began our study with decisions from 2003 to ensure that courts and parties had sufficient time to become accustomed to (1) instructions in the Supreme Court’s 2001 United States v. Mead decision that clarified Chevron’s applicability and Skidmore’s renaissance,138United States v. Mead Corp., 533 U.S. 218 (2001). and (2) certain factors for Chevron eligibility that the Supreme Court mentioned in dicta in 2002’s Barnhart v. Walton.139535 U.S. 212, 222 (2002). The significant time period also permits us or others to consider in future work any differences that may exist during the Bush and Obama Administrations and the changing memberships on the circuit courts that occurred based on retirements and new appointments. Such a large dataset allowed us to draw more meaningful conclusions concerning the circuit courts collectively, each circuit individually, and each agency with a significant number of reviewed interpretations.

Searching for all instances of judicial review of agency statutory interpretations, as Eskridge and Baer did, was not feasible for the appellate courts. Instead, we narrowed our parameters by focusing on references to Chevron, similar to nearly all of the studies concerning circuit courts.140Miles and Sunstein did the same in their study of the Supreme Court. Miles & Sunstein, supra note 11, at 825 n.10. The key difference between our study and others is that other studies generally coded only cases that applied the Chevron framework, while we coded all cases that concerned statutory interpretation and referred to Chevron, whether or not the court applied the Chevron framework. Limiting ourselves to decisions that cited Chevron would give us the best data on Chevron itself and, because of Chevron’s prominence, extremely useful data on decisions in which judicial deference was in dispute. Although our dataset does not permit us to paint as complete a picture as to non-Chevron regimes as the Eskridge and Baer or Hickman and Krueger studies, we anticipated obtaining a sufficient number of applications to inform understandings of those doctrines.

We engaged in a broad, yet practicable, search for decisions. Simply searching for “Chevron” returned too many false positives, since Chevron Corporation is a party in numerous decisions and “Chevron” is part of the case style of numerous other, unrelated decisions. To identify the decisions concerning Chevron deference, we prepared searches that include “Chevron” with relevant terms: agency, ALJ, order, formal adjudication, rule, and 553 (the APA section that concerns notice-and-comment rulemaking).141We performed the following “advanced” searches on WestlawNext in the “CTA” (federal appellate cases) database: (1) (agency and Chevron /p “formal adjudication” ALJ order) & DA(aft 12-31-2002 & bef 04-01-2013); (2) (Chevron /p rule or 553) & DA(aft 12-31-2002 & bef 04-01-2013). We performed the same searches for the period from April 1, 2013, until December 31, 2013. To render it more likely that we captured all Chevron deference decisions even if the court did not use the word “agency” in its opinion, we ran similar, yet even broader, searches for approximately twenty-five federal agencies that we thought would be most likely to be involved in statutory interpretation.142We performed the following “advanced” search on WestlawNext in the CTA database: (chevron /p interp! and “[agency]”) & DA(aft 12-31-2002 & bef 04-01-2013)—for each of the following agencies: USDA, BIA, CFTC, DEA, DHS, EPA, Education, FCC, FDA, FERC, FLRA, FMC, FMSC, FTC, HHS, HUD, Interior, ITC, Labor, NLRB, NTSB, OSHA, SEC, SSA, Transportation, and the Postal Service. We performed the same searches for the period from April 1, 2013, until December 31, 2013. Although we did not perform any sampling to determine whether we likely captured all informal interpretations that may go by names other than rules or orders, we expected that the much broader term “interpretation” with the most well-represented agencies would likely capture most informal interpretations without the terms “guidance” or “letters”—and other forms of informal interpretations—that may lead to too many false positives. Because Mead indicated that Chevron is most applicable to formal adjudication and notice-and-comment rulemaking,143Mead, 533 U.S. at 230–31. we used terms especially designed to capture decisions concerning these methods of agency action. But our terms (e.g., rule, order, and interpretation) would also have captured less formal methods of agency action, such as interpretive rules or informal adjudications, to which Chevron can apply.144Similar to another study, we did not attempt to obtain decisions in which the courts should have applied Chevron but failed to do so or decisions that applied a Chevron-like review without citing Chevron. See Hickman & Krueger, supra note 36, at 1260 (concerning Skidmore). We have little reason to think that these decisions would materially change our findings, see id., and searching for these cases would have led to an impracticable review given the number of courts and years that our search considered. Unlike Schuck and Elliott and some of Kerr’s data, we collected decisions concerning direct and collateral review. We then combined all of the collected decisions into one database, removed duplicates, and removed a handful of obviously irrelevant authorities (such as unpublished decisions or treatises that were included in the results for unknown reasons). We ultimately had a database of 2,272 decisions.145Like Eskridge and Baer, we do not conduct significance tests because our database contains, to the best of our knowledge, the entire population of cases that interest us, as opposed to a sample of that population. Significance testing reveals the likelihood that variation with sampling occurs because of variation in the entire population, as opposed to random variation. Because we are reporting the results of a full population of interpretations, the variations exist with certainty. We, like Eskridge and Baer, can concentrate on why the variations in our findings exist and whether they are material. See Eskridge & Baer, supra note 10, at 1095–96.

Our research assistants initially reviewed the decisions, and we then completed a secondary review of every decision to increase uniformity and validity. In our secondary review, we divided the cases up randomly for one of us to review, and we flagged cases for a third-level review where the other then weighed in. One of us then conducted a more systemic review of the cases in preparing the dataset for analysis in the IBM SPSS statistics software. For all decisions with at least one instance of an agency’s statutory interpretation of a statute that it administers, we coded each instance of interpretation within one case as its own entry (as Kerr, Re, and Hickman and Krueger did, but Eskridge and Baer did not), meaning that one decision could have more than one entry in our dataset.146Although, as discussed below, we largely followed Eskridge and Baer’s lead in how to code certain information, one key difference is that they coded each decision only once and relied on a “mixed” category for when one decision had more than one issue of statutory interpretation. See id. app. at 1213. We had a total of 1,558 separate instances of statutory interpretation from 1,327 judicial opinions.

We coded the decisions on a spreadsheet for thirty-seven147The variables were as follows: (1) year, (2) circuit, (3) relevance, (4) authoring judge, (5) en banc, (6) dissent, (7) other opinions, (8) agency, (9) subject matter, (10) final decisionmaker, (11) agency interpretation’s political valence, (12) agency format, (13) the type of any informal interpretation, (14) continuity, (15) reason for evolution, if any, of agency’s interpretation, (16) congressional delegation, (17) jurisdictional issue, (18) regulatory interpretation with statutory interpretation, (19) state-law preemption, (20) foreign-affairs issues, (21) judicial interpretation’s political valence, (22) whether the court accepted the agency’s interpretation, (23) the outcome of the petition or challenge, (24) whether the court applied Chevron, (25) the application of Chevron step one, (26) the application of Chevron step two, (27) the court’s deference regime, if any, (28) consideration of agency expertise, (29) consideration of accountability, (30) consideration of national standards, (31) consideration of the consistency of an agency’s interpretation, (32) consideration of contemporaneity of the agency interpretation with statutory enactment, (33) consideration of public reliance, (34) consideration of rulemaking authority, (35) consideration of agency procedures, (36) consideration of congressional acquiescence, and (37) subsequent Supreme Court history. variables identified in our Codebook, which provided guidance to our reviewers in coding. In broad strokes, aside from relevance,148We identified as relevant only those decisions that concerned judicial review of an agency’s interpretation of a statute that it administers because Chevron applies only to those interpretations. We did not code agency interpretations of their own regulations or federal or state statutes that they do not administer. Likewise, we marked decisions as irrelevant if we were not able to discern whether the court was reviewing a matter of statutory interpretation or if the court did not command a rationale that commanded a majority, rendering further coding impossible. we coded the decisions for the following:

  • identifying information, such as the relevant circuit, judges, and additional opinions concerning statutory interpretation;
  • the nature of the agency’s interpretation, such as the agency, the subject matter, the agency’s format and final decisionmaker,149If, as was very common, the court merely identified the position at issue as that of the agency or a delagatee, we coded the final decisionmaker as the head of the agency. the political valence of the interpretation largely based on Eskridge and Baer’s definitions,150See Eskridge & Baer, supra note 10, at 1205–06 (defining coding for interpretations). To the interpretations coded as “liberal,” we added trade decisions that favored domestic industry. To the interpretations coded as “conservative,” we added trade decisions that favored foreign industry and instances in which companies accused of polluting the environment or violating business-regulating laws prevailed. and the long-standing nature of the agency’s interpretation or novelty (and reasons for any new interpretation);
  • whether the agency’s interpretation concerned certain sensitive topics, such as its own jurisdiction, state-law preemption, or foreign affairs; and
  • the nature of the judicial decision, such as the result for the agency, political valence of the court’s interpretation based on Eskridge and Baer’s definitions, the applied deference regime, how the court applied Chevron if it was the applicable regime, and traditional factors that Eskridge and Baer identified in their study.

As we discuss our findings in Parts III through VI, we will explain additional methodological matters as relevant to our findings’ implications. Because of the wealth of information provided by the raw numbers alone, in this Article we have chosen to present our findings descriptively, saving more sophisticated statistical analysis of the data for subsequent work. But before turning to our findings, it is important to note a number of significant methodological limitations inherent in our study design.

First, like most of the prior studies, we reviewed only published decisions based on the view that the courts were likely to designate decisions as published in which they reviewed a federal agency’s statutory interpretation and that they were likely to mark decisions as unpublished when they referred to their past review of agency interpretations as circuit precedent. As one of us has documented in an empirical analysis of constitutional litigation, it may well be the case that courts engage in strategic behavior by not publishing certain decisions.151Aaron L. Nielson & Christopher J. Walker, Strategic Immunity, 66 Emory L.J. 55 (2016). Although it is theoretically possible that a court could strategically not publish a decision that strikes down an agency statutory interpretation or refuses to apply Chevron deference, it is more likely that unpublished decisions both apply Chevron deference and uphold agency statutory interpretations. Accordingly, if anything, our findings likely underestimate the overall effect of Chevron deference in the circuit courts.

Second, we reviewed only circuit courts. To be sure, district courts also apply Chevron.152See, e.g., A.T. Massey Coal Co. v. Barnhart, 381 F. Supp. 2d 469, 484 (D. Md. 2005) (“Therefore, before conducting a Chevron deference analysis, this Court must first determine whether Chevron deference is even appropriate.”), aff’d sub nom. A.T. Massey Coal Co. v. Holland, 472 F.3d 148 (4th Cir. 2006); Fontana v. Caldera, 160 F. Supp. 2d 122, 128–29 (D.D.C. 2001) (“[T]his Court should apply Chevron deference to the ABCMR’s statutory interpretation.”), aff’d sub nom. Fontana v. White, 334 F.3d 80 (D.C. Cir. 2003). But we reasoned that many, if not most, review comes to the courts of appeals and significant district-court decisions would likely be appealed—though one could argue that the federal government may not appeal certain losses for fear of creating binding precedent. Similarly, our study can only evaluate the effect of Chevron deference with respect to agency statutory interpretations that actually make it to the circuit courts. Our prior experience litigating such cases suggests that regulated entities and individuals will often not waste resources to bring judicial challenges to agency statutory interpretations precisely because of the deferential standards of review. In other words, our findings may well underestimate the overall effect of Chevron deference on agency interpretive practices.153Indeed, one of us has surveyed 128 agency rule drafters about how they interpret statutes and draft regulations and found some support for the intuitive proposition that a federal agency is more aggressive in its interpretive efforts if it is confident that Chevron deference (as opposed to Skidmore deference or de novo review) applies. See Christopher J. Walker, Inside Agency Statutory Interpretation, 67 Stan. L. Rev. 999, 1059–65 (2015); see also Walker, supra note 94, at 721–28 (exploring these findings in detail).

Third, we culled only those decisions in which courts invoked Chevron by name. Our database did not include instances in which they referred to a Chevron-like doctrine by another name, including the name of a circuit or Supreme Court precedent that functioned similarly,154For instance, the Supreme Court granted the NLRB “substantial deference” and deemed an NLRB interpretation “reasonable” without mentioning its recent Chevron decision. Pattern Makers’ League of N. Am. v. NLRB, 473 U.S. 95, 100 (1985). Likewise, before Chevron, the Supreme Court applied a Chevron-like doctrine to Federal Reserve interpretations of the Truth in Lending Act. Ford Motor Credit Co. v. Milhollin, 444 U.S. 555, 565 (1980) (requiring courts to accept TILA regulations that were not “demonstrably irrational”). or failed to refer to Chevron based on inadvertence of strategic behavior.155Similarly, in an attempt to obtain all of the relevant circuit-court decisions, we included decisions that the Supreme Court later reviewed (whether or not the Court affirmed, reversed, or vacated the lower-court judgment). But we did not include panel decisions that the en banc court vacated on the theory that the en banc court’s decision provided a better guide of the circuit’s practice. Nor does our dataset include cases where Chevron was not mentioned at all. This approach limits our ability to compare agency-win rates under Chevron and other deference standards, though the findings of other studies on Skidmore deference and de novo review are consistent with our findings. For instance, one could imagine instances in which a reviewing court intentionally does not cite Chevron deference when setting aside an agency statutory interpretation. Perhaps more likely, though, are situations where the federal agency strategically decides not to invoke Chevron deference for fear of limiting Chevron’s domain in future cases of similar procedural posture (less-formal procedures) or substantive position (major constitutional or policy questions). Or, conversely, in easy interpretation cases, the agency may forgo more formal procedures and thus not request any deference.156See, e.g., Christopher J. Walker, How to Win the Deference Lottery, 91 Tex. L. Rev. See Also 54, 79 (2012), [].

To further address this methodological limitation, we coded separately every published circuit-court decision that cited Skidmore during that same eleven-year period (2003–2013). The total number of such cases was 168, of which only 55 were deemed relevant. Because this Article focuses on how circuit courts cite and use Chevron, we have decided not to include these Skidmore-only cases in our description of the findings. Instead, we separately note, where helpful for comparison purposes, the findings for the Skidmore-only cases.

Fourth, although we largely based our coding on Eskridge and Baer’s model, our comparative findings are each based on different periods of time. Their study considers decisions from approximately 1985 until 2006, while ours considers decisions from 2003 until 2013. Likewise, one should remember that other studies of Chevron in the circuit courts, to which we often compare our data, consider different time periods, or sometimes limited circuits or subject matters.

Fifth, consistent coding is inherently difficult because of the large number of decisions and the judgments required in the face of unclear judicial language. To mitigate these concerns, we included several procedural checks in our study design, such as continual communication with reviewers during the review process, a secondary review by only the two of us (with significant communication during the secondary review), and a tertiary systemic review in the IBM SPSS statistics software to check for inconstancies across cases. A number of our coding variables facilitated this systemic review since they required consistent answers. We also cross-checked numerous coding fields to improve consistency after our secondary review.157These are, of course, not the only methodological limitations. For instance, any attempt to draw inferences about the legal system from a selection of trial or appellate opinions has problems. See, e.g., Theodore Eisenberg & Michael Heise, Plaintiphobia in State Courts Redux? An Empirical Study of State Court Trials on Appeal, 12 J. Empirical Legal Stud. 100, 100–01 (2015); Theodore Eisenberg & Stewart J. Schwab, What Shapes Perceptions of the Federal Court System?, 56 U. Chi. L. Rev. 501, 503–04 (1989); George L. Priest & Benjamin Klein, The Selection of Disputes for Litigation, 13 J. Legal Stud. 1, 1 (1984). The purpose of this study, however, is not to draw inferences about the larger legal system but to better understand how circuit courts apply administrative law’s deference doctrines—recognizing that there are selection effects embedded in the sample of legal disputes that reach the appellate stage.

C.     Overview of Our Dataset

Before presenting our findings in Parts III through VI, it is helpful to sketch out the composition of the dataset. As noted in Section II.B, our original set of 2,272 circuit-court decisions published from 2003 through 2013 resulted in 1,558 instances (from 1,327 decisions) in which a circuit court reviewed an agency statutory interpretation. The interpretations are evenly spread throughout the eleven-year time period.158There were 165 interpretations in 2003, 165 in 2004, 145 in 2005, 145 in 2006, 134 in 2007, 138 in 2008, 132 in 2009, 117 in 2010, 143 in 2011, 129 in 2012, and 145 in 2013. This does not necessarily mean that the use of Chevron deference has remained stable over time. As noted in Section II.B, courts may not always cite Chevron itself in light of strategic behavior, inadvertence, or reference instead to other circuit precedent that further develops the Chevron doctrine within that circuit under a different name. But they are not as evenly distributed by circuit, agency, subject matter, or format of agency procedure.

For example, nearly one in five interpretations came from the D.C. Circuit (19.7%), followed by the Ninth Circuit (16.9%), Second Circuit (11.0%), Third Circuit (8.5%), and Federal Circuit (7.9%).159Circuit-by-circuit differences are explored further in Part IV. Although scholars and practitioners likely expect the D.C. Circuit’s preeminence based on its role as “a de facto, quasi-specialized administrative law court,”160Christopher P. Banks, Judicial Politics in the D.C. Circuit Court, at xiii (1999). Schuck and Elliott found that, as late as the 1980s, the Federal Circuit reviewed the most agency decisions.161Schuck & Elliott, supra note 17, at 1017–18. As for subject matter, 30.6% of interpretations concerned immigration, perhaps explaining in part the Ninth Circuit’s disproportionate share of the interpretations in the dataset.162This finding is consistent with a prior empirical study of judicial review of immigration adjudications conducted by one of us. See Christopher J. Walker, The Ordinary Remand Rule and the Judicial Toolbox for Agency Dialogue, 82 Geo. Wash. L. Rev. 1553, 1584 (2014) (noting that “[t]he Ninth Circuit’s decisions constitute over 45 percent of the overall sample (154 of 342)” of nationwide circuit-court decisions); see also id. at 1585 n.153 (noting that in 2012, the Ninth Circuit decided 40.5% of all immigration adjudication decisions in the circuit courts). The environment (13.9%) and entitlement programs (8.9%) were the next most predominant subject matters. If the subject matters of employment, labor/collective bargaining, and pensions are combined, they represented 10.5%. The dataset’s breakdown by agency was similar.163We further explore subject-matter and agency-by-agency differences in Part V.

As for agency format, of the 1,558 interpretations, roughly a third resulted from notice-and-comment rulemaking (36.5%) and another third from formal adjudication (36.1%). With the latter, we included immigration adjudications and similar adjudications that perhaps are not APA-defined formal adjudications but nevertheless have been recognized by courts as being sufficiently formal to be accorded Chevron deference. Only four (0.3%) agency interpretations arose in formal rulemaking, whereas the remaining interpretations (24.8%) involved some sort of informal interpretation.164The effects of agency procedure are explored further in Section III.C. Because decisions by FERC were difficult to categorize, those 37 interpretations (2.4%) were treated as a separate category in the dataset.

Finally, nearly two-thirds (63.0%) of the agency interpretations were “conservative” under the Eskridge–Baer model,165This Eskridge–Baer model is defined in supra note 150. with 29.2% “liberal” and the remainder (7.8%) neutral, mixed, or otherwise too difficult to categorize. By contrast, only half (51.3%) of the court decisions on the agency statutory interpretation were “conservative,” with 40.9% “liberal” and the remainder (7.8%) neutral, mixed, or otherwise too difficult to categorize. In other words, the circuit courts tended to decide statutory interpretation issues more liberally than agencies.166Although we discuss these conservative–liberal results where relevant, many of those findings lie outside the scope of this Article. We intend to explore these results and the ideology of the panel judges in subsequent work.

III.    General Findings on Chevron in the Circuit Courts

We begin by considering three categories of findings: (1) agency-win rates under all standards of review and the frequency of standards of review in our dataset, (2) how circuit courts applied Chevron’s two steps when Chevron applied, and (3) agency-win rates under differing interpretive formats.

A.     Agency-Win Rates and Deference Differences

Consistent with prior studies, federal agencies in this study prevailed most of the time—in 71.4% interpretations—when we considered all interpretations together (that is, under any scope of review). None of the prior studies tracks perfectly with ours, but some provide limited comparison. For instance, Schuck and Elliott found that, in 1984 and 1985, the agency prevailed 76.7% of the time based on the overall outcome for the agency (meaning the result for the agency, whether or not the agency prevailed on the statutory-interpretation issue alone) in their review of circuit-court decisions of any agency action on direct review.167Schuck & Elliott, supra note 17, at 1008. Likewise, Eskridge and Baer found that the agency prevailed 68.3% of the time in their review of Supreme Court decisions from 1984 until 2006 in which an agency’s interpretation of a statute was at issue.168Eskridge & Baer, supra note 10, at 1094, 1100. And Tom Merrill’s review of similar Supreme Court decisions from 1984 until 1990 found that the agency prevailed 70.0% of the time.169Merrill, supra note 10, at 981 tbl.1. Perhaps counterintuitively, he also found that in the three years before Chevron, the Court agreed with agencies more often (75% of the time). Id. at 982 tbl.2.

The overall win rate differed somewhat depending on whether the agency statutory interpretation under review was “conservative” or “liberal” per the Eskridge–Baer model: “conservative” agency statutory interpretations were upheld 69.3% of the time (982 total “conservative” interpretations), whereas “liberal” interpretations were upheld 74.5% of the time (455 total “liberal” interpretations). In the remaining 121 interpretations where the agency interpretation was neutral, mixed, or otherwise too difficult to categorize, the agency won 76.0% of the time.170By comparison, among the 55 Skidmore-only decisions, “conservative” agency statutory interpretations were upheld 63.0% of the time (27 total “conservative” interpretations), whereas “liberal” interpretations were upheld 60.9% of the time (23 total “liberal” interpretations). In the remaining 5 interpretations where the agency interpretation was neutral, mixed, or otherwise too difficult to categorize, the agency won 40.0% of the time.

Similarly, in 74.8% of interpretations the circuit courts applied the Chevron deference framework.171With respect to “conservative” agency statutory interpretations, the courts applied the Chevron deference regime 73.8% of the time (725 of 982 interpretations), compared to 75.4% for “liberal” interpretations (343 of 455 interpretations). In the remaining 121 interpretations where the agency interpretation was neutral, mixed, or otherwise too difficult to categorize, the courts applied the Chevron deference standard 81.0% of the time. By contrast, they applied the Skidmore standard to 10.8% of the interpretations and refused to apply any deference (de novo review) to 7.5% of them. In the remaining interpretations (107 interpretations, or 6.9% of total interpretations), the courts declined to choose a deference standard, usually holding that the answer would have been the same under any standard. When discussing these deference-regime findings, care should be taken, especially in comparing the findings from this study with those of prior studies. This study looked only at decisions in which courts cited Chevron deference, so it is no doubt far from a complete picture of the Skidmore and no-deference precedent in the circuit courts. That said, our large number of Skidmore and no-deference decisions provides a meaningful understanding of judicial review of agency action more generally, even if not a complete picture. It is also probably reasonable to conclude that our study captures the vast majority of published circuit-court decisions during the time period where the agency requested Chevron deference (as one would assume that courts would typically address the deference question if a party raised it).

As detailed in Figure 1, the agency prevailed at a higher rate than the overall agency-win rate (77.4% to 71.4%) when the court determined that Chevron applied. Conversely, the win rate dropped considerably when the court did not apply the Chevron standard: 66.4% when the court refused to decide which standard applies; 56.0% under the Skidmore standard; and 38.5% when the court applied de novo review.172By comparison, among the 55 Skidmore-only decisions, the circuit courts applied the Skidmore framework in 100.0% of the cases, and the agency won 60.0% of the time—slightly higher than the 56.0% agency-win rate in our Chevron dataset when circuit courts decided to apply Skidmore deference.


Again, comparison between deference regimes based on the decisions reviewed should be done carefully, since the dataset only includes decisions in which circuit courts expressly mentioned Chevron deference. It would not include decisions in which the court only mentioned Skidmore or reviewed interpretations de novo without mentioning Chevron—perhaps decisions in which one may expect higher agency-win rates whose inclusion would alter the results that we found. But at least in instances in which the court recognizes Chevron expressly in its opinion, the application of the Chevron framework seems to make a meaningful difference as to whether agencies prevail on the interpretive question. Indeed, there was nearly a twenty-four-percentage-point difference in win rates when the circuit courts applied Chevron deference (77.4%) than when they refused to apply it (53.6%). The agency was twice as likely (77.4% to 38.5%) to prevail if the court applied Chevron deference as opposed to reviewing the interpretation de novo and nearly three-fourths more likely (77.4% to 56.0%) to prevail under Chevron than Skidmore. In other words, agencies won more in the circuit courts when Chevron deference applied, at least when the court expressly considered whether to apply Chevron deference.

These findings challenge certain conclusions based on earlier studies. Evaluating affirmance rates in the Supreme Court and circuit courts from earlier studies, Richard Pierce found that, as relevant here, the affirmance ranges for de novo, Skidmore, and Chevron review overlap: 66% for de novo review, 55.1% to 70.9% for Skidmore, and 64% to 81.3% for Chevron.173Pierce, supra note 5, at 83–84. Only one study—Eskridge and Baer’s study of the Supreme Court—provided data on de novo review. See id. He concluded that “a court’s choice of which doctrine to apply in reviewing an agency action is not an important determinant of outcomes in the Supreme Court or the circuit courts.”174Id. at 85. Contrary to his conclusion concerning the circuit courts, our findings suggest that agency-win rates are meaningfully different under different deference regimes.

Our findings concerning Chevron (77.4% agency-win rate) are within the range of prior affirmance rates for circuit courts—from 64.0% to 81.3%.175Id. at 84. But importantly, our findings indicate that the affirmance rate is significantly towards the upper end of the range, and they may have the most validity because our data were the only to consider all agencies and all circuit courts over more than a decade.176See id. (noting shorter time frames of earlier studies). Indeed, the lowest affirmance rate (from Miles and Sunstein’s study) only considered decisions from two agencies (the NLRB and EPA) that were perceived as ideologically contentious. See Miles & Sunstein, supra note 11, at 848. Our affirmance rate was also very close to the one that Re found for circuit-court decisions from 2011. See Re, supra note 136, at 639 (“[T]he government won about three-quarters of all Chevron cases.”). Interestingly, our agency-win rate is almost identical to the one that Eskridge and Baer found for the Supreme Court (76.2%).177Eskridge & Baer, supra note 10, at 1142 tbl.15.

Likewise, our finding that circuit courts agreed with agencies 56.0% of the time when Skidmore applied is consistent with three of four earlier studies finding relatively lower affirmance rates. Those three earlier studies found agency affirmance rates of 55.1% in 1965, 60.6% in 1975, and 60.4% from 2001 to 2005.178Pierce, supra note 5, at 84 (considering Schuck & Elliott’s study of Skidmore decisions in 1965 and 1975, and Hickman & Krueger’s study from 2001 to 2005). The consistency of findings within a range of approximately five percentage points suggests that one study of decisions from 1984 with the highest affirmance rate of 70.9% in the circuit courts is an outlier.179See id. (considering Schuck & Elliott’s study of Skidmore decisions in 1984). The 73.5% affirmance rate for Skidmore—the highest affirmance rate that Pierce considered—concerned the Supreme Court, not the courts of appeals. See id. at 83. Although our Skidmore data come only from decisions that also mentioned Chevron and not all decisions that applied Skidmore, we are comforted that our results are very similar to results from earlier studies that did include all Skidmore decisions.

Finally, the affirmance rates for the circuit courts’ de novo review only further support this view. Although the de novo affirmance rate was 66.0% in the Supreme Court (with no data available for the circuit courts),180See id. at 83–85. our data revealed that the circuit courts affirmed agencies’ interpretation only 38.5% of the time, at least when the court had cited Chevron. Our data, accordingly, suggest that the range of affirming agency interpretation is about forty percentage points from de novo to Chevron review and about twenty percentage points from Skidmore to Chevron. Contrary to Pierce’s conclusions based on earlier studies, agency-win rates do appear to differ significantly under different deference regimes.

Of course, these data cannot demonstrate a causal relationship between deference regimes and agency-win rates. For instance, courts could be strategically choosing deference regimes that more easily allow them to reach an outcome that matches their policy preferences. But one shouldn’t overstate this concern. First, Mead constrains judicial discretion by focusing heavily on force-of-law authority. Second, there is probably no perfect way to test for strategic behavior because we would have to know the “correct” result and compare that to the result that the court reached. That said, there are some ways to see if courts are seeking to get around Mead. One easy way would be to invoke Barnhart’s ad hoc factors or theoretical grounds for deference, but, as described in Section VI.C, they rarely do so. Indeed, the courts applied Chevron at almost identical rates to “conservative” and “liberal” agency interpretations.181See supra note 171 . Future work that considers ideology may provide more insight.

In sum, using agency-win rates as an admittedly less-than-perfect heuristic to assess the meaningfulness of deference regimes, as others before us have done, we see that deference regimes appear to matter.

B.     How Chevron Is Applied

Of the 1,558 total interpretations reviewed, the circuit courts applied the Chevron framework in 1,166 of them (74.8%). Of those 1,166 interpretations, the agency prevailed 902 times (77.4%). The more interesting questions, however, may concern how the circuit courts applied the two-step framework. In other words, how many decisions were decided at step one? How many were decided at step two? And, perhaps most importantly, what were the agency-win rates at each step? Figure 2 depicts the overall win/loss numbers at both steps, with the percentages reflecting the portion of the set of interpretations in which the circuit courts applied the Chevron framework.


Consistent with prior studies, the vast majority of agency interpretations (817 interpretations, or 70.0%) made it to step two.182See Kerr, supra note 17, at 31 (noting that out of 253 interpretations, 72.7% were resolved under a “reasonableness” inquiry under a Chevron analysis with only one step (72 interpretations) or under step two of a two-step inquiry (112 interpretations)). Notably, Merrill found almost the inverse in his study of Supreme Court decisions from 1984 to 1990—only 44% of Chevron decisions made it to step two. See Merrill, supra note 10, at 980–81. And an even greater percentage of interpretations that made it to step two (766 interpretations, or 93.8%) were upheld. Indeed, we found that the agency won slightly more under step two (whether the court describes its analysis as one of “reasonableness” in one step or two) than in an earlier study. In comparison to our agency-win rate of 93.8% under step two, Kerr found that agencies in 1995 and 1996 won at step two or in a one-step “reasonableness” inquiry a combined total of 84.7% (156 out of 184 interpretations) of the time.183Kerr, supra note 17, at 31. For our coding, we did not distinguish decisions that applied a Chevron “one step” from a “two step.” We attempted, instead, to categorize judicial decisions under a two-step framework. For comparative matters, we treated all of Kerr’s one-step “reasonableness” cases as getting to step two because we coded decisions that only referred to an agency’s interpretation as “reasonable” or “unreasonable” as reaching step two. To be sure, it is not true that Chevron, at least as an empirical matter, has collapsed into just one step of statutory ambiguity.184Compare Matthew C. Stephenson & Adrian Vermeule, Chevron Has Only One Step, 95 Va. L. Rev. 597, 598 (2009), with Kenneth A. Bamberger & Peter L. Strauss, Chevron’s Two Steps, 95 Va. L. Rev. 611, 612–13 (2009), Re, supra note 136, at 608–09, and Christopher J. Walker, Avoiding Normative Canons in the Review of Administrative Interpretations of Law: A Brand X Doctrine of Constitutional Avoidance, 64 Admin. L. Rev. 139, 174 n.163 (2012). In particular, fifty-one agency statutory interpretations in our dataset—6.2% of those cases that made it to step two—were deemed unreasonable even though the court found the statute to be ambiguous as to the question at issue.

What happens at step one is perhaps even more noteworthy. Courts decided 30.0% of interpretations at Chevron’s step one. This finding is consistent with Kerr’s earlier finding that circuit courts resolved 27.2% of Chevron deference interpretations at step one.185See Kerr, supra note 17, at 31 fig.A (noting that 69 interpretations were resolved at step one out of 253 interpretations or 27.2%). But step-one resolution did not mean that the agency lost. Our data indicated that the agencies still prevailed 39.0% of the time, meaning that the agency’s interpretation was the only possible one under the statute. Our finding, once again, is similar to the 42.0% win rate Kerr found.186Id. This number is roughly the same as the agency-win rate when the circuits reviewed interpretations de novo (38.5%).

Nevertheless, limited data on the Supreme Court differ. Tom Merrill found that the Court resolved matters in agencies’ favor 59.0% of the time at step one over seven Terms,187Merrill, supra note 10, at 981 tbl.1 (considering decisions from 1984 to 1990). significantly more often than the circuit courts in our study. To put these numbers in perspective, Figure 1 is reproduced here as Figure 3, but now with Chevron step one and step two broken out into distinct standards.

These findings concerning Chevron step one—where the Court finds that the statute has only one clear meaning—are important for at least two reasons. First, a step-one ruling in favor of the agency cements the agency’s current interpretation in place, such that subsequent presidential administrations will not be able to change positions. Nor may the agency change positions based on changed circumstances short of statutory amendment. Second, these findings suggest that the circuit courts may well be taking the Brand X Court’s lesson that they clarify the nature of their holding to ensure that the agency knows whether it has discretion (or not) to change its statutory interpretation in the future.188See Nat’l Cable & Telecomms. Ass’n v. Brand X Internet Servs., 545 U.S. 967, 982–83 (2005) (“Only a judicial precedent holding that the statute unambiguously forecloses the agency’s interpretation, and therefore contains no gap for the agency to fill, displaces a conflicting agency construction.”). We anticipate further investigating the effect of Brand X in future work.


C.     Rulemaking Versus Adjudication

As detailed in Section I.B.1, a trilogy of Supreme Court decisions has suggested that the formality of the agency procedure may affect the level of deference accorded to the agency statutory interpretation. Moreover, those decisions suggest that all legislative rulemaking and formal adjudication should be treated similarly. The scholarship and empirical studies on point are plentiful. Our dataset sheds substantial empirical light on the role of formality in Chevron in the circuit courts.

1.       Agency Procedure and Overall Agency-Win Rates

Roughly a third (36.5%) of the 1,558 interpretations in the dataset resulted from notice-and-comment rulemaking and another third from formal adjudication (36.1%).189In this latter category, we included immigration adjudications and similar adversarial adjudications where courts have consistently applied Chevron deference, even if they are not APA-defined formal adjudications—that is, adjudications that a statute requires to be “on the record after opportunity for an agency hearing.” 5 U.S.C. § 554(a) (2012). In this Section, we break out immigration decisions at times to demonstrate their impact on the overall findings, especially as immigration accounts for nearly a third (30.6%) of the total dataset. Only 4 (0.3%) agency interpretations arose in formal rulemaking (rulemaking required by statute to be “on the record after opportunity for an agency hearing”190Id. § 553(c).)—a finding that reinforces the well-settled understanding that formal rulemaking “has become almost extinct” since 1973.191Aaron L. Nielson, In Defense of Formal Rulemaking, 75 Ohio St. L.J. 237, 253 (2014) (discussing United States v. Fla. E. Coast Ry., 410 U.S. 224 (1973)). The remaining interpretations (24.8%) involved some sort of informal adjudication. Due to difficulty in coding as rulemaking or adjudication, the 37 FERC interpretations (2.4%) were treated as a separate category in the dataset and not addressed in the following discussion.192We return to the FERC interpretations in Part V in the agency-by-agency analysis.

Perhaps in light of the Supreme Court’s focus on procedural formality, it should be no surprise that agencies win nearly three-fourths of the time when their interpretation is the product of notice-and-comment rulemaking (72.8%) or formal adjudication (74.7%). By contrast, the win rate falls to 65.0% when an agency uses a less formal means. (The win rate for formal rulemaking is only 50.0%, but with only 4 interpretations in the dataset, one should read little, if anything, into that finding.) Figure 4 depicts these findings.193By comparison, among the Skidmore-only cases, 50 of the 55 interpretations dealt with agency interpretations made in a less-formal means, with the remaining 5 interpretations dealing with agency interpretations in formal adjudications. Agencies prevailed in 62.0% of informal interpretations (31 of 50) and 40.0% of interpretations made in formal adjudication (2 of 5).

Among the 562 interpretations from formal adjudication, however, there are 386 immigration interpretations, whose agency-win rate is lower (70.2%). If those interpretations are removed, the agency-win rate in formal adjudication rises to 84.7%—more than ten percentage points greater than the rate for informal rulemaking.194Schuck and Elliott’s data are not easy to compare to ours because they considered all administrative decisions on direct review—whether legal or factual—and coded only remand rates to the agency, regardless of whether the courts agreed with an agency’s statutory interpretation, if any. See Schuck & Elliott, supra note 17, at 990 (noting that the coded decisions concerned agency actions on direct review and discussing criticism for case-selection and remand-variable methodology). But their data also indicated that agency adjudications prevailed more often in the circuit courts than rulemakings in three separate time periods: 1965, 1975, and 1984–1985. Id. at 1021–22. A key difference, however, is that the comparative win rates were more disparate when the rates for the three time periods were combined: 57.8% for adjudications and 43.9% for rulemakings. See id. at 1022. This difference may not be surprising because many of the adjudications would presumably have presented only factual matters for judicial review under very deferential substantial-evidence or arbitrary-and-capricious standards. 5 U.S.C. § 706.


Our results for rulemakings and informal interpretations are similar to those that Eskridge and Baer found for the Supreme Court. They found that legislative rules and executive orders prevailed 72.5% of the time under all deference regimes combined,195Eskridge & Baer, supra note 10, at 1147, 1148 tbl.17. which is almost the exact same as our finding of 72.8% in the circuit courts. Similarly, our agency-win rate for informal interpretations of 65.0% in the circuit courts is similar to theirs of 68.1% in the Supreme Court.196Id.

But findings concerning formal adjudication significantly differ. In contrast to our finding that agencies prevailed slightly more often in formal adjudication (74.7% including immigration cases and 84.7% without them) than in informal rulemaking (72.8%), Eskridge and Baer found that agency interpretations in formal adjudication prevailed in the Supreme Court only 65.4% of the time, slightly less than the win rate of 72.5% for agency interpretations from rulemakings.197Id. Likewise, our findings indicated that, depending on whether one includes immigration interpretations, agency interpretations from formal adjudication prevailed slightly or significantly more often than the overall agency-win rate for all formats (71.4%) in the circuit courts. Eskridge and Baer found the opposite, with the agency-win rate from formal adjudication in the Supreme Court slightly below the average win rate (68.8%).198Id. Although the nearly ten-percentage-point difference between our formal adjudication agency-win rates (our 74.7% to their 65.4%) is meaningful by itself, it is perhaps more appropriate to compare our 84.7% finding that excluded immigration decisions with their 65.4% finding because Eskridge and Baer indicated (without percentages or absolute numbers) that the two largest groups of formal adjudications that they considered were from the NLRB and the FLRA.199See id. (“Almost half of the formal adjudications reaching the Court come in labor cases (primarily orders from the NLRB and the Federal Labor Relations Authority).”). When we considered the FLRA’s and NLRB’s agency-win rates under formal adjudication, those rates in the circuit courts were higher than the average win rate for formal adjudication in the Supreme Court (65.4%). The 4 FLRA interpretations in our database all came from formal adjudication, and the agency had a 75.0% win rate. Of the NLRB’s 32 interpretations, 26 of them occurred in formal adjudication, and the NLRB had a win rate of 88.5% (winning 23 of 26). If these are the more appropriate comparative groups, then the difference in agency-win rates in formal adjudication increases by nearly twenty percentage points between the circuit courts and the Supreme Court.

2.       Agency Procedure and Chevron

Turning to whether the courts applied Chevron deference, we found that the data, depicted in Figure 5, are consistent with what one would expect from the Supreme Court precedent in one respect but not another. The findings are consistent with Mead in that formal interpretations receive Chevron deference at higher rates than informal interpretations, but they are inconsistent with expectations that rulemaking and formal adjudication are treated the same.

As detailed in Figure 5, the circuit courts applied the Chevron framework in 91.9% of notice-and-comment rulemakings and in all 4 formal rulemakings in the dataset. For formal adjudication, however, courts applied the Chevron deference framework to only 76.7% of interpretations. Not surprisingly based on Mead and Christensen’s preference for formal interpretations, the rate dropped significantly below 50% for informal interpretations (44.8%). (Nonetheless, although the Court had indicated in those cases that Chevron’s application would be rare for informal interpretations, the circuit courts apply Chevron nearly half the time.) Thus, formal interpretations (all rulemakings and formal adjudication) obtained the Chevron framework more frequently than informal interpretations. But notice-and-comment rulemaking obtained the Chevron framework fifteen percentage points more frequently than formal adjudication, despite their doctrinal parity under Mead.

Again, however, if the 386 immigration adjudications were removed from the formal adjudication category,200The circuit courts applied Chevron in 72.8% of immigration adjudications (281 of 386 interpretations). Of those 281 cases, the agency won 79.4% of the time. the frequency of applying Chevron deference to formal adjudications would rise nearly ten percentage points to 85.2% and bring the formal formats into closer parity. The difference between immigration decisions and other agencies’ formal adjudication likely arises in part because many immigration decisions are affirmed by only one Board of Immigration Appeals member (instead of the entire board), a procedure for which most circuit courts refuse to give Chevron deference.201See Lawson, supra note 51, at 591 (citing De Leon-Ochoa v. Attorney Gen., 622 F.3d 341, 349–51 (3d Cir. 2010)) (discussing split in circuit courts over whether to apply Chevron to decision from only one member of the BIA). By excluding immigration interpretations for which the BIA has an idiosyncratic review process, the Chevron-application differential between informal rulemaking and formal adjudication significantly shrinks to fewer than seven percentage points.


That Chevron deference applies almost as often to agency statutory interpretations promulgated in formal adjudication (when excluding immigration proceedings) as in notice-and-comment rulemaking may have important implications for administrative law. The Court has repeatedly held, most notably in SEC v. Chenery, that agencies have extremely broad discretion to choose whether to engage in rulemaking or adjudication.202NLRB v. Bell Aerospace Co., 416 U.S. 267, 294 (1974); SEC v. Chenery Corp., 332 U.S. 194, 203 (1947). Lower courts have rarely found that an agency abuses its discretion in selecting formats. See M. Elizabeth Magill, Agency Choice of Policymaking Form, 71 U. Chi. L. Rev. 1383, 1408 (2004). In deciding whether to use rulemaking or adjudication, the agency may consider various factors, such as the benefits of case-by-case development, the novelty of the issue, or time constraints in fashioning an interpretation.203See Chenery, 332 U.S. at 202–03. But agencies may also ponder whether they will pay a price in using formal adjudication instead of rulemaking. Our findings indicate that they may pay only a slight price—with a slightly lower rate of obtaining Chevron—for choosing formal adjudication (at least without idiosyncratic procedures).

But this small price seems worth it when one considers the better agency-win rates for formal adjudication once the circuit courts apply Chevron. As detailed in Figure 6, formal-adjudication win rates increased to 81.7% when Chevron applied, compared to a 74.7% overall win rate for formal adjudications and a 51.9% formal-adjudication win rate when Chevron did not apply. Indeed, if the 281 immigration adjudications to which Chevron applied were excluded, the win rate would rise to 86.0% (compared to 81.3% under all standards of review combined). Notably, Kerr found in his study of circuit courts in 1995 and 1996 that agency interpretations from adjudication (of all formality stripes) prevailed 72% of the time under Chevron, a rate of almost ten or fourteen percentage points below our findings.204Kerr, supra note 17, at 30 n.138. Conversely, our win rates for notice-and-comment rulemaking under Chevron and under all scopes of review were about the same: 74.4% under Chevron and 72.8% overall (though only 54.3% when Chevron did not apply). This win rate is similar to the 74% that Kerr found for prevailing rulemakings on direct review in his study.205See id.

The difference in agency-win rates between formal adjudication (81.7% with all subject matter, or 86.0% without immigration) and notice-and-comment rulemaking (74.4%), seemingly absent in Kerr’s earlier findings, may cause agencies to consider adopting Chevron-eligible agency statutory interpretations in formal adjudication as opposed to the more time- and resource-intensive notice-and-comment rulemaking.206See, e.g., Aaron L. Nielson, BeyondSeminole Rock, 105 Geo. L.J. 943 (2017) (exploring how the elimination of Auer deference could lead agencies to use adjudication more to adopt Chevron-eligible statutory interpretations, as opposed to notice-and-comment rulemaking). As a caveat, our data do not take into account whether the agency is less or more aggressive in its interpretations depending on whether the agency uses formal adjudication or notice-and-comment rulemaking, although we have no reason to think that agency behavior differs. Because of the higher agency-win rate with formal adjudication, formal adjudication may well be the better option for agencies in more cases than agencies may first surmise.


Agency adjudication has had a rough go of it the past few decades. Rulemaking has increased in popularity as adjudication has come under fire for its comparative downsides207See Magill, supra note 202, 1398 & n.44; Schuck & Elliott, supra note 17, at 1013–14.: it is less efficient because it does not address numerous issues at once, is less appropriate for determining “legislative” facts with input from numerous interpreted persons,208See Richard E. Levy & Sidney A. Shapiro, Administrative Procedure and the Decline of the Trial, 51 U. Kan. L. Rev. 473, 479 (2003). Magill describes the nearly unanimous scholarly consensus against adjudication in the 1970s. See Magill, supra note 202, at 1403 n.69. provides case-by-case decisions that provide less prospective notice,209See Levy & Shapiro, supra note 208, at 480. relies on enforcement actions for compliance that may not be necessary for existing rules,210See id. at 480–81. targets one regulated party as a test case and creates a retroactive norm,211See Magill, supra note 202, at 1396. provides agencies less agenda control depending on how matters are docketed,212See Jason Webb Yackee & Susan Webb Yackee, Testing the Ossification Thesis: An Empirical Examination of Federal Regulatory Volume and Speed, 1950–1990, 80 Geo. Wash. L. Rev. 1414, 1472–73 (2012). and provides a less audible “fire-alarm” to enable congressional oversight.213See Bressman, supra note 80, at 2043–45. Both formal and informal adjudication face existential attacks based on their fairness to regulated parties.214See generally Kent Barnett, Against Administrative Judges, 49 U.C. Davis L. Rev. 1643, 1670–86 (2016) (discussing recent concerns over fairness in both informal and formal adjudication).

Nonetheless, adjudication has numerous benefits. Aside from those mentioned in Chenery II,215See supra note 202 and accompanying text. adjudication permits agencies to escape review by the White House’s Office of Information and Regulatory Affairs and thus obtain more independence,216See Daniel A. Farber & Anne Joseph O’Connell, The Lost World of Administrative Law, 92 Tex. L. Rev. 1137, 1162–63 (2014). conserves more onerous rulemaking resources by using adjudication, allows agencies to act in an incremental way with a light regulatory touch,217See Magill, supra note 202, at 1396–97; see also Anne Joseph O’Connell, Political Cycles of Rulemaking: An Empirical Portrait of the Modern Administrative State, 94 Va. L. Rev. 889, 896 (2008) (noting that agencies’ increased use of less formal interpretations suggests “that there are considerable costs to notice-and-comment rulemaking”). Indeed, one study found that the EPA’s significant notice-and-comment rules took three to five years to promulgate, and 75% of those rules were challenged in court. See Stephen M. Johnson, Ossification’s Demise? An Empirical Analysis of EPA Rulemaking from 2001–2005, 38 Envtl. L. 767, 771 (2008). But Anne O’Connell found that other agencies are significantly speedier. See O’Connell, supra, at 958–59. provides broad participation by interested parties,218See Magill, supra note 202, at 1440. permits retroactive standard setting when necessary,219See Richard J. Pierce, Jr., Seven Ways to Deossify Agency Rulemaking, 47 Admin. L. Rev. 59, 76 (1995). avoids onerous congressionally imposed constraints on rulemaking,220See, e.g., Jonathan R. Siegel, The REINS Act and the Struggle to Control Agency Rulemaking, 16 N.Y.U. J. Legis. & Pub. Pol’y 131, 181 (2013) (“However, if rulemaking becomes impossible because of the constraints of the REINS Act, agencies may be compelled to use adjudication instead.”). and has more flexibility than it is often given credit for.221See Barnett, supra note 214, at 1700–01 (citing William Funk, The Rise and Purported Demise of Wong Yang Sung, 58 Admin. L. Rev. 881, 892 (2006)) (describing flexibility for formal adjudication under the APA). And our data suggest that there is one more to add: better agency-win rates under Chevron.222Our review of eleven leading administrative law casebooks indicates that the relationship of (1) agency choice between notice-and-comment rulemaking and formal adjudication, and (2) deference or agency-win rates has gone unnoticed. None of the texts addressed this relationship in either their discussion of Mead or Chenery II. See, e.g., Daniel J. Gifford, Administrative Law: Cases and Materials (2d ed. 2010); Kristin E. Hickman & Richard J. Pierce, Jr., Federal Administrative Law: Cases and Materials (2010); Lawson, supra note 51; John M. Rogers et al., Administrative Law (3d ed. 2012).

Although we do not enter the rulemaking–adjudication debate here, our findings that agencies prevailed more frequently under Chevron in adjudication than rulemaking may matter to agencies. One of us previously surveyed 128 agency rule drafters.223Walker, supra note 94, at 703–04. Among more than twenty interpretive tools included in the survey, Chevron deference was reported by most agency rule drafters (90.0%) as being used when interpreting statutes and drafting regulations.224Walker, supra note 153, at 1020 fig.2. The vast majority of agency rule drafters surveyed thinks about judicial review when interpreting statutes and views their chances of prevailing in court as better under Chevron. “Indeed, two in five rule drafters agreed or strongly agreed—and another two in five somewhat agreed—that a federal agency is more aggressive in its interpretive efforts if it is confident that Chevron deference (as opposed to Skidmore deference or de novo review) applies.”225Id. at 1059–65; see also Walker, supra note 94, at 721–28, 722 fig.3 (exploring these findings in greater detail). To be sure, one must be cautious in drawing strong inferences from these data because the “somewhat agree” (as opposed to agree or strongly agree) responses predominated and some volunteered comments discounted the effect of judicial review.226See Walker, supra note 94, at 722–28. Moreover, these drafters were not addressing whether the presence of agency-win rates would alter agency formats. But they are candid indications that agencies think about standards of review and that those standards may affect their statutory interpretations.

Finally, it is worth noting the differences in agency-win rates for informal interpretations. The overall agency-win rate for informal interpretations was 65.0%. But when the circuit courts applied Chevron, the win rate rose to 78.6%—near the rate for formal adjudication (81.7%) and slightly better than notice-and-comment rulemaking (74.4%). And, when the courts refused to apply Chevron deference to informal interpretations, the win rate dropped to 54.0%, which again is similar to the win rate without Chevron for notice-and-comment rulemaking (54.3%) and formal adjudication (51.9%). Although our findings here do not demonstrate what relationship the review standards and the agency-win rates have, these findings, along with our findings concerning the application of Chevron to informal interpretations, suggest that agencies should seek Chevron deference for every interpretation, regardless of its formality. Although Christensen and Mead both suggested that Chevron’s application to informal interpretations would be rare, our findings indicate that courts apply the Chevron framework, if not as a matter of course, almost half the time (44.8%) in decisions in which the Court referred to Chevron. This is especially true, as we shall see, when the litigation is in the D.C. Circuit.227See infra notes 231–232 and accompanying text. And when courts applied the framework, the agency-win rate was extremely high, higher than that for even notice-and-comment rulemaking. Again, however, our data do not allow us to account for whether the agency is less or more aggressive in its interpretive efforts depending on the level of formality involved in the regulatory effort.

IV.     Findings on Circuit Disparities

As reported in Section III.A, our findings suggest that standards of review matter for agency statutory interpretations. Recall that the overall agency-win rate in the 1,558 interpretations—regardless of the deference standard applied—was 71.4%. Recall, too, that the agency-win rate increased to 77.4% for the 1,166 interpretations subject to the Chevron framework, which was significantly higher than the rate under Skidmore (56.0%) and de novo review (38.5%). These findings provide some support that Chevron deference matters in the federal circuit courts. In other words, there may well be a Chevron Supreme and a Chevron Regular. But disaggregating the data by circuit, as depicted in Figure 7, complicates this story.


As detailed in Figure 7, the overall agency-win rates varied significantly by circuit. The most deferential circuit was the First Circuit (82.8%), followed by the Tenth (78.5%) and Eleventh (75.5%) Circuits. The two circuits that specialize in administrative law—the D.C. (72.6%) and Federal (73.2%) Circuits—are right around the mean (71.4%) and median (72.2%). The least deferential was the Ninth Circuit (65.8%), followed by the Fifth (67.8%), Sixth (69.0%), and Third (69.9%) Circuits.228By comparison, the Skidmore-only cases are spread across ten of the circuits with the following agency-win rates by circuit: 55.6% in Second Circuit (9 total interpretations); 66.7% in the Third Circuit (3 interpretations); 80.0% in the Fourth Circuit (5 interpretations); 66.7% in the Fifth Circuit (3 interpretations); 50.0% in the Sixth Circuit (6 interpretations); 100.0% in the Eighth Circuit (2 interpretations); 50.0% in the Ninth Circuit (12 interpretations); 25% in the Tenth Circuit (4 interpretations); 100.0% in the D.C. Circuit (1 interpretation); and 70.0% in the Federal Circuit (10 interpretations). These results may not be too surprising based on one’s intuitions about the circuits and their reputations vis-à-vis the federal government.

Perhaps some of the agency-win rates differ in the circuits based on subject matter, but subject matter or other effects require careful inquiry in future work. For instance, we intuitively—and it turns out correctly—thought that the Ninth Circuit’s large number of immigration cases likely affected the agency-win rate. Indeed, agencies prevailed in immigration cases 55.9% of the time in the Ninth Circuit, ten percentage points fewer than in all cases within the Ninth Circuit (65.8%). Put differently, when immigration cases are excluded, the agency-win rate in the Ninth Circuit rises to 73.8%, much more in line with the median and mean circuit. By contrast, the agency-win rates in immigration cases were significantly higher in other circuits that also had large number of immigration cases: 82.4% in the Fifth Circuit, 73.2% in the Second Circuit, and 70.9% in the Third Circuit. Notably, the Fifth Circuit’s overall agency-win rate (67.8%) was significantly lower than its agency-win rate in only immigration matters (82.4%). In the Second and Third Circuits, the overall and immigration-specific agency-win rates were nearly identical. Other factors, such as political valence and panel effects, appear to have more influence. The key point is that readers should keep in mind that more sophisticated analysis is necessary to understand why various circuit disparities exist.

Assessing the circuits based on the frequency at which they applied the Chevron framework paints a somewhat different picture, as depicted in Figure 8. As to the frequency of Chevron’s application, five circuits were well above the average (74.8%) and median circuit (73.2%). The D.C. Circuit led the way by applying the Chevron standard to 88.6% of interpretations, followed by the First (87.9%), Eighth (85.7%), Federal (84.6%), and Fourth (80.6%) Circuits. The Sixth Circuit, by contrast, applied Chevron the least frequently, only 60.7% of the time. Five other circuits were below 70%.


And inside these Chevron-application statistics is another fascinating finding. The “Mead Puzzle” arises from the difficulty lower courts have had in determining whether informal interpretations have the force of law and thus are entitled to Chevron deference.229See supra Section I.B.1. All but two circuits refused to apply Chevron to informal interpretations (at least when the court referred to Chevron) more than 50% of the time.230Here is the breakdown of the rate at which an informal interpretation received the Chevron framework, from greatest to least: D.C. Circuit (80.7%, n=88), Eighth Circuit (57.1%, n=14), First Circuit (42.9%, n=7), Third Circuit (41.4%, n=29), Tenth Circuit (40.0%, n=20), Ninth Circuit (39.7%, n=63), Sixth Circuit (36.8%, n=38), Fourth Circuit (31.3%, n=16), Seventh Circuit (30.8%, n=13), Fifth Circuit (29.2%, n=24), Eleventh Circuit (28.0%, n=25), Federal Circuit (25.0%, n=12), and Second Circuit (16.2%, n=37). And the median circuit rate was 36.8%.

But the D.C. Circuit, a Chevron early adopter, applied the Chevron framework to informal interpretations 80.7% of the time, nearly twenty-five percentage points more often than the next circuit (the Eighth Circuit, at 57.1%), more than forty percentage points more than the median circuit, and approximately sixty-five percentage points more than the circuit least likely to apply Chevron in these cases (the Second Circuit, at 16.2%). Accordingly, the circuit that reviewed the most agency interpretations in our dataset does not appear to have found the “Mead Puzzle” enigmatic. This finding bolsters our early conclusion that agencies should seek Chevron deference even for informal interpretations;231See supra Section III.C. not doing so in the D.C. Circuit borders on malpractice.

But to appreciate the circuit-by-circuit effect of Chevron deference (regardless of an interpretation’s formality), one needs to compare the agency’s win rate overall with its win rate when courts applied the Chevron framework. The average win-rate difference for the dataset is six percentage points, with an overall win rate of 71.4% compared to a win rate of 77.4% when the court applied the Chevron deference framework.

Several circuits were dramatic outliers with respect to win-rate differential. The greatest difference came from the Sixth Circuit, where the overall win rate was 69.0%, whereas the win rate when Chevron applied was 88.2%—nearly twenty percentage points higher. Agency-win rates in the Second (72.5% to 83.2%) and Seventh (72.0% to 83.7%) Circuits were also more than ten percentage points higher when Chevron applied. In those circuits, it was harder to obtain Chevron deference, but, once obtained, the agency’s chances of winning improved considerably. Conversely, the win-rate differential was within three percentage points in six of the thirteen circuits: the First (82.8% to 84.3%), Eighth (75.5% to 76.2%), Tenth (78.5% to 81.3%), Eleventh (70.4% to 73.1%), D.C. (72.6% to 75.4%), and Federal (73.2% to 76.0%) Circuits. In other words, in those circuits, whether Chevron applies does not seem to meaningfully affect agency-win rates.

Of course, that may not be the correct inference from these data. For many of these circuits where there was little difference in win rate, that is because the court applied Chevron deference at such a high rate that the Chevron win rate and overall win rate were basically the same. Such a win rate could be the result of those circuits having imbued Chevron’s deference principles into judicial decisionmaking. Figure 9 attempts to tease out those nuances by comparing the win rate under Chevron versus the win rate when Chevron does not apply. The circuits are ordered left to right in Figure 9 starting with the circuits with the greatest difference with and without Chevron. The overall win rate when Chevron applied was 77.4%, as noted above. When Chevron did not apply, however, the win rate plummeted nearly twenty-five percentage points to 53.6%.

For the six circuits whose differential between win rates overall and win rates under Chevron were within three percentage points (First, Eighth, Tenth, Eleventh, D.C., and Federal Circuits), the numbers are quite different when comparing win rates with and without Chevron’s application. The differential in the D.C. Circuit, for instance, was over twenty percentage points (75.4% to 51.4%). Of all thirteen circuits, the Eighth Circuit (76.2% to 71.4%) was the only outlier with a difference of less than five percentage points, with the only other under ten percentage points being the Eleventh Circuit—just barely (73.1% to 63.2%). And, before leaving Figure 9, we note that the largest differences were striking: 48.8 percentage points in the Sixth Circuit, 36.5 points in the Fourth Circuit, 33.7 points in the Seventh Circuit, and 31.5 points for the Second Circuit.


To compare circuits, it is perhaps helpful to create a composite score of the three indicators of deference in our dataset: overall agency-win rate; frequency of Chevron framework; and win rate when Chevron applied. Table 1 takes the average of these three percentages and turns that into a composite score on a ten-point scale—with 10.00 being a perfectly deferential score where the agency always wins and the court always applies Chevron deference and 0.00 being a perfectly nondeferential score where the agency never wins and the court never applies Chevron. The circuit rankings for each of the three deference indicators are provided in parentheses.

Utilizing these composite scores, the First Circuit (8.38 out of 10.00) emerges as the most deferential circuit, followed by the Eighth (7.91), D.C. (7.89), Federal (7.79), and Fourth (7.74) Circuits. On the other end of the spectrum, the Ninth and Fifth Circuits (6.85) tie as the least deferential circuits, followed by the Third (7.21) and Eleventh (7.22) Circuits.

Prior studies have not considered similar circuit effects. But these effects can be meaningful for agencies and litigating parties. For instance, if an agency seeks to bring an enforcement action to test one of its statutory interpretations, the Ninth Circuit may not be the best place to do so. Moreover, if the agency is worried about receiving Chevron deference, the Eighth Circuit is a promising venue because agency-win rates are similar with or without Chevron. Or, if the agency is confident that it will receive Chevron deference, the Sixth Circuit appears promising because agencies have the highest win rates under Chevron in that circuit. Of course, these strategic decisions should involve more considerations, including the ideological valence of the advanced interpretation, the subject matter, the agency, and the regulated parties at issue. We investigate some of these findings in the next Part.


V.       Findings on Agency and Subject-Matter Differences

Just as the circuit disparities discussed in Part IV complicate the story regarding Chevron in the circuit courts, so do the differences uncovered in Chevron’s application by subject matter and agency. This Part turns to those findings. Section V.A explores differences based on subject matter, whereas Section V.B looks at agency-by-agency disparities. Section V.C focuses on the differences between executive and independent agencies. Our purpose here is to provide an overview of the data and brief discussions of the most noticeable findings. We leave for future work (whether ours or others’) to dive into the findings for individual subject matters and agencies.

A.     Subject-Matter Differences

Differences in agency-deference rates based on subject matter are particularly interesting in light of the extensive focus of late in the literature and the real world on administrative law exceptionalism—“the misperception that a particular regulatory field is so different from the rest of the regulatory state that general administrative law principles do not apply.”232Walker, supra note 107, at 149.

To better appreciate differences based on subject matter, Table 2 presents the composite scores for all subject matters where there were at least ten agency interpretations in the dataset. This composite score is based on the same methodology used as that for ranking circuits in Part IV, with the three indicators of deference in our dataset weighted equally: overall win rate; frequency of Chevron framework; and win rate when Chevron applied. Table 2 takes the average of these three percentages and turns that into a composite score on a ten-point scale—with 10.00 being a perfectly deferential score where the agency always wins and the court always applies Chevron deference and 0.00 being a perfectly nondeferential score where the agency never wins and the court never applies the Chevron standard. The rankings for each indicator are provided in parentheses.


As Table 2 indicates, the subject matters for which courts defer most often to agency interpretations included telecommunications (8.67), Indian affairs (8.33), federal government (8.18), pensions (8.17), education (8.15), health and safety (8.14), and entitlement programs (8.03). Conversely, the subject matters for which courts defer the least were civil rights (5.99), followed by housing (6.04), prisons (6.64), tax (6.74), and employment (6.96).

The findings depicted in Table 2 merit article-length treatment and further exploration. But this Section merely highlights a few noteworthy findings. For instance, it perhaps should come as no surprise that tax ranks nineteen out of twenty-two in light of entrenched tax exceptionalism. At 56.3%, moreover, tax was second to last in the rate at which circuit courts applied Chevron deference. Indeed, it was not until 2011 that the Supreme Court announced that certain IRS interpretations are entitled to Chevron deference233Mayo Found. for Med. Educ. & Research v. United States, 562 U.S. 44, 55 (2011). For more on tax exceptionalism, see, for example, Kristin E. Hickman, The Need for Mead: Rejecting Tax Exceptionalism in Judicial Deference, 90 Minn. L. Rev. 1537 (2006), and Stephanie Hoffer & Christopher J. Walker, The Death of Tax Court Exceptionalism, 99 Minn. L. Rev. 221 (2014).—a position that the Supreme Court may have qualified last year in King v. Burwell, at least with respect to questions of deep political or economic significance.234King v. Burwell, 135 S. Ct. 2480, 2489 (2015). Similarly, it is perhaps no surprise to see immigration in the latter half of the rankings in light of the current discussion regarding immigration exceptionalism.235For an overview, see Chris Walker, The Costs of Immigration Exceptionalism, Yale J. on Reg.: Notice & Comment (Feb. 9, 2016), [].

The range of circuit-court deference by subject matter is also worth underscoring. For instance, the overall agency-win rate ranged from 86.7% for Indian affairs to 50.0% for civil rights.236By comparison, the 55 Skidmore-only interpretations arose in 17 different subject matters, with 5 of those subject matters having at least 5 interpretations. The agency-win rates for those subject matters are 44.4% for collective bargaining/labor (9 interpretations); 62.5% for tax (8 interpretations); 88.9% for employment (9 interpretations); 60% for health and safety (5 interpretations); and 40% for immigration (5 interpretations). The rate of circuit courts applying Chevron deference ranged from 100.0% for federal government matters to 52.0% for housing. Similarly, when circuit courts applied the Chevron deference framework, agency-win rates ranged from 92.3% for agency interpretations involving pensions (as opposed to an overall win rate of just 76.5% for pensions) to 57.1% for prisons. (Strangely, agencies were more successful when Chevron deference did not apply in the prison context, with an overall 68.4% win rate.) These disparities based on subject matter complicate the findings discussed in Part III regarding how Chevron matters on the ground in the circuit courts.

The data reveal some differences and similarities between the treatment of various subject matters in the Supreme Court and the circuit courts. As for the key differences, Eskridge and Baer reported that interpretations concerning energy had the highest overall agency-win rate of 93.3%,237Eskridge & Baer, supra note 10, at 1145 tbl.16. but in the circuit courts the agency prevailed 60.0% of the time, rendering it one of the lowest-prevailing agencies. On the flip side, Indian affairs had the second lowest overall agency-win rate of 51.6% in the Supreme Court,238Id. but it had the highest rate of 86.7% in the circuit courts. At the same time, some subject matters performed about the same in the Supreme Court and the circuit courts: environmental (68.4% and 70.8%, respectively), immigration (67.7% and 67.9%), business regulation (77.1% and 79.2%), and transportation (78.6% and 79.1%).239See id.

Contrary to Eskridge and Baer’s conclusion as to the Supreme Court, we cannot conclude with much confidence that the circuit courts defer based on notions of their perceived institutional advantage or disadvantage. Eskridge and Baer divided subject matter with at least ten decisions in the Supreme Court into six categories (foreign affairs, technical and economic regulation, procedural rules, socioeconomic regulation, criminal law, and federal governance) and indicated which subject matters within each category performed better and worse than the overall average.240Id. They determined that the Supreme Court was generally more deferential to foreign affairs (despite being less deferential to immigration, one of only two subject matters in the foreign affairs category, than the overall average), technical and economic matters, and procedural rules; the Supreme Court was generally less deferential to subject matter in criminal, socioeconomic, and federal governmental matters. Eskridge and Baer concluded that the justices were less likely to defer to matters that were not viewed as technical and matters that the Court thought that it could just as easily answer.241Id. at 1144.

Our dataset does not tell the same story with as much certainty when we organize our data similarly to theirs. While the circuit courts’ treatment of foreign affairs and technical matters are similar to the Supreme Court’s (but less deferential in each category), the circuit courts do not appear to defer meaningfully more or less to social-economic regulations, and, in fact, they defer much more to issues concerning the federal government than the Supreme Court. Moreover, although the circuit courts defer less to criminal matters than the overall average, the agency-win rates in this category (70.0% for criminal law and 68.4% for prisons) are very close to the average (71.4%) and higher than the agency-win rate in the Supreme Court (62.3% for criminal law, the only category there).242Id. at 1145 tbl.16. Our dataset did not include sufficient cases for procedural matters in the circuit courts.

B.     Agency-Deference Rankings

Although there is an obvious overlap between analyzing deference based on subject matter and agency, disaggregating the deference rankings by agency provides some helpful additional granularity. Table 3 presents the composite scores for all agencies with at least ten interpretations in the dataset. This composite score is based on the same methodology used as that for ranking circuits in Part IV and subject matters in Section V.A, with the three indicators of deference in our dataset weighted equally: overall win rate, frequency of Chevron framework, and win rate when Chevron applied. Table 3 takes the average of these three percentages and turns that into a composite score on a ten-point scale. The rankings for each of the three deference indicators are provided in parentheses.

Similar to subject matters, Table 3 illustrates the disparate win rates by agency. The ICC/STB was the agency to which the circuit courts most deferred (9.38), followed by the FCC (8.67), Treasury Department (8.37), NLRB (8.26), Commerce Department (8.18), Defense Department/Armed Services (8.13), FDA (8.08), and Education Department (8.06). On the other end, the least-deferred-to agency was the EEOC (5.08), followed by HUD (5.19), Energy Department (6.21), FTC (6.74), Justice Department (6.77), IRS (6.78), and Bureau of Prisons (6.79).


We underscore that these agency composite deference scores have the potential to mask some of the underlying deference differentials. For instance, the FTC is the fourth-worst agency on the composite score despite having the second-                highest overall win rate (90.9%). That is because circuit courts applied the Chevron doctrine to only 36.4% of the FTC’s statutory interpretations; indeed, the FTC’s win rate fell to 75.0% when the circuit courts applied Chevron.243This refusal to apply Chevron to FTC interpretations may be tied to a history of antitrust exceptionalism. See, e.g., Gus Hurwitz, Administrative Antitrust, 21 Geo. Mason L. Rev. 1191 (2014); Gus Hurwitz, Chevron and the Limits of Administrative Antitrust, 76 U. Pitt. L. Rev. 209 (2014). (The agency-win rate similarly fell for the ITC [72.7% to 70.0%] and Bureau of Prisons [73.7% to 61.5%] when Chevron was applied.)

It is likewise interesting to evaluate the stark disparities between agencies dealing with similar subject matters. For instance, the Energy Department (6.21) was the third worst among the twenty-eight agencies, whereas the EPA (7.49) ranked thirteenth overall. Although the frequency of Chevron being applied was somewhat similar (89.3% for the EPA to 90.9% for Energy), there was a difference of more than twenty percentage points in overall agency-win rates (67.9% to 45.5%) and in agency-win rates when Chevron applied (67.6% to 50.0%). Meanwhile a third energy-related agency, FERC, always received Chevron deference, but only prevailed 60.5% of the time—for a composite score (7.37) ranking of sixteen.

Likewise, the Treasury Department (8.37) ranked third overall on the composite scale, whereas the IRS (6.78), an agency within Treasury,244Note that we largely adopted the same coding methodology as Eskridge and Baer with respect to the agency advancing the statutory interpretation. We coded for the most specific agency on the Eskridge and Baer list. In other words, if the IRS was the agency that promulgated the interpretation, we would code the interpretation as an IRS interpretation, as opposed to a Treasury interpretation. The same is true, for instance, of the FDA versus HHS and the Bureau of Prisons versus Justice Department. Conversely, because the various immigration agencies were reorganized when the Department of Homeland Security was created, we decided to group all immigration decisions separately as “immigration agencies,” as opposed to DHS or Justice Department interpretations. ranked twenty-third out of the twenty-eight agencies. This divergence further illustrates the tax-exceptionalism phenomenon discussed in Section V.A. By contrast, the Department of Health and Human Services (HHS) (7.89) and the FDA (8.08), an agency within HHS, were basically ranked the same.

Consider, too, the stark differences in agencies dealing with labor and employment: the NLRB (8.26) surprisingly ranked fourth overall, whereas the EEOC (5.08) came in last place of twenty-eight agencies, and the Labor Department (7.14) was nineteenth. In terms of overall agency-win rates in the circuit courts, the NRLB prevailed 78.1% of the time, compared to 70.4% for the Labor Department and 42.9% for the EEOC. These circuit-court findings differ substantially from findings on the Supreme Court where, in one study, agency-win rates post-Chevron were virtually the same for the EEOC (51.9%) and NLRB (52.4%) and somewhat better for the Labor Department (66.7%).245James J. Brudney, Chevron and Skidmore in the Workplace: Unhappy Together, 83 Fordham L. Rev. 497, 509 tbl.2 (2014). And the findings for the NLRB differ significantly from Miles and Sunstein’s findings in the circuit courts concerning the NLRB’s win rate under Chevron. They found that the agency prevailed 70.1% of the time, see Miles & Sunstein, supra note 11, at 853, while we found that it prevailed significantly more—82.1% of the time.

Again, it is worth noting the wide range of circuit-court deference by agency, which is similar to the range for subject matter. For instance, the overall agency-win rate ranged from 100.0% for the ICC/STB to 45.5% for the Energy Department and 42.9% for the EEOC.246By comparison, the 55 Skidmore-only cases arose from twenty different agencies, with only three of those agencies having at least 5 cases. The agency-win rates for those agencies are: 57.1% for the Customs and Border Protection (7 cases); 55.6% for the Department of Labor (18 cases); and 40% for the collective immigration agencies (EOIR, BIA, and DHS) (5 cases). The rate of circuit courts applying Chevron ranged from 100.0% for FERC to 36.4% for the FTC and 41.7% for HUD. Similarly, when circuit courts applied Chevron, agency-win rates ranged from 100% for the ICC/STB and 93.3% for the Treasury Department to 50.0% for the Energy Department. Like the disparities based on subject matter, these agency-by-agency differences complicate the findings discussed in Part III regarding how Chevron matters in the circuit courts.

C.     Executive Versus Independent Agencies

One may also wonder how circuit courts treat executive and independent agencies differently. After all, the Chevron Court itself emphasized how political accountability may be a justification for deferring to agency statutory interpretations.247See Chevron U.S.A. Inc. v. Nat. Res. Def. Council, Inc., 467 U.S. 837, 865–66 (1984) (“While agencies are not directly accountable to the people, the Chief Executive is, and it is entirely appropriate for this political branch of the Government to make such policy choices . . . .”). Figure 10 separates out the key findings as to executive and independent agencies.

Of the 1,558 interpretations in our dataset, 1,284 interpretations (82.4%) were made by executive agencies, whereas the remaining 274 interpretations were made by independent agencies.248Although there are various ways to categorize agencies as either executive agencies or independent agencies, we relied on the Administrative Conference of the United States’ Sourcebook of the United States Executive Agencies and categorized agencies as independent if they either have heads who can be removed only for cause or the agencies are otherwise usually categorized as independent regulatory commissions. See David E. Lewis & Jennifer L. Selin, Admin. Conference of the U.S., Sourcebook of United States Executive Agencies 51 n.138, 52–53 tbl.4, 56 tbl.6 (2012). Accordingly, we categorized the following agencies as independent: CFPB, CFTC, FCC, FDIC, FEC, FERC, Federal Marine Commission, Federal Reserve, FLRA, FMSHRC, FTC, ICC/STB, ITC, MSPB, National Indian Gaming Commission, National Mediation Board, NLRB, NTSB, Nuclear Regulatory Commission, Post Office, Social Security Administration, and SEC. We also categorized the two joint rulemakings between the Federal Reserve and Treasury Department as independent. Perhaps surprisingly, the composite deference score for independent agencies (7.97) was higher than for executive agencies (7.34). Indeed, independent agencies were more successful, to varying degrees, as to all three indicators of deference: the overall agency-win rate (77.0% to 70.2%); the rate of circuit courts applying Chevron deference (82.5% to 73.2%); and the agency-win rate with Chevron deference (79.6% to 76.8%).


Again, one should be cautious inferring causation here. Especially in light of the nearly ten-percentage-point difference in Chevron deference being applied, one may be tempted to declare dead the political-accountability theory for Chevron deference. Indeed, higher agency-win rates and significantly more Chevron applications are seemingly contrary to one scholar’s view that independent agencies should receive less deference because they lack the same political accountability as executive agencies.249See Randolph J. May, Defining Deference Down: Independent Agencies and Chevron Deference, 58 Admin. L. Rev. 429 (2006). But there may well be other explanations. Independent agencies may be more cautious in seeking Chevron deference, and they may also be less aggressive in their interpretive efforts due to their independence from the President. The stark difference in agency-win rates (64.6% for independent agencies to 52.0% for executive agencies) when the circuit courts refused to apply the Chevron framework may support the theory that independent agencies are less aggressive.

VI.     Additional Findings: What Else Matters?

In this final Part, we consider some additional findings that are especially relevant to ascertain whether the circuit courts have internalized certain, often vague, nudges from the Supreme Court, especially when the Court’s practice is to the contrary. We begin by looking at how the circuit courts approach two sensitive subjects in Section IV.A, move in Section IV.B to whether stable interpretations fare better than inconsistent ones (despite similar doctrinal treatment), and conclude in Section IV.C by evaluating the salience of certain traditional deference factors in the courts of appeals.

A.     Sensitive Matters

As discussed in Section I.B.2, certain sensitive subjects—such as regulatory jurisdiction, state-law preemption, and significant political or economic questions—have created wrinkles, at one time or another, in the Supreme Court’s deference jurisprudence. Because the Court did not clearly identify significant questions as relevant to all Chevron step-zero inquiries until 2015,250King v. Burwell, 135 S. Ct. 2480, 2489 (2015). The Court had, however, strongly suggested some form of this exception in Gonzales v. Oregon, 546 U.S. 243, 258–68 (2006). very recently and well after our selected timeframe, we cannot say what impact that decision has in the circuit courts. But our data can provide insight as to regulatory jurisdiction and state-law preemption. Figure 11 compares the overall agency-win rate with the win rates for jurisdictional and preemption interpretations (with the frequency of Chevron’s application for each also depicted).


The Court clarified in May 2013 in City of Arlington v. FCC that Chevron applied to regulatory-jurisdiction questions largely because of the difficulty of distinguishing run-of-the-mill interpretation questions from so-called jurisdictional ones.251See supra note 84 and accompanying text. Before that ruling, however, Eskridge and Baer had found that the Court applied Chevron to regulatory-jurisdiction questions only 34.4% of the time.252Eskridge & Baer, supra note 10, at 1131. The circuit courts, however, appeared to do a better job of anticipating City of Arlington. Interpretations concerning regulatory jurisdiction made up 105 out of our 1,558 interpretations (6.7%).253Like the Supreme Court, our impression, too, was that this variable was difficult to code because of the difficulty in identifying which interpretations were “jurisdictional,” so we erred on the side of being underinclusive. Of those 105, the circuit courts applied Chevron deference to 78 of them (74.3%).254Our coded decisions included only four decided after City of Arlington. Riffin v. Surface Transp. Bd., 733 F.3d 340 (D.C. Cir. 2013); Dandino, Inc. v. U.S. Dep’t of Transp., 729 F.3d 917 (9th Cir. 2013); Oklahoma v. EPA, 723 F.3d 1201 (10th Cir. 2013); Helicopter Ass’n Int’l v. FAA, 722 F.3d 430 (D.C. Cir. 2013). Only Dandino, which had a mixed question of judicial and regulatory jurisdiction, did not apply Chevron. See Dandino, 729 F.3d at 920 n.1. Notably, this Chevron-application rate to regulatory-jurisdiction interpretations (74.3%) was basically the same for all interpretations (74.8%).

Although we did not directly code for major questions, we can get a sense of how the courts responded to Oregon v. Gonzales’s exception that declines to apply Chevron to changed agency positions as to major questions.255See supra note 99 and accompanying text. To do so, we can parse the regulatory-jurisdiction interpretations further by considering the frequency to which the circuit courts applied Chevron to agency interpretations concerning their jurisdictional or regulatory authority that replaced a prior, inconsistent interpretation (what we refer to as “evolving interpretations”). Of the 19 evolving interpretations that concerned regulatory jurisdiction, the circuit courts applied the Chevron framework 17 times (89.4%). This application rate was significantly higher than the average Chevron-application rates for all regulatory-jurisdiction interpretations (74.3%) and all interpretations, regardless of type, combined (74.8%). Despite these small numbers and the limited inferences that we can draw from them, this finding suggests courts have not internalized Gonzales’s step-zero exception.

Nevertheless, agency-win rates suggest that the circuit courts may be slightly uncomfortable deferring to agencies on these seemingly more significant matters. Under any deference regime, agencies prevailed on regulatory-jurisdiction matters only 63.8% of the time (67 of 105 interpretations). That win rate is somewhat lower than the overall agency-win rate of 71.4%. Similarly, despite receiving Chevron deference at basically the same rate as normal, agencies’ regulatory-jurisdiction interpretations prevailed 70.5% of the time, a lower win rate than that of 77.4% for all Chevron applications. (That said, Chevron still mattered, as agencies prevailed on regulatory-interpretations 70.5% of time with Chevron, and only 44.4% of time without it.) In the 17 of 19 instances when Chevron applied to an agency’s evolving regulatory-jurisdiction interpretation (similar to the Gonzales issue), the agency-win rate was 63.2% (12 wins). These slightly lower rates perhaps arise from the general significance of agency decisions or aggressive agency interpretations to expand their dominion.

As for state-law preemption, the doctrinal and scholarly dispute concerning the suitability of Chevron deference to state-law preemption may not be significantly meaningful to agencies. We uncovered only 25 interpretations concerning preemption in our dataset. Of those, the agency prevailed 80.0% of the time (20 of 25 cases). The agencies always prevailed when the court applied no deference or did not indicate whether deference applied, although there were only three of these decisions. The courts applied Chevron to 76.0% of the interpretations (19 of 25), and agencies prevailed 78.9% of the time under Chevron, meaning that the Chevron-application and agency-win rates were approximately the same as our database averages for both variables in all interpretations (74.8% application rate and 77.4% agency-win rate). This win rate under Chevron of 78.9% is near the agency-win rate for preemption questions when the circuit courts did not apply Chevron (83.3%).

Despite the scholarly call for Skidmore deference to apply to state-law preemption (from one of us and others)256See Barnett, supra note 76, at 22–51; Kent Barnett, Improving Agencies’ Preemption Expertise with Chevmore Codification, 83 Fordham L. Rev. 587 (2014) [hereinafter Barnett, Improving Agencies’ Preemption Expertise]; Mendelson, supra note 92, at 742; Nina A. Mendelson, A Presumption Against Agency Preemption, 102 Nw. U. L. Rev. 695, 698 (2008); Catherine M. Sharkey, Products Liability Preemption: An Institutional Approach, 76 Geo. Wash. L. Rev. 449, 491–98 (2008). and the finding (from a study by the other of us) that a majority of 128 agency rule drafters surveyed indicated that Congress does not delegate preemption matters to agencies,257See Walker, supra note 94. the Skidmore-application rate is 12.0% (3 of 25), roughly the same rate for our entire database (10.8%). Indeed, only one of those applications involved an agency rulemaking, where Chevron would be more likely to apply under Mead.258See Farina v. Nokia, Inc., 625 F.3d 97, 127 n.27 (3d Cir. 2010). The agency prevailed in this decision. Based on this small number of Skidmore decisions, the agency-win rate is more than ten percentage points greater than the database average for Skidmore (66.7% to 56.0%). The agency-win differential between Chevron and Skidmore deference, therefore, decreases from more than twenty percentage points for all relevant interpretations in our database to about twelve points for preemption-related interpretations. Again, however, we are dealing with small numbers.

Regulatory jurisdiction and state-law preemption together provide findings concerning two sensitive matters. These findings suggest that the circuit courts have not internalized the Supreme Court’s often vague and conflicting signals over limiting Chevron’s application to certain matters because they applied Chevron at higher rates to these matters than to all matters combined. But the lower agency-win rates under Chevron for regulatory jurisdiction suggest that agencies may account for judicial unease as part of their overall judicial review. All of this said, our findings do not allow us to make any definite conclusions based on the nature of the Court’s unclear directives, the relatively small number of decisions that arise in the circuit courts on these matters, the limited questions that we coded, and the inherent limitations in our coding methodology that cannot account for ad hoc concerns in the opinions or concerns that the courts did not express.

B.     Interpretive Continuity

Interpretive continuity has a complex role in deference doctrines and judicial interpretation generally.259See generally Anita S. Krishnakumar, Longstanding Agency Interpretations, 83 Fordham L. Rev. 1823 (2015), for a thoughtful treatment of interpretive continuity in judicial deference and interpretation. Interpretive continuity is relevant to whether agencies receive Skidmore deference,260See supra note 40 and accompanying text. but Chevron itself stated that such continuity is not germane to Chevron deference.261See supra note 47 and accompanying text; see also Merrill, supra note 10, at 977 (“[Chevron] appeared to downgrade the frequently cited factor stressing the importance of agency views that were long-standing and consistent.”); Scalia, supra note 3, at 517 (“[Under Chevron,] there is no longer any justification for giving ‘special’ deference to ‘longstanding and consistent’ agency interpretations of law.”). Nevertheless, both before and after Chevron, the Court identified its presence at times as a factor to consider when reviewing an agency’s interpretation.262See Barnhart v. Walton, 535 U.S. 212, 222 (2002) (noting that interpretive duration, among other things, influences deference); Merrill, supra note 10, at 972–73 (discussing deference before Chevron). Eskridge and Baer found that, despite the Court’s tendency not to apply Chevron where it would appear to apply,263See Eskridge & Baer, supra note 10, at 1134–35. “the overwhelming majority of the cases in which the Court invokes Chevron (70.6%) involve a long-standing or fairly stable interpretation. Indeed[,] this category dwarfs applications of Chevron where the agency interpretation is recent (27.1%) or evolving (2.4%),”264Id. at 1133–34. suggesting that the Court does not follow its own pronouncements as to Chevron’s applicability. Long-standing interpretations had an overall success rate under any deference regime of 73.2%, while recent and evolving interpretations had lower win rates of 66.9% and 60.5%, respectively, in the Supreme Court.265Based on these and other findings, Eskridge and Baer determined that the Court’s favorable treatment of long-standing interpretations “stands the Chevron Revolution on its head.” Id. at 1150. We sought to determine how long-standing and newer interpretations fared in the circuit courts.

Based on information that we could glean from the opinion itself, we coded the duration of interpretations as long-standing, evolving (meaning that one interpretation replaced a prior one), recent (meaning that a new interpretation did not replace a prior one), and unclear. Our coding was similar to Eskridge and Baer’s, except that we added an “unclear” category. We coded interpretations where the court made some reference to the stability or date of the agency interpretation, while we coded those for which we could not discern the longevity from the decision as “unclear.” We had a fairly even sample of interpretations of long-standing, recent, and unclear vintage. Approximately one-third of our interpretations were long-standing (34.5%), one-third were of unknown duration (35.0%), and one-third were either recent or evolving (30.5%).

Our data indicate that long-standing interpretations prevailed more frequently than other interpretations. Of all long-standing interpretations regardless of deference regime, agencies prevailed 82.3% of the time—far ahead of ones that were evolving (59.8%), recent (65.9%), or of unknown duration (67.8%). As compared to Eskridge and Baer’s findings, the long-standing interpretations fared even better in the circuit courts (about nine percentage points better), while the recent and evolving interpretations fared about the same (both only one percentage point worse).266See supra notes 263–265 and accompanying text. That said, if we combine all interpretations under any deference regime for long-standing interpretations and those whose duration is unclear (1,086 interpretations), as it appears that Eskridge and Baer did, the agency-win rate (813 wins out of 1,084 interpretations) falls to 75.0%, almost the same as theirs for long-standing interpretations. Figure 12 summarizes these comparisons, with the unknown category in our study broken out separately.


When it came to applying the Chevron framework, however, circuit courts were not more likely to apply Chevron to long-standing interpretations than other interpretations. Courts applied Chevron to 76.2% of long-standing interpretations (410 out of 538) and roughly the same frequency to recent interpretations (76.1%, or 194 of 255 interpretations). The surprise came with evolving interpretations. Circuit courts applied Chevron even more frequently to them (86.3%, or 189 of 219 interpretations). When the interpretation was unclear, courts applied Chevron 68.3% of the time (373 of 546 interpretations). When we parsed the data further to see whether courts applied Chevron at different rates for long-standing versus new or evolving interpretations that were presumptively Chevron-eligible (meaning those from formal rulemaking or adjudication or notice-and-comment rulemaking), the disparity disappeared. Circuit courts applied the Chevron framework to 88.1% of long-standing formal interpretations and 87.7% of evolving or recent formal interpretations (92.0% and 83.3%, respectively).267Long-standing informal interpretations received the Chevron framework only 32.5% of the time, while evolving or recent informal interpretations received the Chevron framework more often—both at 59.0%.

Once Chevron applied, long-standing agency interpretations triumphed again, especially over evolving ones. Long-standing interpretations prevailed 87.6% of the time. Interpretations of recent or unclear vintage were affirmed at lower rates of 74.7% and 73.5%, respectively. Evolving interpretations, the interpretations most likely to have Chevron apply, had the lowest agency-win rate of 65.6%.268Because of differences in coding, it is difficult to compare our data with Kerr’s. He compared consistent and inconsistent interpretations by scanning opinions to see if the courts mentioned “changes in the agency’s interpretation over time” or not. If they didn’t, he assumed consistency. See Kerr, supra note 17, at 24. We distinguished evolving interpretations from all others based on a similar methodology, except that we marked those with no indication of an interpretation’s duration as unclear. If we combine our unclear, recent, and long-standing interpretations that were subject to the Chevron framework (all of which would appear “consistent” under Kerr’s methodology), agencies prevailed 79.6% of the time (in 778 of 977 interpretations), a slightly higher rate than the 74% win rate that Kerr found. See id. at 32 chart 1. For inconsistent (or evolving) interpretations, his agency-win rate (68%), id., is slightly higher than ours (65.6%). Ultimately, our differential between the win rates of consistent and inconsistent interpretations (fourteen percentage points) is quite larger than his (six percentage points). Despite having the lowest agency-win rate under Chevron, this win rate for evolving interpretations was actually its highest by a significant margin under any of the deference regimes. Evolving interpretations have the lowest win rate under every deference regime except de novo review, often by wide margins. For instance, they had a 0.0% win rate in the three instances when the courts identified no deference regime, in comparison to a win rate of 72.7% for long-standing interpretations. Likewise, evolving agency interpretations prevailed only 30.8% of the time under de novo review (13 instances), with only recent interpretations doing more poorly with a win rate of 16.7% (24 instances). Under Skidmore, evolving interpretations prevailed only 21.4% of the time (14 instances), while recent ones prevailed 46.2% of the time (26 instances) and long-standing ones 67.6% of the time (71 instances). The findings are fully presented in Figure 13.

Based on our coding, we can further mine the data on recent and evolving interpretations. When courts reviewed evolving or recent interpretations under Chevron, certain of those interpretations did significantly better than others. Of the 383 recent or evolving interpretations to which courts applied Chevron, they arose in response to new or amended statutes (98), agencies facing new issues (95 interpretations), changed facts or judicial decisions (91), the agency’s practical experience (67), new presidential administrations (8), reevaluated litigating positions (3), or in response to a judicial decision (1)—with the remainder for unclear reasons (20). Agency interpretations in the four largest categories all prevailed under Chevron at relatively consistent and high rates: from a high of 73.1% and 72.5% for practical experience and changed circumstances, respectively, to 70.4% and 69.5% for new statutory provisions and new issues, respectively. A sharp drop occurred when the reasons weren’t clear (60.0%, based on 20 interpretations) or the changed interpretation came from a new administration (50.0%, based on 8 interpretations).


What to make of this continuity data?

First, the findings suggest that the circuit courts have followed Chevron’s command that Chevron applies with equal force to all agency positions, whether they are changed, new, or long-standing. The circuit courts’ Chevron-application rate was similar for recent and long-standing interpretations, and the rate even increased for evolving interpretations, perhaps because the government went out of its way to point out Chevron’s command on the duration issue.269This strategy is consistent with the “playbook” that federal government litigators use to defend agency statutory interpretations, which “tends to be based on general principles of administrative law.” Walker, supra note 107, at 154; see also Walker, supra note 156, at 77–87 (exploring in greater detail this playbook, which is based in part on experiences working on the Justice Department’s Civil Appellate Staff). Moreover, when we filtered the data further to compare long-standing with new or evolving interpretations that were presumptively Chevron-eligible, the courts applied Chevron at almost the same rate (88.1% and 87.7%, respectively).

Second, once Chevron applied, interpretive duration seems to matter, although the nature of that relationship is unclear. Long-standing interpretations prevailed 87.6% of the time, approximately thirteen and fourteen percentage points more often than new interpretations and those of unclear duration, respectively, and twenty-two percentage points more often than evolving interpretations. Accounting for an interpretation’s longevity in the deference process, despite seeming contrary to Chevron itself, would be consistent with courts thinking of deference on a sliding scale, as Justice Breyer has long advocated, perhaps most successfully in Barnhart. And it would be consistent with the Court’s recent invocation of interpretive duration when it blessed a Patent and Trademark Office rule under Chevron step two.270See Cuozzo Speed Techs., LLC v. Lee, 136 S. Ct. 2131, 2145 (2016). But it may also be that long-standing interpretations are more likely to be better thought-out and less aggressive than more recent, especially changed, ones.

The noticeable lack of agency success when a new administration simply changes the interpretation might suggest that circuit courts have not fully embraced the political-accountability theory that undergirds Chevron. Chevron recognized that the political branches had more accountability than unelected judges and were in a better position to make policy choices inherent in interpretive issues.271Chevron U.S.A. Inc. v. Nat. Res. Def. Council, Inc., 467 U.S. 837, 865–66 (1984). Indeed, the Chevron Court deferred to the Reagan Administration’s interpretation, despite the fact that the Carter Administration had interpreted the term at issue differently.272See id. at 857–58, 866. Or it could show judicial discomfort with APA arbitrary-and-capricious review, which some decisions have folded into Chevron step two (as opposed to treating it as a distinct step).273See Kathryn A. Watts, Proposing a Place for Politics in Arbitrary and Capricious Review, 119 Yale L.J. 2, 8 n.15 (2009) (discussing courts that combine step two and arbitrary-and-capricious review and scholars who support them). In Motor Vehicles Manufacturers Ass’n v. State Farm Mutual Automobile Insurance Co., Justice Rehnquist’s partial dissent, joined by three other justices, blessed an agency’s reasonable reappraisal of costs and benefits in light of a new administration,274463 U.S. 29, 59 (1983) (Rehnquist, J., concurring in part and dissenting in part). but the majority’s silence on this point and preference for technocratic analysis has been understood to mean that changes based on political forces are improper.275Watts, supra note 273, at 5. Ultimately, however, the small number of interpretations limits the inferences that one can draw from them. Indeed, that only 9 of the 474 total recent or evolving interpretations expressly implicated change in administration may reflect agencies’ strategic decisionmaking to avoid justifying a new or different interpretation on political grounds. And, notably, these data do not tell us when courts are expressly referring to an interpretation’s duration as part of their analysis. We discuss the invocation of factors, including duration, in Section VI.C.

Third, Skidmore seems to be working much as expected. Interpretive consistency is a germane factor under the doctrine that favors an agency’s position. Long-standing interpretations prevailed more frequently (67.6%) than others, indeed at a rate above the average rate for all Skidmore decisions (56.0%). The other interpretations’ agency-win rates were below the average, as one would expect: 21.4% for evolving ones, 46.2% for new ones, and 54.4% for ones of unclear duration. It makes sense that if consistency were the concern, new decisions would not evidence inconsistency (because there is no prior interpretation with which to be inconsistent) and thus should prevail more frequently than evolving ones that do, even if at a lesser rate than long-standing, consistent ones.

Finally, agencies seeking to issue evolving interpretations should be mindful of how they do so. Although agency-win rates were at their nadir for those interpretations under nearly every deference regime, agencies seem to be able to significantly improve their win rates by providing the interpretations with the force of law to render it more likely that they obtain Chevron deference, under which evolving interpretations prevailed 65.6% of the time. When agencies use less-formal means, courts are much less likely to apply the Chevron framework—only 59.0% for informal evolving interpretations but 92.0% for formal ones. With Chevron, the agency-win rate was 65.6%, but it plummeted to 30.8% with de novo review. And they plummeted forty-four percentage points from the 65.6% win rate under Chevron to the 21.4% win rate under Skidmore. Moreover, even with Chevron deference, agencies should carefully consider the reasons for the change. Changes based on differing political administrations or unclear changes suffered significantly lower win rates. When changing interpretations, agencies will likely place themselves on better footing by clearly pointing to changed facts and their experience to support the change.

C.     Traditional Deference Factors or Theoretical Grounds

Before Chevron, the courts evaluated various factors in an ad hoc manner to determine whether to defer to agency interpretations. These factors included whether the matter fell within the agency’s expertise, its careful consideration over a long period of time, congressional delegation, its contemporaneity with the statute’s enactment, or vague notions of congressional ratification.276See Merrill, supra note 10, at 973–74. Although Chevron and Mead had suggested that some of these factors were more important than others (delegation) or no longer relevant (consistency) when deciding whether Chevron applied, the Court’s dicta in Barnhart referred to more than delegation and force-of-law authority. It invoked some of these traditional factors—agency expertise, congressional acquiescence, and the agency’s careful consideration over a long period of time—and some additional ones concerning the nature of the legal question and the complexity of the statute.277Barnhart v. Walton, 535 U.S. 212, 222 (2002).

To get a sense of these factors’ relevance in the circuit courts, we followed Eskridge and Baer’s coding, where they added three theoretical factors and combined some of the contextual factors: agency expertise, accountability, national standard, long-standing interpretation, contemporaneity, public reliance, rulemaking authority, agency procedures, and congressional acquiescence.278See Eskridge & Baer, supra note 10, at 1216. Eskridge and Baer did not discuss their findings on these factors. Although it appears that Merrill’s identified factors largely overlap with Eskridge and Baer’s, we were less certain exactly which factors he considered in his coding. See Merrill, supra note 10, at 981. Similarly, Kerr coded two of these variables and an additional one: contemporaneity, longstandingness, and consistency. See Kerr, supra note 17, at 22–24. But he did not code for judicial invocation of them. See id. Like Eskridge and Baer, we coded each variable if the circuit court expressly referred to one of them in its opinion, whether specifically in the step-zero context or as part of its analysis of the interpretation itself. Similarly, we coded these factors whether courts noted their presence or absence; the findings reported in this Section do not disaggregate them. We found that only four of these factors had even an arguably regular place in circuit courts’ deference discourse under any regime. Figure 14 depicts these findings.


The most-invoked factors were not surprising: agency procedures utilized (25.7% of the time), rulemaking authority (18.3%), agency expertise (18.4%), and interpretive stability (10.7%). One would have expected, if anything, the first two to figure more prominently because they are the two factors that relate most closely to Mead’s delegation inquiry and concern for formality.

Expertise’s limited prominence in the dataset was also contrary to expectations. It is one of the relevant factors for Skidmore deference, and it is likely to come up as part of an inquiry into whether Congress intended to delegate certain issues. But once again, if there is any surprise here, it is that expertise played such a small role in our Skidmore interpretations. Courts referred to expertise in 42.9% of the 168 Skidmore decisions, 15.3% of the 1,166 Chevron decisions, and 24.8% of the 117 de novo decisions. Despite its serving as the theoretical basis for the Skidmore doctrine and its relevance to the agency’s reasoning and consideration,279Barnett, Improving Agencies’ Preemption Expertise, supra note 256, at 589. it was invoked less than half the time for interpretations to which the Skidmore framework applied.

And so the story goes for interpretive stability. The courts referred to the duration of an interpretation in only 10.7% of all their discussions and only 8.9% of interpretations where Chevron applied. These numbers are smaller than expected, considering that courts agreed with long-standing agency interpretations at higher rates regardless of deference regime as well as under Chevron. Courts were, as with agency expertise, more likely to refer to this factor when they applied Skidmore, the regime under which consistency is a factor. They referred to it in 23.8% of all 168 Skidmore decisions. But because it is a Skidmore factor, one would have expected it, as well, to be referred to more frequently than only about a quarter of the time. The circuit courts’ ambivalence in expressing its thoughts on the long-standing nature of the agency statutory interpretation—no matter its actual impact on decisionmaking—ultimately confirms one leading scholar’s view that the federal courts have not thought out interpretive durability’s place in judicial review.280See Krishnakumar, supra note 259, at 1830–43.

The five remaining factors were obscure in circuit-court decisions. Courts invoked political accountability in 0.5% of all interpretations, public reliance in 0.7%, contemporaneity in 1.9%, national standards in 2.2%, and congressional acquiescence in 3.1%.

These results provide some (albeit limited) insights on the place of Mead, Barnhart, and the remaining contextual factors. Mead’s focus on delegation and formality, unsurprisingly, has a firm grasp on the circuit courts. Two of the most significant factors that courts invoked were agency procedures and rulemaking authority, both of which focus on the ability of the agencies to speak with the force of law and use of that authority. Relatedly, given the high rates at which formalized agency interpretations received Chevron deference,281See supra Figure 5. it appears that courts and parties considered formality even if they did not usually mention it. Given Mead’s relatively straightforward view as to formal interpretations, these factors’ prominence is not surprising.

The salience of Barnhart’s dicta, in contrast, is less certain. Of the three Barnhart factors that we coded (expertise, longevity, and congressional acquiescence), courts invoked the first two more frequently than other contextual factors, but still at relatively low rates of 18.4% and 10.7% and more frequently in the context of Skidmore review in which they are doctrinal factors. But similar to our inferences from our data on Chevron applications to formal interpretations above, our data on interpretations’ duration—where consistent agency interpretations prevailed at higher rates under all deference regimes combined, despite not receiving Chevron deference at increased levels282See supra Section VI.B.—suggest that the factor may be doing silent work in the circuit courts’ decisionmaking after a step-zero inquiry. Unlike with long-standing interpretations, we do not have another variable that might illuminate whether agency expertise informs circuit-court decisionmaking even when the courts do not refer to it. As to the third factor, circuit courts referred to congressional acquiescence only 3.1% of the time, significantly less than the other two factors, suggesting perhaps that it has little salience in judicial decisionmaking. Yet, as with the other variables, we cannot rule out the chance that courts consider congressional acquiescence without mentioning it.

The remaining ad hoc contextual factors or theoretical concerns appear to have little purchase on the circuit courts, which referred to any one of them only, at most, approximately 2% of the time. Their low salience suggests that certain traditional factors have faded from judicial memory. Most prominently, contemporaneity (along with long-standing consistency), a traditional factor of long provenance, has essentially lost its hold on circuit courts.283See supra note 32 and accompanying text. This finding was not surprising given the Supreme Court’s consistent view that “neither antiquity nor contemporaneity with [a] statute is a condition of [a regulation’s] validity.”284Mayo Found. for Med. Educ. & Research v. United States, 562 U.S. 44, 55 (2011) (alteration in original) (quoting Smiley v. Citibank (S.D.), N.A., 517 U.S. 735, 740 (1996)). Perhaps, though, like other factors, courts accept contemporaneous interpretations more frequently and thus the factor is doing more work behind the scenes than expressed invocations suggest.285Indeed, Kerr found that agency interpretations promulgated within four years of a statute’s enactment were 12% more likely to prevail. See Kerr, supra note 17, at 33–34, 33 chart 3.

Although we coded for courts’ express references to contemporaneity, we did not code specifically for contemporaneous interpretations by themselves. But if we use our variable of recent interpretations arising from a new or amended statute as contemporaneous (117 interpretations), contemporaneous interpretations prevailed under any deference regime 63.2% of the time and under Chevron 69.5% of the time. Notably, both of these numbers were lower than the overall agency-win rate under any regime (71.4%) and the average Chevron agency-win rate (77.4%), suggesting that contemporaneity does not have the same pull on courts as the win rates suggest that stability and formality do. That said, our variable for recent interpretations arising from a new or amended statute wouldn’t include all contemporaneous interpretations, such as those that are long-standing (and thus not new) but issued contemporaneously with a statute’s enactment or amendment. And it may include interpretations that, while new, did not occur until many years after a statute’s enactment or amendment because, after all, rulemaking or adjudication takes time. Because our variable doesn’t track contemporaneity perfectly, our conclusions are necessarily limited.

Whatever the normative value of the Barnhart and other contextual factors in judicial deference, their largescale absence from deference discussions in the circuit courts suggests that courts prefer the relatively more rule-like certainty of Mead than the ad hoc approaches before Chevron or offered by Barnhart. This is so despite the fact that the ad hoc approaches would provide circuit courts more discretion and allow them to better hide strategic decisionmaking to allow courts to align policy preferences with their interpretations. Like Odysseus tied to the mast, circuit courts seem to have found some benefits in having others limit their agency.


Let us briefly return to where we began with our findings in Part III—the big picture. We have discussed particular findings and their implications in each Part. But what broader insights about Chevron Regular and Chevron Supreme can we glean from stepping back and considering our findings as a whole?

We have demonstrated empirically that, contrary to how they fare in the Supreme Court,286See supra note 22 and accompanying text. agencies usually prevail more under Chevron than other standards of review in the circuit courts (at least when those courts refer to Chevron).287See supra Section III.A. This finding is meaningful for agencies and litigating parties because circuit courts review far more agency statutory interpretations than the Supreme Court. Although we cannot say in our discussion here how the deference standards affect judicial decisionmaking, we can say outcomes do vary. Because they do, one leading scholar’s call, based on findings from past empirical studies, for practitioners, teachers, courts, and scholars to deemphasize review standards appears premature.288See Pierce, supra note 5, at 93–98; cf. Richard Pierce, Circuit Courts Do Strange Things with Chevron, Jotwell (Sept. 6, 2016), [] (reviewing an earlier draft of this Article and noting, inter alia, that “whatever Chevron means in circuit courts, the circuit court version differs from the Supreme Court version in many ways”). They seem to matter, even if no one, including us (based on methodological limitations), can yet say exactly how.

If Chevron matters, we should consider whether it is functioning properly. The Supreme Court indicated that Chevron exists to provide agencies a congressionally delegated space to regulate, where courts keep agencies in their space without imposing their own policy judgments.289See supra notes 47–49 and accompanying text. The doctrine largely appears to fail at achieving these aims in the Supreme Court based on its rare invocation290See supra note 10 and accompanying text. and failure to constrict the justices’ perceived preferences.291See Miles & Sunstein, supra note 11, at 825–26. Prior studies of the circuit courts have also found that Chevron does not appear to meaningfully constrict judges from deciding in accord with their perceived political preferences292See id. at 826–27 (considering NLRB and EPA).—at least when a judge on a panel with different political preferences isn’t on the panel.293See Cross & Tiller, supra note 14, at 2175–76. Although we leave our ideology data and more sophisticated statistical modeling for future work, our initial, descriptive findings suggest, based on a larger dataset than in prior studies, that Chevron has some kind of disciplining effect in the aggregate on circuit courts because agency-win rates are so disparate between when Chevron applies and when it does not, even when the agency statutory interpretations use the same formal interpretive methods.294See supra Figure 6.

More specifically, our thirty-nine-percentage-point difference between agency-win rates under Chevron and de novo review suggests that courts distinguish looking for the best answer from permitting a reasonable one.295See supra Figure 3. If they are able and willing to do so, then the Supreme Court’s recently invoked “stabilizing purpose”—to render outcomes from thirteen circuit courts more predictable296City of Arlington v. FCC, 133 S. Ct. 1863, 1874 (2013). and thereby further the uniformity goals that Peter Strauss highlighted decades ago297See Strauss, supra note 102, at 1117.—becomes more compelling, regardless of the delegation theory’s normative force.298See Barnett, supra note 76, at 14–22 (discussing views on Chevron’s delegation theory). Indeed, as federal dockets have swelled, Chevron may be one more device that federal courts have used to avoid what they perceive as low-value or low-interest cases.299See Aziz Z. Huq, Judicial Independence and the Rationing of Constitutional Remedies, 65 Duke L.J. 1, 68 (2015) (noting that an appellate review model over agency action is an example of federal courts seeking to “mitigate[] [the] caseload demands created by the new federal regulatory state” and to remove “ ‘[p]etty’ cases” from their docket).

But, at the same time, our data indicate that the Supreme Court needs to provide better guidance to lower courts if it seeks to create a stabilizing doctrine. The circuit-by-circuit disparity in the circuit courts’ invocation of Chevron and agency-win rates reveals that Chevron may not be operating uniformly among the circuits.300See supra Figures 9 & 10, Table 1. To ameliorate uniformity, the Court should provide clearer guidance to numerous issues, which other scholars have noted: What are the “traditional tools of statutory construction”301Chevron U.S.A. Inc. v. Nat. Res. Def. Council, Inc., 467 U.S. 837, 843 n.9 (1984). to which Chevron referred for step one that courts should use?302See Caleb Nelson, Statutory Interpretation and Decision Theory, 74 U. Chi. L. Rev. 329, 347 (2007) (discussing the Court’s failure to define the “traditional tools”). Should the long-standing nature of agency interpretations matter?303See generally Krishnakumar, supra note 259. What role exactly should legislative history or a purposivist inquiry have?304See generally Linda Jellum, Chevron’s Demise: a Survey of Chevron from Infancy to Senescence, 59 Admin. L. Rev. 725 (2007), for a thorough discussion of the Supreme Court’s unclear and conflicting treatment of how to discern congressional intent under Chevron step one, whether with all or only some of the following: text, legislative history, or purpose. See generally Michael Herz, Purposivism and Institutional Competence in Statutory Interpretation, 2009 Mich. St. L. Rev. 89, 119, for an argument that only agency interpretations that further a statutory scheme’s purpose can be reasonable. Very recently, the Court suggested that text, purpose, and history all inform Chevron’s step-one inquiry. Cuozzo Speed Techs., LLC v. Lee, 136 S. Ct. 2131, 2144 (2016) (“Finally, neither the statutory language, its purpose, or its history suggest that Congress considered what standard the agency should apply . . . .”). Is there an “order of battle” in which the circuit courts proceed through certain steps or interpretive canons to interpret statutes?305See Stephenson & Vermeule, supra note 184, at 608. Is step two different from arbitrary-and-capricious review and, if so, how?306See supra note 273 and accompanying text. The Court may have provided some guidance on this point when it refused to defer to an agency rule under Chevron when the agency’s failure to sufficiently explain why it replaced a prior statutory interpretation with a new one was arbitrary and capricious. See Encino Motorcars, LLC v. Navarro, 136 S. Ct. 2117, 2126 (2016). Because the Court did not engage in a two-step inquiry, it appears that arbitrary-and-capricious review is its own inquiry, unrelated to Chevron’s step-two reasonableness inquiry. And perhaps more prominently, what role do agency expertise, formality, and the significance of the question have when determining when Congress has delegated authority to agencies?307See supra Section I.B.1. If Chevron is a means of controlling the lower courts, the case for providing more guidance becomes urgent.

And our findings, albeit to a limited degree, suggest that lower courts will view more rule-based guidance as a comforting swaddling blanket rather than handcuffs. Circuit courts rarely invoked various values—including those mentioned in Barnhart—that they could have used to gain additional discretion in deciding whether to invoke Chevron or ultimately side with the agency.308See supra Section VI.C. And they appeared to largely ignore troubling step-zero questions concerning sensitive matters, perhaps having difficulty discerning the Supreme Court’s vague or inconsistent signals as to these matters.309See supra Section VI.A. If Chevron can function as a welcomed supervisory doctrine, the differences between Chevron Supreme—functioning as a malleable, discretionary canon of construction310See Connor N. Raso & William N. Eskridge, Jr., Chevron as a Canon, Not a Precedent: An Empirical Study of What Motivates Justices in Agency Deference Cases, 110 Colum. L. Rev. 1727, 1734–35 (2010) (arguing that based on Supreme Court practice, Chevron is a canon of construction, not a precedent).—and Chevron Regular—functioning as precedent—become less troubling.

Exceptional questions, rare theoretical grounds, and Chevron’s inconsistent use can permit the Supreme Court to keep the delegation theory in check at the margins without, as our data suggest, creating confusion and, as we plan to consider in future work, promoting ideological decisionmaking in the circuit courts. Indeed, two scholars have recently argued that distinctions between Chevron Supreme and Chevron Regular, at least as to major questions, are normatively justified.311See Michael Coenen & Seth Davis, Minor Courts, Major Questions, 70 Vand. L. Rev. 777, 781–82 (2017). But see Kent Barnett & Christopher J. Walker, Response, Short-Circuiting the New Major Questions Doctrine, 70 Vand. L. Rev. En Banc147, 155 (2017), [] (arguing that lower courts should apply the new major questions doctrine because the doctrine directly implicates congressional delegation and because they can provide useful “percolation” benefits for the Supreme Court). Their argument follows another scholar’s call for the degree of deference to agency interpretations to vary based on the deciding court’s place in the federal judicial hierarchy, with more deference in lower courts and less deference in superior courts.312See generally Aaron-Andrew P. Bruhl, Hierarchically Variable Deference to Agency Interpretations, 89 Notre Dame L. Rev. 727 (2013). But even if differences in deference among courts defy normative justification as to all interpretive matters or exceptional questions, our data suggest that any problematic characteristics of Chevron Supreme do not necessarily trickle down to the lower courts. Ultimately, Chevron Supreme, with its comparatively broader discretion, will shift power from the circuit courts to the Supreme Court and agencies but leave Chevron Regular in place to create more certainty in the lower courts and, thus, greater national uniformity in federal administrative law.313See, e.g., Walker, supra note 107, at 156–58 (arguing that Supreme Court’s application of Chevron to substantive patent law could be “a means of weakening the Federal Circuit”).

This is not our last word on what our data say about Chevron, and we hope that it furthers numerous other conversations concerning deference to agency statutory interpretations—whether about its normative place, its operation, or its meaningfulness.

*      Associate Professor, University of Georgia School of Law.

**      Associate Professor, The Ohio State University Moritz College of Law. For helpful comments on prior drafts, many thanks to Michael Asimow, Nick Bagley, Emily Bremer, Aaron-Andrew Bruhl, Bill Eskridge, David Hausman, Kristin Hickman, Brian Kalt, Ron Levin, Gillian Metzger, Aaron Nielson, Jennifer Nou, Jim Oleske, Dick Pierce, Daphna Renan, Usha Rodrigues, Guy Rub, Michael Sant’Ambrogio, Jed Stiglitz, Peter Strauss, and Adrian Vermeule and to the participants at the American Association of Law Schools Annual Meeting, the American Bar Association’s Annual Administrative Law Conference, and the First Annual Administrative Law New Scholarship Roundtable. We are extremely grateful to our research assistants: Morgan Allyn, Megan Bracher, Mathew Doney, Sidney Eberhart, Lauren Farrar, JD Howard, Gregg Jacobson, Mariam Keramati, Patrick Leed, David McGee, James Mee, Andrew Mikac, Justin Nelson, Meghna Rao, Serge Rumyantsev, Jonathan Stuart, and Molly Werhan. Finally, we appreciate the Michigan Law Review editors’ careful attention to our Article.