Regulating Black-Box Medicine
W. Nicholson Price II*
Data drive modern medicine. And our tools to analyze those data are growing ever more powerful. As health data are collected in greater and greater amounts, sophisticated algorithms based on those data can drive medical innovation, improve the process of care, and increase efficiency. Those algorithms, however, vary widely in quality. Some are accurate and powerful, while others may be riddled with errors or based on faulty science. When an opaque algorithm recommends an insulin dose to a diabetic patient, how do we know that dose is correct? Patients, providers, and insurers face substantial difficulties in identifying high-quality algorithms; they lack both expertise and proprietary information. How should we ensure that medical algorithms are safe and effective?
Medical algorithms need regulatory oversight, but that oversight must be appropriately tailored. Unfortunately, the Food and Drug Administration (FDA) has suggested that it will regulate algorithms under its traditional framework, a relatively rigid system that is likely to stifle innovation and to block the development of more flexible, current algorithms.
This Article draws upon ideas from the new governance movement to suggest a different path. FDA should pursue a more adaptive regulatory approach with requirements that developers disclose information underlying their algorithms. Disclosure would allow FDA oversight to be supplemented with evaluation by providers, hospitals, and insurers. This collaborative approach would supplement the agency’s review with ongoing real-world feedback from sophisticated market actors. Medical algorithms have tremendous potential, but ensuring that such potential is developed in high-quality ways demands a careful balancing between public and private oversight, and a role for FDA that mediates—but does not dominate—the rapidly developing industry.
*Assistant Professor of Law, University of Michigan Law School. For helpful comments and conversations, I wish to thank Kate Andrias, Sam Bagenstos, Nicholas Bagley, Valarie Blake, Ana Bracic, Tom Burroughs, Nathan Cortez, Mary Crossley, Kelly Dineen, Rebecca Eisenberg, Barbara Evans, Roger Ford, Rob Gatter, Jesse Goldner, Tim Greaney, Sharon Jacobs, Sandra Johnson, Kyle Logue, Nina Mendelson, Bill Novak, Efthimios Parasidis, Elizabeth Pendo, Adam Pritchard, Gabriel Rauterberg, Chris Robertson, Rachel Sachs, Margo Schlanger, Carl Schneider, Bradley Thompson, Sidney Watson, Lindsay Wiley, Patricia Zettler, and others I may have neglected to name. Sudhana Bajracharya, Rebecca Kaplan, Cassandra Simmons, and Jonathan Tietz provided excellent research assistance. This work benefited from feedback at the Health Law Professors’ Conference at Saint Louis University; the Health Law Scholars Workshop at Saint Louis University; ASU’s Governance of Emerging Technologies Conference; the Stanford Law and the Biosciences Workshop; the Petrie-Flom Center’s Conference on Big Data in Health at Harvard Law School; the Governance Workshop at Michigan Law School; and the Harvard Health Law Policy, Biotechnology, and Bioethics Workshop. All errors are my own.
