Nov 11, 2024 Rebecca Tushnet
Sometimes, it’s the small details that hobble even the most easily explained policies. When California decided to expunge felony records for marijuana offenses, relief for former felons was hampered by a lack of comprehensive recordkeeping and reliance on proactive individual action (the expungement wasn’t automatic; you had to ask for it). These and similar stumbling blocks can be weaponized by opponents, as occurred with the restoration of voting rights to felons in Florida. It’s a technological spin on the well-known legislator’s warning, “If I let you write the substance and you let me write the procedure, I’ll screw you every time.”
In Recoding America, Jennifer Pahlka makes the argument that there doesn’t even have to be a bad guy on the procedure side for this to happen. This is a book by a technocrat with a persuasive argument for a measure of technocracy: America’s ways of lawmaking could be greatly improved by borrowing from the project management concept of agile development, which allows people lower in the hierarchy to make consequential decisions rather than being burdened by having all the rules have to be specified in advance. The latter, “waterfall” development, can lead to deadly (sometimes literally) complexity and policy failure. When policy is too rococo and reticulated, such as having nine different definitions of a “group” of doctors for Medicare purposes, throwing money at the problem rarely helps. Neither does outsourcing and oversight, both of which Pahlka believes can help when properly deployed but often end up generating more layers of bureaucracy.
Pahlka argues that teams building tech to implement a government policy should have the authority to alter it as they go. They should build something that works at least a bit as soon as possible, shift edge cases to human review, and automate the easy stuff rather than building software that’s supposed to accommodate all possible situations. The worst part of directives from above, from her view, is that “nowhere in government documents will you find a requirement that the service actually works for the people who are supposed to use it. The systems are designed instead to meet the needs of the bureaucracies that create them—they are risk-mitigation strategies for dozens of internal stakeholders”—even though they also fail at that regularly.
The failure of service and benefit systems is a special problem for government because people interpret their experiences with bureaucracies as evidence of how government works more generally. Involvement with the criminal system, getting a construction permit, or filing taxes, can be unpleasant enough that it erodes faith in government and deters political participation. (This observation suggests that making voting complex, as when absentee voters are required to use two envelopes and sign and date one of the envelopes, doesn’t just disenfranchise individual voters—the ultimate effect is to deter citizens from even trying.) Unfortunately, the problem is also worse in government because obsolete tech is paired with obsolete policies—not just obsolete, but accreted over rounds and layers of attempted reform, which is how you get those nine definitions of a “group” of doctors in Medicare.
Pahlka compares current policymaking frameworks to waterfall development in software, where directives come from above. Waterfall development uses new data to grade only after the fact. “For people stuck in waterfall frameworks, data is not a tool in their hands. It’s something other people use as a stick to beat them with.” Naturally, they aren’t that interested in collecting ammunition against themselves. In addition, “[e]ven when legislators and policymakers try to give implementers the flexibility to exercise judgment, the words they write take on an entirely different meaning, and have an entirely different effect, as they descend through the hierarchy, becoming more rigid with every step.” She gives numerous examples.
One thing that Pahlka suggests might be fixable is policymakers’ cultural contempt for implementation. They think/hope/expect/imagine that if they write the right rules, everything will be fine. But it isn’t and won’t be. Pahlka criticizes the Administrative Procedure Act rulemaking process that most of government uses, because it essentially invites and requires interest group lobbying for every rule. The required process is more like a jury trial than an expert evaluation. Leftists, she argues, got really good at suing the government to stop bad stuff, but that contributed to an environment of risk aversion at agencies (and didn’t stop the Supreme Court from harming agency power anyway).
At points, Pahlka is pretty clear that there are some no-win scenarios here: Equity usually requires data, which requires paperwork, which favors the powerful. So, what is to be done? One very concrete recommendation from Pahlka is to focus on making things simpler for most people and devote human resources to the tougher situations. She argues that new programs should be launched when they’re ready to handle 85% of the cases, though the edge cases should be addressed technologically eventually. In reality, she points out, policies are launched incrementally anyway, because the systems built under current processes don’t work for a lot of people. Waterfall policymaking merely ensures that rollouts are incremental in the worst possible way.
As one person quoted in the book says of welfare applications, “Every time you add a question to a form, I want you to imagine the user filling it out with one hand while using the other to break up a brawl between toddlers.” Documentation should be required only when needed and responsive to actual circumstances. Sadly, Pahlka doesn’t give much shrift to the idea of just removing eligibility constraints for services and benefits, whether that’s sending money to people with kids or implementing universal health care. More universal safety nets could lead to lots less waste and failure in the administration of exceptions.
For an example of successful agile development in government, Pahlka points to free COVID tests—not for nothing, a universal policy. The rule was that each unique address could order only a certain number of free tests. Initially, the postal service just asked for a requester’s address. But it turned out that, occasionally, “one apartment dweller requesting tests would blacklist other units in the same building.” This wasn’t a programming error. It was a problem with the post office’s records, which hadn’t been updated to reflect division of a building into apartments, even though individual mail carriers were compensating for that when they walked their routes. To compensate, the team added a process involving human review of edge cases, whereby an individual could fill out a short form appealing a denial as error. Pahlka acknowledges that this process disproportionately burdened lower-income individuals. But it also cleaned up around two-thirds of the residential address database as a result.
Even though it sounds scary and even undemocratic to have a random technologist embedded deep in the hierarchy making important distinctions, Pahlka argues that it’s necessary for the success of the actual intended outcomes decided upon by elected representatives. When no one can go ahead and make decisions about how a program should work, but lots of people have the power to add requirements to it—as is now the case—you get lots of paperwork and few good outcomes. Good product management can “reimagine representation and voice so as to honor the values our government is supposed to be founded on.”
To further improve things, Pahlka argues that the government should spend money improving its human resources, especially at the levels of program management/operating expenses. Oversight should ask less about whether a team stuck to a plan, and more about what the team learned in implementation and what user tests are showing now.
There are some things this book misses. Pahlka, who’s not a lawyer, doesn’t suggest that new laws should explicitly allow regulators to easily simplify and even eliminate earlier categories and rules. There are certainly reasons why we don’t do that. For example, lots of systems rely on past categories and rules and changing them could cause a cascade of incongruencies. But the accretion of legal complexity keeps making things worse. In my view, it would be a worthwhile exercise to try to write a law that actually allowed agile development of implementation policies—and then probably a painful exercise to see what happened when courts got their hands on it.
Give people leeway to implement the intent of the overall policy, Pahlka suggests, and you can avoid the layers of bureaucracy that stymie well-intentioned attempts at reform. While there’s merit in the argument, she doesn’t give a lot of weight to the reasons that policymakers try to be comprehensive. Although the perfect shouldn’t be the enemy of the good, it’s also the case that if you get a policy running that works for 90% of people, the 10% excluded are likely to share some demographic characteristics, and historically the policy is unlikely to be revisited to fix it for them. That result is usually worse when it comes from government than when it comes in private software. But if the perfect is not to be the enemy of the good, then oversight that focuses on implementation success, and flexibility to keep working for that 10%, are the proper solutions.
Oct 8, 2024 Natalie Ram
Following the Supreme Court’s decision in Dobbs v. Jackson Women’s Health, pregnant women seeking to terminate a pregnancy, and medical providers who care for them, have found themselves increasingly subject to invasive law enforcement scrutiny in many states. For instance, while many states’ anti-abortion laws permit abortion if the pregnant person was the victim of a sexual assault, many of these laws require that physicians verify that the sexual assault was reported to law enforcement. The exception thus compels physicians to serve as handmaidens to the police.
Yet the abortion context is hardly the first or only one where policing has thrust itself into medical practice. As Teneille R. Brown observes in her new article, When Doctors Become Cops, from gender-affirming care, to prescription drug monitoring programs, to law enforcement demands for DNA samples from hospital staff, policing often encroaches on patient privacy. These intrusions generate medical mistrust that undermines both individual and public health. Moreover, this medical mistrust is likely to exacerbate inequities in population health, as police mistrust is at “record highs” and structural inequities are present in “virtually all aspects of the criminal legal system.” Brown persuasively argues that “[t]o respect patient autonomy, repair medical mistrust, and promote individual and public health,” “law enforcement and health care need to be more completely divorced from one another.”
Brown begins by demonstrating that medical mistrust is a social determinant of health that can only be worsened by injecting policing further into medical relationships. Observing that trust is “vital” to clinical care and that mistrust is a “major barrier to a strong patient-clinician relationship,” Brown cautions that medical mistrust is already a persistent concern across the modern American medical system. Under the common American fee-for-service medical model, “patients and physicians have precious little time to build trust,” as hospitals and physicians are paid “for doing things, but not for talking about whether and how to do things.”
Brown then explains how medical mistrust negatively affects health, both individual and population-wide. Medical mistrust “leads patients to refuse prescribed medications, to miss cancer screenings, to not see their doctor for regular visits, to discourage others from seeking treatment, to not share sensitive medical information with their providers, and to be less likely to comply with the prescribed treatment or health care plan.” Moreover, medical mistrust and its consequences are even worse for already marginalized communities. Medical history is replete with injustice and mistreatment based on race, sex, and other characteristics. Modern medical practice is often no better. As Brown observes, “[i]nfant mortality for Black babies is higher now than it was during the antebellum period.”
Turning to policing practices, Brown explains that there are “few legal hurdles” preventing law enforcement from accessing or using confidential patient data. Fourth Amendment law, which could act as a robust barrier to access, has instead often bent to law enforcement demands. Even if the Fourth Amendment were a hardier guardrail, it is practically triggered only by introducing medical data at trial. Prosecutors might simply avoid doing so, where possible, leaving many privacy violations unremedied. In the case of DNA identifications, police might use medical data from a suspect’s genetic relatives, rather than that from the suspect himself, in an effort to immunize their investigation from Fourth Amendment scrutiny. (In my own work, I have argued that this kind of end-run should not negate a Fourth Amendment claim.) Even where courts conclude that police conduct has violated the Fourth Amendment, moreover, courts often deny exclusion of tainted evidence anyway, based on officers’ “good faith” misunderstanding of the law.
Statutory protections for medical data, including both state privacy laws and the federal HIPAA Privacy Rule, are equally unavailing. HIPAA permits medical providers to share otherwise-protected health information with law enforcement in response to as little as an “administrative subpoena”—a statement police write themselves, without any judicial oversight. Medical providers receiving such requests are likely to cooperate with them, given messy Fourth Amendment law and the power dynamics exerted by (often armed) police.
Brown then argues that it essential to protect the “culture of medicine” from the “culture of policing” because they have very different norms and cultures regarding self-regulation, privacy, accountability, efficacy, honesty, autonomy, and trust.
With respect to self-regulation and accountability, while physicians “extensively self-regulate through governing bodies and professional associations” and “are frequently civilly sued and held accountable for malpractice,” police “rarely hold themselves accountable for the violence that they perpetrate, which is often not just careless, but intentional.” Legal doctrines like the “public duty doctrine” and qualified immunity also often shield law enforcement from liability for violence they could have prevented. By contrast, legal accountability for medical providers has expanded, following the famous Tarasoff case, often to require medical professionals to breach patient confidentiality to warn of “imminent risks to third parties.”
Similarly, with respect to privacy, while every state has codified a physician-patient privilege, there are no confidentiality norms or legal requirements for information shared with law enforcement. Police have tapped clinical laboratories, biobanks, and newborn screening programs for biological samples used to identify or confirm the identity of a criminal suspect.
With respect to efficacy, law enforcement doles out pseudo-medical interventions that are largely ineffective, substandard, and not evidence based, like administering ketamine for “excited delirium” (a highly contested diagnosis) or acting as first responders more broadly. “Treatment courts,” which divert offenders to addiction or other treatment programs overseen by courts, often rely on underregulated programs and clinics and blend punishment with disease treatment. By contrast, medical norms require evidence of safety and efficacy before treatments should be offered to patients.
Regarding honesty, modern medical ethics “universally condemn” deception, while police routinely use deception to get witnesses to cooperate or obtain evidence. Similar are differing approaches to autonomy. Medical providers seek to equip patients to make decisions consistent with the patient’s values. Meanwhile, legislatures charge law enforcement to criminally enforce moral judgments across the population, as in the context of abortion regulation or banning of gender-affirming care.
Finally, Brown returns to trust itself, showing that prescription drug monitoring programs (PDMPs), which track prescriptions and patient requests for controlled substances like opiates, “place law enforcement between a patient and their physician and can violate the trust between them.” Physicians, fearing law enforcement oversight, may under-prescribe needed pain medications, and patients may sensibly view their doctor as an extension of law enforcement when the doctor “check[s] a police database to see if the patient is telling the truth.”
Brown concludes by identifying five strategies to protect medical care from cooptation by law enforcement. First, the federal government should amend HIPAA to make it more difficult for law enforcement to obtain medical data. Second, courts should reconsider Tarasoff-style duties to warn third parties, which have had the perverse effect of “tak[ing] the very thing that makes health care special—confidentiality and patient trust—and exploit[ing] it in a way that harms not only public health, but also medical ethics.” Third, physicians must have autonomy to practice medicine in an ethical manner, something the criminalization of abortion care, for instance, has undermined even where the standard of care is clear. Fourth, medical providers would benefit from training about which disclosures the HIPAA Privacy Rule are permissive, rather than mandatory. Finally, we all must “reimagine health care as being off-limits from police.” Many social ills, currently funneled through the carceral system, would benefit from more nuanced, sensitive, and effective responses.
Brown’s article skillfully brings together three strands of law enforcement encroachment that have largely been analyzed separately to date. The first are recent state laws that functionally compel medical providers to comply with new prosecutorial demands or face criminal penalties or loss of licensure. The second consists of law enforcement efforts to tap the enormous “reservoir of evidence” held in individual medical files and biological samples. The third includes instances in which substandard medical care is dispensed by the criminal legal system directly. By weaving these entanglements into a single story, Brown broadens the narrative on medical mistrust and lends urgency to her call to build stronger, higher barriers between medicine and policing.
Sep 10, 2024 Rebecca Crootof
When a digital financial or medical advisor gives bad advice, when ChatGPT confabulates that a law professor committed sexual assault, when an autonomous weapon system takes action that looks like a war crime—who should be held liable?
Bryan Choi’s excellent AI Malpractice makes an important but often overlooked point: the answer isn’t as simple as choosing between negligence and various other potential regimes (strict liability, products liability, enterprise liability, etc.). That’s an important first step, and for a host of reasons, I share Choi’s conclusion that strict liability is the preferable near-term standard. But as AI agents and decisionmaking technologies proliferate and judges consider the applicability of negligence, there is a critical second order question: In a negligence regime, what standard should be applied for evaluating if a duty was breached? Should AI developers’ choices be evaluated according to the default reasonable person standard? Or, like doctors and lawyers, should their acts be evaluated under a professional standard of care? Under the former, a jury evaluates whether a defendant’s act was reasonable; under the latter, the profession sets the bar.
The choice has far-reaching implications. As Choi notes, if the professional standard is applied, the “law will enforce the customary practices among the AI community.” This would empower those pushing for better design and development policies, as AI ethics principles would have new legal weight and enforcement mechanisms. Meanwhile, if the reasonable care standard is applied, AI modelers may face far greater liability risk, as those harmed by AI systems might be better able to obtain civil recourse.
Consider OpenAI, the company which developed and launched ChatGPT. What is its potential tort liability for the myriad types of harm its generated output facilitates? Cybercriminals have already leveraged ChatGPT’s capabilities to improve social engineering attacks, resulting in malicious phishing emails increasing by 1,265% in the year following its release. Fact checkers are scrambling to address the deluge of LLM-created misinformation. ChatGPT can produce dangerous content, like recommending self-harm or providing instructions on how to commit crimes: A man committed suicide after allegedly being encouraged to do so by ChatGPT, and a Vice reporter learned how to make crack cocaine and smuggle it into Europe. And it can enable the discovery and deployment of new chemical and biological weapons, as when ChatGPT gave amateurs step-by-step instructions on how to cause a pandemic.
If a suit based on any of these harms is evaluated under the reasonable person standard, OpenAI may face a lot of liability. These risks were all foreseeable. In fact, a March 2023 OpenAI publication details its identification of and attempts to mitigate these categories of harms. The question would be whether those efforts were reasonable—and a jury could find that the mitigations were insufficient or that the technology shouldn’t have been released as widely as it was. If, however, a claim is evaluated under a professional standard of care, OpenAI has a credible argument that they are doing far more than most similarly-situated companies to mitigate harms—certainly more than custom requires, given the lack of responsible release norms—and therefore cannot be held liable for resulting unintended harms.
Talk about incentives.
(This is often where some folks default to talking points about the need to promote innovation and protect innovators from liability—and that is a valid consideration, but let’s say the silent corollary out loud. Harms have been created. If AI producers are not liable for the harms they cause and there are no alternative means of redress, the full costs of those harms are borne by the public, individually and collectively.)
To answer the question of which standard should be employed, Choi employs a framework he previously developed for assessing when the professional standard should be applied to a particular industry. One might think that certain professions set their own standard of care because they have particular traits—required degrees, licensing professions, and codes of ethics. But Choi argues that this is historically inaccurate, as doctors were evaluated under the professional standard long before they were “professionals” as we think of them today. Rather, he suggests that the professional standard is applied in situations where judges have good reason to not trust jury instincts and sentiments, as things can go terribly wrong even when a defendant doctor or lawyer does everything right.
Choi’s professional standard framework requires evaluating three factors: (1) “whether the core elements of the work involve substantial uncertainties in knowledge, and therefore require latitude for discretionary judgment”; (2) “whether there are serious harms that are statistically unavoidable because of the lack of scientific precision or control”; and (3) whether “[the defendants] perform an essential societal service even when their customary practices cause harm.”
One thing I love about this piece is that Choi’s exploration of the first two factors provides an impressively detailed and nuanced yet succinct description of what AI development entails. In describing the core elements of the work and where judgement is exercised, Choi clarifies where the relevant design decisions occur, the benefits and risks associated with different choices, and common sources of bias and error.
After reviewing the AI development process in his evaluation of the first factor, Choi concludes that its status as more of an ‘art’ than a ‘science’ weighs in favor of judges applying a professional standard. He notes that, while “[m]uch of the work involved in training neural networks is either menial or guided by well-established mathematical principles,” there is still “an important component [that] involves subjective judgements that are guided by customary practices derived from trial-and-error.”
In his analysis of the second factor, Choi creates a useful 3×2 typology matrix of potential harms. Along one axis, he distinguishes accidental harms, intended harms, and foreseeable misuses; along the other, he distinguishes between harms resulting from inaccurate and incomplete data from harms resulting from the accurate perpetuation of historical bias in data sets. He determines that “harmful outcomes are an expected feature even of competent AI modeling work,” which in his view also weighs in favor of a professional standard.
Compared with the first two factors, Choi’s analysis of the third is slightly perfunctory: He quickly states that AI developers do not (yet) perform an essential societal service, which favors the reasonable care standard. He concludes that, given the nascency of the field, strict liability is the appropriate current standard; however, once “AI becomes an ordinary fixture of everyday society”—at which point AI modelers may be providing more of an essential service—courts will need to wrestle with what negligence standard should be applied.
Jack Balkin has observed that legal analysis has a fractal nature, as the resolution of one question raises a host of others. Deciding to evaluate AI liability under negligence raises the “which standard” question; Choi’s exploration of the “which standard” question raises further ones.
First, each of Choi’s factors include major wiggle words. When is an uncertainty sufficiently “substantial”? When are harms sufficiently “serious”? At what point does a tool become a “societal service”—and when does that service become “essential”? As any despairing 1L will tell you, wiggle words pervade the negligence analysis—what, after all, constitutes “reasonable” care?—but resolving the squishy questions is usually left to the jury and can vary dramatically based on the facts of a case. In contrast, the selection of a standard is the judge’s role, which more easily becomes precedential—and precedent, once set, can be difficult to shift. How should law approach the risk of inapt legal lock-in in this context?
Second, need this be an all-or-nothing determination? Just as AI companies might have certain acts evaluated under strict liability or negligence regimes, it is possible to apply a professional standard to some types of AI development and deployment choices and an ordinary care standard to others. Even doctors’ acts are sometimes evaluated under the reasonable person standard, such as when a pharmacist inadvertently misfills a prescription.
As a teacher, I would recommend the piece for the introduction-to-AI-development inherent in Choi’s analyses alone. It would make a great opener for any AI and the Law course. As a scholar, I’m grateful to Choi for his thought-provoking exploration of a critical second-order question in the AI liability conversation.
Jul 31, 2024 James Grimmelmann
Sarah B. Lawsky’s Coding the Code: Catala and Computationally Accessible Tax Law offers an exceptionally thoughtful perspective on the automation of legal rules. It provides not just a nuanced analysis of the consequences of translating legal doctrines into computer programs (something many other scholars have done), but also a tutorial in how to do so effectively, with fidelity to the internal structure of law and humility about what computers do and don’t do well.
Coding the Code builds on Lawsky’s previous work on formal logic and its advantages for statutory interpretation. (Formal logic, sometimes called “symbolic” or “mathematical” logic, involves the precise and rigorous analysis of symbolic expressions representing arguments, such as “p & ¬q” to mean “p is true and q is not true”.) In her 2017 A Logic for Statutes, she observed that many statutory provisions have a characteristic structure: rules subject to exceptions. A typical rule says that WHEN certain conditions are satisfied, THEN certain consequences follow, UNLESS one of several exceptions applies. Exceptions have exceptions of their own: interest payments are deductible, unless they are personal, unless they are mortgage payments.
Lawsky’s great insight about law and logic is that this characteristic structure of nested exceptions is most naturally modeled using a branch of formal logic called “default logic.” Default logic, unlike standard “monotonic logic,” allows for tentative conclusions. On the basis of what I know now, this is a nondeductible personal interest payment, but let me investigate further, and oh, I see that this is qualified residence interest, so I am withdrawing my tentative conclusion and replacing it with another tentative conclusion that the payment is deductible. And so on, until there are no more clauses of the statute to check, no more exceptions to explore, and the most recent tentative conclusion becomes a definitive one. It is a process of successive refinement, converging on certainty. Monotonic logic, by sharp contrast, requires ruling out all possibilities before drawing a conclusion, which remains valid for all time once drawn.
Default logic is not more powerful than standard logic, but for some kinds of reasoning it is cleaner, and Lawsky’s point is that the back-and-forth of exceptions and subexceptions in statutory analysis maps naturally onto default logic’s structure of defaults and defeats. A formal logician applying a default logic’s inference rules follows a reasoning process that naturally corresponds to the reasoning process followed by a lawyer working through a statute.
Default logic is also a good tool for programming. (It is a formal logic, after all.) Once a human has translated a natural-language statute into a formal-logic representation, it becomes possible to reason automatically and algorithmically about the statute and how it treats various fact patterns. In 2021, a trio of computer scientists—Denis Merigoux, Nicolas Chataing, and Jonathan Protzenko—published Catala: A Programming Language for the Law, which turned Lawsky’s default-logic analysis of statutes from an abstract formalism useful for pencil-and-paper analysis into a concrete implementation useful for programming. (Lawsky herself is now a co-designer of the Catala language.)
As someone who sank years of his life into programming a body of law, I can say that Catala is the cleanest and most broadly useful advance towards making law programmable I have ever seen. A 29-page paper filled with equations and code blocks may be quite daunting (our research group took several weeks to read through the formalisms together in detail), but the basic idea of what it does is beautifully simple and clear. Catala allows a programmer to write the way that lawyers think: by laying out rules that trigger consequences, together with the exceptions that can prevent the consequences from happening.
Tax law in particular has two advantages that make it well-suited for this kind of formalization. First, it depends on—and attempts to produce—clear and determinate answers. Everything comes down to, or should, a specific amount due. And second, much of tax law is what a programmer would “declarative” rather than “imperative”; instead of telling people what to do, it describes the consequences of what they have already done. The Internal Revenue Code, as Lawsky has shown, comprises declarative provisions that are particularly clean to implement in a Catala-style language that uses defaults and exceptions. Taking advantage of this affinity between tax law and programming languages, Merigoux and Protzenko, along with Raphaël Monat, have been developed a toolchain to help the French tax authority modernize its antiquated systems. Their work puts directly into practice Lawsky-ian ideas about the value of clean formal reasoning to improve the application of tax statutes.
Coding the Code is in many ways the summa of Lawsky’s project over the last decade. After an accessible introduction to default logic, other portions of Coding the Code draw on what Lawsky has been up to lately: actually using Catala to code up tax law. She and her collaborators have approached the task with humility and care—virtues that Lawsky describes the need for in interdisciplinary collaborations in the recently-published Computational Law and Epistemic Trespassing. One approach they use is “pair programming”: a lawyer and a computer scientist sit side by side at one computer, discussing a statutory section and making sure that they agree on its translation into code. Another is “literate programming”, in which code is interwoven with comments that document what each part of it is doing. For statutory translations, these comments can include the statutory text itself, making the isomorphism between specification (the statute) and implementation (the code) wholly explicit. Neither pair programming nor literate programming directly affects what the code-ified version of the law does; instead, they are tools to make sure that the people who do the translation do so faithfully, in a way that others who come later can recognize as correct. (Lawrence Lessig, patron saint of code-as-law, would approve.)
Coding the Code, like the rest of Lawsky’s work, stands out in two ways. First, she is actively making it happen, using her insights as a legal scholar and logician to push forward the state of the art. Her Lawsky Practice Problems site—a hand-coded open source app that can generate as many tax exercises as students have the patience to work through—is a pedagogical gem, because it matches the computer science under the hood to the structure of the legal problem. (Her Teaching Algorithms and Algorithms for Teaching documents the app and why it works the way it does.)
Second, Lawsky’s claims about the broader consequences of formal approaches are grounded in a nuanced understanding of what these formal approaches do well and what they do not. Sometimes formalization leads to insight; her recent Reasoning with Formalized Statutes shows how coding up a statute section can reveal unexpected edge cases and drafting mistakes. At other times, formalization is hiding in plain sight. As she observes in 2020’s Form as Formalization, the IRS already walks taxpayers through tax algorithms; its forms provide step-by-step instruction for making tax computations. In every case, Lawsky links carefully links her systemic claims to specific doctrinal examples. She shows not that computational law will change everything, but rather that it is already changing some things, in ways large and small.
It is unusual for an established law professor to go back to school for a PhD. In philosophy. With a dissertation on formal logic. But Coding the Code, published five years after Lawsky submitted her (highly technical) thesis, shows the great value for legal scholars of the approach she developed in her PhD. It refines her distinctive approach to statutory analysis—which mixes careful legal reading with technical tools from formal logic and computer science—in a way that has great potential to help other lawyers and legal scholars be more precise about what tax laws say. All they need to do is talk to computer scientists, and Lawsky provides a roadmap for how. There is no epistemic trespassing in Sarah Lawsky’s work. Everywhere she goes, she is a welcomed guest.
Cite as: James Grimmelmann,
When Law is Code, JOTWELL
(July 31, 2024) (reviewing Sarah B. Lawsky,
Coding the Code: Catala and Computationally Accessible Tax Law, 75
SMU L. Rev. 535 (2022)),
https://cyber.jotwell.com/when-law-is-code/.
Jun 5, 2024 Scott Skinner-Thompson
The power of surveillance capitalists to know us, nudge us, and exploit us continues to expand with alacrity despite Europe’s General Data Protection Regulation (GDPR) and state privacy regulations within the United States. Likewise, consequential and concrete privacy harms (such as sexual privacy violations) and, at first glance, more ethereal but also troublesome privacy losses (such as data breaches) remain under-compensated or under-redressed. Why? And what can be done about it? Ignacio Cofone’s incredible new book, The Privacy Fallacy: Harm and Power in the Information Economy, answers those two questions with a healthy dose of realism about the stakes, the need for substantive command-and-control regulation of the information economy, and the imperative of compensatory liability.
In this beautiful and accessible book Cofone explains that, to date, efforts to rein in data exploitation and redress privacy harms have fallen short because of a few underlying fallacies about how information ecosystems work. One of those misconceptions is law’s failure to understand that many egregious privacy harms, such as online sexual harassment, don’t occur because of just one or two bad actors. Rather, the very existence of data ecosystems makes those harms possible in the first instance and then magnifies them. Glossing over the role of the ecosystem (and the companies that create that ecosystem) in perpetuating these privacy harms means that law-reform efforts targeted toward the initial privacy violator under-deter such conduct and stymie efforts to, in effect, put the cat back in the bag. To address the systematic nature of privacy harms, Cofone boldly—but rightly—suggests that modified private causes of action need to be created through statute or tort law to target the myriad corporations that enable such privacy violations. Importantly, Cofone also explains why class actions are an important part of the solution in order to provide for more efficient and comprehensive liability.
A second fallacy is the belief that procedural justice efforts to amplify users’ control over their data through notice and/or consent regimes will somehow mitigate data losses. Drawing from behavioral economics, Cofone shows how such approaches—embodied in laws like the GDPR—will never be more than a band-aid that lends a false sense of security to users, while allowing surveillance capitalists to harvest information at will. He rightly explains that individuals who “consent” to privacy-invading policies aren’t necessarily apathetic about privacy; rather because of the daunting and incomprehensible nature of privacy notices, unforeseeable risk, and manipulative design choices, they are cajoled into surrendering their privacy. To overcome this fallacy, Cofone grabs the bull by the horns and makes a persuasive case for command-and-control regulation. That is, Cofone explains why simply holding surveillance capitalists to the weak promises contained in the privacy notices/policies won’t ever be sufficient because users don’t have the bandwidth to meaningfully police the content of those polices through notice/choice. As such, the government needs to do the policing for them—needs to dictate the substantive contours of privacy protections, rather than leaving it to the consumers and the market, which will always favor powerful tech companies.
In terms of the substance of those privacy regulations, Cofone also explains why some of the existing privacy regulations have fallen short over time: they focus on a particular technology (video rentals, cookies, etc.) rather than “the underlying relationship intermediated by the technology (surveillance).” As such, Cofone advocates the implementation of rigorous but flexible standards for digital privacy protection that can adapt as technology evolves, helping to prevent law from always playing catch-up to tech.
Make no mistake, Cofone’s proposals for substantive privacy regulation of and remedial causes of action against information capitalists are big. And difficult. They will require significant political will and capital to implement. But to the extent that society values privacy and individual freedom as our lives continue to become digitally mediated, then lawyers, politicians, and voters will need to put their resources behind meaningful substantive privacy laws and meaningful compensatory regimes that target the corporate information ecosystems themselves. In a nutshell, piecemeal procedural interventions and remedies against individual privacy violators are simply not going to cut it. Refreshingly, Cofone’s book gets real about these problems while offering real solutions—if we have the courage to implement them.
May 7, 2024 Nicholson Price
Artificial intelligence (AI) is moving increasingly rapidly into health care (as indeed into everything else). But it has problems there (as indeed everywhere else!). What’s to be done, in particular, about the deeply embedded biases along racial and other lines that permeate the whole world of health and, as such, are likely to be encoded in AI?
Khiara Bridges gives an answer that seems mild but carries roots of revolution. In Race in the Machine: Racial Disparities in Health and Medical AI, she argues that informed consent is a key lever to pull in fighting these racial disparities. But not because informed consent—at present, mostly a formality, a begrudging nod to autonomy—will fix the problem in its current state. Instead, Bridges argues, informed consent, beefed up and focused on conveying the brutal truth about encoded racial disparities, can form the foundation for revolutionary social changes in health care, health, and beyond. Curious? Read on!
The first half of the article comprises Parts I-III. These parts aren’t breaking too much new ground, but they do an excellent job bringing together the literature, often including a host of data and examples, to make their own cases—each of which is prerequisite to the piece’s second half. Part I covers the landscape of health and health-care bias. Part II does AI. And Part III brings the first two together to describe bias in medical AI. To elaborate a bit: Part I traces the causes of different health outcomes for marginalized groups (e.g., substandard housing, poverty, persistent stress caused by racism) and the different treatment of marginalized groups by the medical system (e.g., doctors offering different treatments to Black patients than to White patients). It’s replete with infuriating examples, most reflecting the endemic bias experienced by Black patients in America and many referring to Black maternal health care (the subject of a prior Bridges opus). Part II provides a basic primer on AI (probably skippable by AI-conversant readers, but otherwise a helpful foundation), a handy overview of AI in medicine (again, skippable if familiar, but that’s far fewer folks), and a discussion of the potential uses of medical AI in prenatal care—this last a bringing together of technologies and the medical literature that is both novel and insightful. In Part III, discussing bias in medical AI, Bridges explores pre-mapped territory, but she walks it carefully and thoughtfully and shines new light on it, including through trenchant examples from prenatal care. She lays out different sources of bias—design choices, inadequate data, data that accurately reflect inequitable systems, and the pernicious encoding of race (often deliberate). After this recounting, Bridges emphasizes an underappreciated point: the problem isn’t just that AI will encode existing biases—it’s that it will wrap them in “the veneer of objectivity,” thus leaving minorities ultimately worse off—because they suffer the same injustice, but this time it’s coded as just the machine being impartially correct (what Ifroma Ajunwa dubs “data objectivity”).
The second half (Part IV) is the heart of the article; it starts out smart and interesting and reasonable and winds up smart and interesting and audacious. In a good way!
Starting with the new and smart but not socks-removing: Bridges presents convincing evidence that people of color may well not want to have medical AI involved in their care. She recounts studies of “algorithmic aversion,” where folks are, well, averse to using or trusting algorithms—and demonstrates how this is likely to be particularly forceful for Black patients and medical AI. It’s not just the legacy of Tuskegee, she recounts; it’s the pervasive and ongoing evidence of bias, inequality, and inequity in the health systems of today that compromises trustworthiness. (Indeed, AI may worsen this dynamic.)
So what’s the intervention? Bridges argues for informed consent—telling all patients, but especially those of color, not only that AI is being used in their care, but that the AI is very likely to be biased, based on the deep multimodal biases embedded in health. Why? Well, for starters, there’s the classic story that informed consent respects autonomy, and patients (especially of color) would want to know, so physicians should tell them. And that’s likely enough.
But there’s more, and here’s where it gets radical. Bridges draws on literature grounding informed consent in the Nuremberg Trials and recasting it “as a rebuke of Nazi ‘medicine,’ eugenics, anti-Semitism, racism, and white supremacy.” (P. 316.) This “rebelliousness” underlying informed consent, Bridges forcefully posits, can be grounds for a broader social revolution—if we tell patients, truly and meaningfully, about the inequities embedded in the system and informing their care, that may plant the seeds for making society actually better—a goal vastly preferable to the Faustian outcome where somehow-improved algorithms paper over unaddressed social inequities.
It’s a provocative, fascinating, and persuasive argument. (I’m primed to resist—I’m part of the benighted crew that’s argued that informed consent probably isn’t specially needed for medical AI—but I’m far less complacent than I used to be.)
And of course Bridges’ argument stretches beyond AI in medicine, to reach medical care generally and indeed areas outside medicine. Bridges mentions this, and if it’s not deeply explored in the piece, that may be because the piece is already just shy of a hundred pages. But one can see the radical, beguiling arguments reaching forward. And that’s a gratifying uneasiness with which to leave this challenging and excellent piece.
A postscript: It’d be a shame to review the article without mentioning Bridges’ engrossing prose. Some sesquipedalian sentences are pure pleasure to peruse: “These defiant, revolutionary origins have been expunged from the perfunctory form that the informed consent process has taken at present.” (P. 250.) Some paragraphs are short and pungent; in others, Bridges deploys an avalanche of data and studies. One crushing paragraph in the introduction has a relentless and irresistible litany of “for examples”’s, each a drumbeat pounding home her point about embedded disparity. It’s a pleasure to move through the weighty arguments with Bridges’ writing carrying you along.
Apr 3, 2024 Ari Waldman
A few years ago, the CBS show 60 Minutes ran a segment about algorithmic bias. Among other things, the segment told the story of a Black man in Detroit who had been misidentified by a facial recognition algorithm as the thief who had been caught on video stealing $4,000 worth of watches. Anderson Cooper interviewed the victim, his wife, the Detroit police chief, a lawyer at Georgetown’s Center on Privacy and Technology, and a computer scientist at the National Institute of Standards and Technology (NIST). The NIST computer scientist, Patrick Grother, was interviewed because “[e]very year, more than a hundred facial recognition developers around the world send his lab prototypes of test for accuracy.” For 60 Minutes that day, Mr. Grother was the expert on these algorithms.
No offense to Mr. Grother, who I’m sure is a smart guy, but the real experts were missing. Mr. Cooper did not interview the Black women who first brought the problem of algorithmic bias in facial recognition to academic and public attention: Joy Buolamwini, Timnit Gebru, and Deborah Raji. The incident captured in just a few minutes the long history of erasing the contributions of women and of women of color in particular. NIST, which had cited Buolamwini, Gebru, and Raji in its report, got its moment in the sun but never mentioned the underlying research; the police chief got his chance to cast the wrongful arrest as an isolated mistake. The Black women who founded the field were ignored, a fact even more remarkable given that 60 Minutes researchers and producers spent time with Buolamwini in preparation! That this happened in 2021 highlights the desperate need for Meg Leta Jones’s and Amanda Levendowski’s edited volume, Feminist Cyberlaw.
With contributions from established and up-and-coming scholars, including, among others, Elizabeth Joh (UC Davis), Ngozi Okidegbe (Boston University), Hannah Bloch-Wehba (Texas A&M), Jasmine McNealy (University of Florida), Alexandra Roberts (Northeastern), and Anjali Vats (University of Pittsburg), Feminist Cyberlaw aims to inform and challenge. It informs by looking at cyberlaw issues like ownership, privacy, the First Amendment, labor, and security through a feminist lens. It challenges its readers to consider not just how feminist theory reconsiders old internet law debates, but how the work of cyberlaw over the last few decades has marginalized feminist perspectives.
In what ways have feminist perspectives been marginalized? The cyberlaw canon, which Professor Jones frames in her introduction as advocating a kind of universality and perfect equality, elided the connective tissue between cyberspace and the physical world. Feminist work, particularly feminist critiques of Science and Technology Studies, inhabits that connective tissue and, thus, gives the readers of Feminist Cyberlaw a new way of looking at technology. It gives voice to the people who may not be in the room when technology is designed but who are most definitely acted upon and act with technologies both designed for them and without them.
So, Feminist Cyberlaw gives us context. It also centers the lived experiences of women, those living with disabilities, queer people, gender variant populations, and women of color. Novelty and generativity may have been the watchwords of the cyberlaw canon, but feminist approaches to technology surface the real-world effects of innovation, law, and institutions that have never been and never will be borne equally. Notice, too, that Feminist Cyberlaw is not limited by narrowness of first wave feminism. The “feminism” in Feminist Cyberlaw is hooksian, that is, indebted to people like bell hooks, who saw recognition of the many different ways different women are subordinated by different systems of oppression, including white supremacy, heteronormativity, ableism, ageism, and more.
What I like most about Feminist Cyberlaw is its diversity of perspectives within an overarching frame. Although broadly clustered into sections on ownership, access, and governance, contributions to the volume range from intellectual property to data security, from the First Amendment to labor, from the Global South to China. The authors are law professors, practitioners, independent researchers, and community organizers. Elizabeth Joh’s chapter on the effects of eliminating a federal right to abortion in a world of ubiquitous surveillance serves as an important reminder that the Supreme Court’s decision in Dobbs v. Jackson Women’s Health does not mark a return to the pre-Roe v. Wade world, but rather places those who can get pregnant in a state of perpetual risk. Michela Meister and Karen Levy go even further in their excellent essay on what feminist lens can teach us about privacy invasions. They show that Dobbs is actually a broader case study in the ways in which women’s loss of control over their bodies is part of a broader pattern of patriarchal surveillance. Gabrielle Rejouis’s essay on taking an intersectional lens to content moderation is a welcome contribution to a content moderation literature that is too often satisfied with elite liberals’ best efforts.
I also appreciate Professor Jones’s eagerness to situate Feminist Cyberlaw in its own historical context. She places the text on the shoulders of a litany of critical scholars, many of them women, who have been challenging patriarchy, oligarchy, and heteronormativity in law and technology for decades. Julie Cohen, Danielle Keats Citron, Catherine MacKinnon, and Anita Allen deserve particular mention here. Professor Allen, who has been writing about cyberlaw broadly defined for nearly forty years, must be considered among the original canonical scholars of cyberlaw. That she is often defined—including in this text—not as canon but as an early critical voice responding to likes of John Perry Barlow and Larry Lessig (Professor Jones leaves out the late Joel Reidenberg)—is itself a testament to how academia and academic culture view the contributions of Black women. Julie Cohen has been offering polymathic articles and treatises on intellectual property and privacy for decades, as well, challenging the tech bro-y insistence that cyberspace is somehow above the law, above the physical world, and above regulation. Professor Cohen’s latest book, Between Truth and Power, is a political, legal, and economic tour through the ways law has been integral to the formation of informational capitalism’s hierarchies. Feminist Cyberlaw takes many of the broad themes of Between Truth and Power and of Professor Citron’s groundbreaking work on intimate privacy and establishes them as Feminist—with a capital F—inspirations.
By the time readers finish Feminist Cyberlaw’s economical and digestible essays, they will come away with an appreciation for traditionally marginalized voices and more questions to pursue in their own research. Feminist Cyberlaw is a call to action to build from critique to change, to use the feminist lens to develop new paths forward in the information age. Whether that means assessing the value of proposals against how they rebalance power or centering the experiences of women as the future of law and technology, Feminist Cyberlaw is a profoundly important step in a positive direction.
Mar 7, 2024 Paul Ohm
Emily Black, John Logan Koepke, Pauline T. Kim, Solon Barocas & Mingwei Hsu,
Less Discriminatory Algorithms, __
Geo. L.J. __ (forthcoming);
Wash. U. Legal Studies Rsch. Paper (forthcoming), available at
SSRN (Oct. 2, 2023).
We are likely to see as many new law review articles about artificial intelligence and law in the next year as the legal academy has produced since the dawn of AI. Those writing about AI for the very first time (*eyes a bunch of copyright scholars suspiciously*) would do well to engage deeply with the work of bias and discrimination scholars who have been writing some of the best, most insightful articles about AI and law for more than a decade. The frameworks and insights they have developed give us a way of thinking about AI law and policy beyond just considerations about Title VII and Equal Protection. A wonderful place to start is with the best contribution to this scholarship I have seen in years, Less Discriminatory Algorithms, by an interdisciplinary team of technical, policy, and legal experts.
This article is a form of my favorite genre of legal scholarship: “you all are making an important mistake about how the technology works.” In particular, it takes on the received wisdom that there is a vexing tradeoff between fairness and accuracy when training machine learning models. This supposed tradeoff—that in order to make a biased algorithm fairer, you need to sacrifice some of the model’s accuracy—may be true in theory for idealized and never-seen-in-the-wild maximally accurate models. But nobody ever has the time, compute, money, or energy to even approach this ideal, meaning real models out in the world are far from maximally accurate. People instead declare success as soon as their models are just accurate enough for their purposes.
The thing about these less-than-maximally-accurate models is that there are many other possible models the model builders could have trained that would have been nearly as accurate, while doling out different false positives and false negatives. Some of these nearly-as-accurate models would likely have better distributional outcomes for protected classes and other vulnerable populations. This is what two of the article’s authors, Emily Black and Solon Barocas, have dubbed “model multiplicity” in the computer science literature. Model multiplicity means that if those training a model would continue to experiment—tweaking a hyperparameter here, selecting or deselecting a feature there, or preprocessing the data a little bit more—they would probably find many other equally (or nearly as) accurate models that allocated the winners and losers differently.
Importantly, many of these other nearly-as-accurate models might also be more fair, less biased, and less discriminatory than the one actually deployed. Rather than saying that there is a tradeoff between fairness and accuracy, we should instead understand that the tradeoff is between accepting a given level of bias or discrimination versus spending a little more time, money, and (carbon emitting) compute to find an alternative that improves on fairness without sacrificing accuracy.
Happily, Black and Barocas have found a brilliant and interdisciplinary team of coauthors: noted antidiscrimination scholar Pauline Kim and two amazing tech-meets-law researchers from the incredible nonprofit Upturn, Logan Koepke and Mingwei Hsu. The group successfully connects these engineering insights to legal doctrine. Model multiplicity’s recasting of the fairness-accuracy tradeoff has a direct bearing on the so-called third step of Title VII’s disparate impact analysis: the requirement that plaintiffs demonstrate a less discriminatory alternative (LDA). (I’m omitting the article’s careful discussion of how other civil rights statutes, including ECOA and the FHA, handle disparate impact analysis a bit differently, but to the same general effect.) This means that plaintiffs might be able to win in the third step of the test, by citing this paper and bringing on an expert who can help find the fairer, just-as-accurate model-not-taken.
Finding the less discriminatory alternative would still be a daunting path. It would require a plaintiff to have access to the exact training environment the defendant used—including all of the training data, which defendants are sure to resist producing. If plaintiffs clear this formidable discovery challenge, they will still need to spend a lot of time and money to find the less discriminatory alternative. All of this might be insurmountably burdensome for plaintiffs, limiting the change that model multiplicity might bring to civil rights law.
The authors, however, have both technical and legal responses to these evidentiary challenges. If the burden were instead placed on the defendant at step two to show that they looked for less discriminatory alternatives during training, it would avoid the inefficiency, delay, and discovery hassles associated with burdening the plaintiff after the fact. Kim and her coauthors find judicial precedent for imposing this obligation on defendants at step two, while conceding that this interpretation has not yet been broadly adopted. Importantly, once this approach is understood to be required by civil rights law, it will impose on model builders the duty to start looking for these alternatives during training. According to model multiplicity, they will often find them! The result will be more model builders finding and deploying less discriminatory alternatives, meaning the win-win of fewer people suffering from discrimination and less litigation exposure for employers!
I like (lots!) so much about this article. It imports a simple but powerful and elegant technical insight—model multiplicity—into legal scholarship. It does so while also advancing a much-needed reform in the way we interpret civil rights law, putting the burden of looking for less discriminatory alternatives on defendants rather than plaintiffs. It is grounded and specific, explaining exactly why this should happen and how the three-step test and the correlative duty to search for less discriminatory alternatives ought to be interpreted; I can imagine courts across the country implementing this article’s prescriptions directly. It does not shy away from technical and legal detail, taking deep dives into the techniques used to find a less discriminatory alternative, how nearly accurate a less discriminatory alternative must be, or how much effort a model builder must spend in trying to find one, to list only three examples.
Finally, legal scholars must understand that model multiplicity matters beyond civil rights law. An obvious extension is to products liability law. Model multiplicity will provide a pathway for injured plaintiffs looking for a “reasonable alternative design” to support claims of defective products design.
We will soon see AI models that do not live up to our legal and societal ideals in many other ways, such as large language models that spout misinformation, deep fake models used to terrorize women, and facial recognition models that destroy privacy. As we fight over what we should do about the bad effects of models, we should understand that they are often the effects of bad models. Model multiplicity means refusing to let model builders rest on the expediency of having done the bare minimum. Keep trying; keep searching; keep looking, and you just may find models that both do well and do good.
Cite as: Paul Ohm,
Debunking AI’s Supposed Fairness-accuracy Tradeoff, JOTWELL
(March 7, 2024) (reviewing Emily Black, John Logan Koepke, Pauline T. Kim, Solon Barocas & Mingwei Hsu,
Less Discriminatory Algorithms, __
Geo. L.J. __ (forthcoming);
Wash. U. Legal Studies Rsch. Paper (forthcoming), available at SSRN (Oct. 2, 2023)),
https://cyber.jotwell.com/debunking-ais-supposed-fairness-accuracy-tradeoff/.
Feb 6, 2024 Margot Kaminski
How should we think about liability when AI systems generate illegal speech? The Journal of Free Speech Law, a peer-edited journal, ran a topical 2023 symposium on Artificial Intelligence and Speech that is a must-read. This JOT addresses two symposium pieces that take particularly interesting and interlocking approaches to the question of liability for AI-generated content: Jane Bambauer’s Negligent AI Speech: Some Thoughts about Duty, and Nina Brown’s Bots Behaving Badly: A Products Liability Approach to Chatbot-Generated Defamation. These articles evidence how the law constructs technology: the diverse tools in the legal sensemaking toolkit that are important to pull out every time somebody shouts “disruption!”
Each author offers a cogent discussion of possible legal frameworks for liability, moving beyond debates about First Amendment coverage of AI speech to imagine how substantive tort law will work. While these are not strictly speaking First Amendment pieces, exploring the application of liability rules for AI is important, even crucial, for understanding how courts might shape First Amendment law. First Amendment doctrine often hinges on the laws to which it is applied. By focusing on substantive tort law, Bambauer and Brown take the as-yet largely abstract First Amendment conversation to a much-welcomed pragmatic yet creative place.
What makes these two articles stand out is that they each address AI-generated speech that is illegal—that is, speech that is or should be unprotected by the First Amendment, even if First Amendment coverage extends to AI-generated content. Bambauer talks about speech that physically hurts people, a category around which courts have been conducting free-speech line-drawing for decades; Brown talks about defamation, which is a historically unprotected category of speech. While a number of scholars have discussed whether the First Amendment covers AI-generated speech, until this symposium there was little discussion of how the doctrine might adapt to handle liability for content that’s clearly unprotected.
Bambauer’s and Brown’s articles are neatly complimentary. Bambauer addresses duties of care that might arise when AI misrepresentations result in physical harm to a user or third parties. Brown addresses a products-liability approach to AI-generated defamation. Another related symposium piece that squarely takes on the question of liability for illegal speech is Eugene Volokh’s Large Libel Models? Liability for AI Output. The Brown and Bambauer pieces speak more directly to each other in imagining and applying two overlapping foundational liability frameworks, while Volokh’s piece focuses on developing a sui generis version of developer liability called “notice-and-blocking” that he grounds in Brown’s idea of using products liability as a starting point. That is, Bambauer and Brown provide the necessary building blocks; Volokh’s article is an example of how one might further manipulate them.
Bambauer writes of state tort liability, as it might be modified by state courts incorporating free speech values. She explains that she has “little doubt that the output of AI speech programs will be covered by free speech protections” (P. 347) (as do my co-authors and I) but also that “the First Amendment does not create anything like an absolute immunity to regulatory intervention,” especially when it comes to negligence claims for physical harm. (P. 348.) Bambauer convincingly claims that the duty element of negligence is where the rubber will hit the road in state courts when it comes to determining the right balance between preventing physical harms and protecting free speech values. She identifies different categories of duty as an effective way of categorizing existing cases that address analogous problems (from books that mis-identify poisonous mushrooms as edible, to doctors who provide dangerously incorrect information to patients).
Bambauer divides her discussion of duty into three broad categories, followed by additional subcategories: 1) situations where AI systems provide information to a user that causes physical harm to that user; 2) situations where AI systems provide information to a user who then causes physical harm to a third party; and 3) situations where AI systems would have provided accurate information that could have averted harm, had a user consulted them (reminiscent of Ian Kerr’s and Michael Froomkin’s prescient work on the impact of machine learning on physician liability). Throughout, this article is logical, clearly organized, factually grounded, and neatly coherent, even where a reader might depart from its substantive claims.
These categories allow Bambauer to tour the reader through available analogies, comparing AI “to pure speech products, to strangers, or to professional advisors” and more. (P. 360.) If an AI system’s erroneous output is analogized to a book, Bambauer argues that developers will not and should not be found liable, as with a book that misidentified poisonous mushrooms as edible as in the Ninth Circuit’s Winter case. (Eerily, this exact fact pattern has already arisen with AI-generated foraging books.) If, under different factual circumstances, AI-generated content is more appropriately analogized to professional advice in a specialized domain such as law or medicine, there might be a higher duty of care. Or, courts might use a “useful, if strained analogy” of “wandering precocious children,” where parents/developers might be held liable under theories of “negligent supervision” for failing to anticipate where their child/generative AI might be doing dangerous things. (P. 356.) This might, Bambauer muses, nudge courts to focus on what mechanisms an AI developer has put in place to find and mitigate recurring harms. This is a classic “which existing analogy should apply to new things?” article, but done well. Others might take this logic further by pulling analogies from other spaces (I’m thinking here for example of Bryan Choi’s work on car crashes, code crashes, and programmers’ duties of care).
This takes us to Brown’s intervention. Brown examines defamation claims through a products liability lens, asking what interventions a developer might be required to take to mitigate the known risk of defamatory content. Brown starts with a summary of how chatbots work, so the rest of us don’t have to. (I will be citing this section often.) She quickly and clearly explains the defamation puzzle: that the current law focuses largely on the intent of the speaker/publisher of defamatory content. This approach runs into issues when we are talking about the developers of AI systems, who Brown argues will almost never have the requisite intent under current defamation law.
Brown then turns to dismantling hurdles to a products liability approach (is it a product? What’s the role of economic loss doctrine?). Readers may find this part more or less convincing, but resolving the hurdles (it’s a product, she thinks economic loss doctrine is not a problem) allows her to get to the really interesting part of the article: what substantive duties a developer might have, if AI-generated defamation gets framed as a products liability problem. Brown argues that “a design defect could exist if the model was designed in a way that made it likely to generate defamatory statements.” (P. 410.) She provides concrete examples grounded in current developer practices: the use of flawed datasets rife with false content; the prioritization of sensational content over accuracy; a failure to take steps to reduce the likelihood of hallucinations; a failure to test the system.
I’m still not sure a products liability approach will survive the Supreme Court’s recent emphasis on scienter in First Amendment cases, but one can hope. In several recent cases, most prominently in Counterman v. Colorado, the Supreme Court has insisted on a heightened intent standard for unprotected speech in order to protect speakers from a chilling effect that occurs if one cannot clearly determine whether one’s speech is unprotected or protected. In Counterman, the unprotected speech category at issue was true threats, which the Court found could not be determined under an objective standard but required a query of speaker intent. The Court reasoned that a heightened intent standard creates a penumbra of protection for borderline speech that is close to but not unprotected speech—such as opinionated criticism of a public figure bordering on defamation, or vigorous political speech at a rally bordering on incitement. Brown presents the products-liability approach as a sort of hack to get around the specific intent requirement of “actual malice” for defamation of public figures (private figures require only negligence, but arguably a heightened form of it). She does not really inquire about whether this is possible—whether today’s Court, post-Counterman, would accept this move. I personally think there is space in the Court’s reasoning in Counterman for moving away from specific intent, but it would have been nice to know Brown’s thoughts.
Together, these two articles offer a trio of important contributions: foundations for First Amendment debates about unprotected speech and AI systems; creative but grounded ways of imagining duties of care in the context of developer liability (relevant, too, to evolving discussions of platform liability); and an important basis for discussions about the role of tort law in establishing risk mitigation for content-generating AI systems in the U.S. legal context. Regulators have increasingly defaulted to a regulatory approach to risk mitigation for AI systems, including or especially in the EU. If, as is likely, the United States fails to enact its counterpart to, the Digital Services Act (DSA), Europe’s massive new law regulating content moderation, tort law may be where AI risk mitigation plays out in the United States.
Cite as: Margot Kaminski, Risky Speech Systems: Tort Liability for AI-Generated Illegal Speech, JOTWELL (February 8, 2024) (reviewing Jane Bambauer, Negligent AI Speech: Some Thoughts about Duty, 3 J. Free Speech L. 344 (2023); Nina Brown, Bots Behaving Badly: A Products Liability Approach to Chatbot-Generated Defamation, 3 J. Free Speech L. 389 (2023)), https://cyber.jotwell.com/risky-speech-sys…d-illegal-speech/.
Jan 5, 2024 Stacy-Ann Elvy
Schools increasingly use various technologies to monitor and collect information about students. The COVID-19 pandemic, which led to a large number of school closures and a transition to online learning, has also raised alarming questions about student privacy. For instance, virtual software used during remote exams to monitor students can scan students’ bedrooms, collect data from the microphones and cameras of students’ computers, and discern students’ keystrokes. In her article, The Private Life of Education, Professor Fanna Gamal makes a noteworthy contribution to scholarship in the privacy law and education law fields by highlighting embedded assumptions and significant shortcomings in privacy law governing student data. In doing so, she advances existing debates on the legal conception of information privacy. Gamal argues that student privacy laws’ immoderate focus on nondisclosure of students’ data outside of the school context fails to effectively consider the various ways in which schools can serve as the primary perpetrators of student privacy violations. She further contends that schools’ data practices may have disproportionate negative implications for members of historically marginalized groups, such as disabled and low-income students.
Gamal expertly critiques the provisions of the Family Educational Rights and Privacy Act (FERPA). She argues that FERPA’s excessive focus on the prohibition of data disclosures outside of schools spuriously assumes that schools should, by default, receive treatment as privacy protectors that act in the best interest of students’ privacy. Gamal aptly acknowledges that FERPA’s heavy reliance on non-disclosure is not unique to American privacy law. However, after unpacking the legal conception of student data privacy, Gamal goes on to convincingly argue that student data privacy law also assumes that students do not have a significant privacy interest in “data creation, collection and recording.” (P. 1319.)
She posits that educational records contain data that “assumes an aura of [uncontestable] truth,” a truth that follows students indefinitely and can impact their lives well beyond the age of majority (P. 1319.) Gamal argues that there is a significant imbalance of power between students and schools. She contends that FERPA grants schools too much power to determine this truth and its life cycle while giving students and parents insufficient mechanisms to contest educational records that contain misleading or false truths. Gamal notes that even when parents have the ability to participate in hearings regarding students’ records, schools have excessive power in those hearings since parents and students have the burden of convincing the educational institution to amend educational records. Gamal suggests that privacy law unnecessarily shelters the internal data practices of educational entities from scrutiny, thereby permitting educational institutions to amass “power over the official archives that shape students’ lives” (P. 1318.) Educational institutions may use this power to infringe on students’ privacy.
Gamal perceptively highlights the impact of student privacy laws’ shortcomings on historically marginalized groups, such as disabled students. She convincingly argues that disability documentation is a “poor proxy for disability” and can further entrench pre-existing inequities (P. 1321.) Gamal admits that disability documentation may help to ensure special education resources go to students in need of such services, but she also notes that heightened documentation requirements may instead stem from “the fear of the ‘disability con,” that is, an irrational fear that some individuals may be dishonest about their disabilities (P. 1321.) She contends that the Individuals with Disabilities Education Act’s (IDEA) data disclosure requirements limit the ability of students who use special education services to obtain privacy from their educational institutions. In contrasting the educational records of non-disabled students and disabled students, Gamal observes that the records of disabled students contain data about their social, medical, physical, and mental diagnoses. Fear of the so-called disability con, Gamal contends, results in requirements that ignore the challenges individuals from marginalized groups may face, such as possible limited access to documentation providers. She also points out that students from racial minorities experience over-representation among special education groups and, as such, disproportionality fall subject to the heightened documentation processes required of students seeking access to special education services.
The well-written article concludes by offering a path forward. Gamal argues for expanding the concept of information privacy in the school setting via a collaborative process that gives voice to various stakeholders. She also proposes several amendments to FERPA, including correcting FERPA’s excessive reliance on non-disclosure outside of the school context, redefining the term “educational records,” and providing students and parents with better tools to amend and delete educational records. She also recommends limits on educational institutions’ power over their internal data practices. Gamal’s convincing description of the limits of the current legal framework regulating student privacy should capture the attention of privacy and educational law scholars interested in learning more about the ways in which narrow conceptions of information privacy can further cement institutional data practices that contribute to existing disparities.
