When a digital financial or medical advisor gives bad advice, when ChatGPT confabulates that a law professor committed sexual assault, when an autonomous weapon system takes action that looks like a war crime—who should be held liable?
Bryan Choi’s excellent AI Malpractice makes an important but often overlooked point: the answer isn’t as simple as choosing between negligence and various other potential regimes (strict liability, products liability, enterprise liability, etc.). That’s an important first step, and for a host of reasons, I share Choi’s conclusion that strict liability is the preferable near-term standard. But as AI agents and decisionmaking technologies proliferate and judges consider the applicability of negligence, there is a critical second order question: In a negligence regime, what standard should be applied for evaluating if a duty was breached? Should AI developers’ choices be evaluated according to the default reasonable person standard? Or, like doctors and lawyers, should their acts be evaluated under a professional standard of care? Under the former, a jury evaluates whether a defendant’s act was reasonable; under the latter, the profession sets the bar.
The choice has far-reaching implications. As Choi notes, if the professional standard is applied, the “law will enforce the customary practices among the AI community.” This would empower those pushing for better design and development policies, as AI ethics principles would have new legal weight and enforcement mechanisms. Meanwhile, if the reasonable care standard is applied, AI modelers may face far greater liability risk, as those harmed by AI systems might be better able to obtain civil recourse.
Consider OpenAI, the company which developed and launched ChatGPT. What is its potential tort liability for the myriad types of harm its generated output facilitates? Cybercriminals have already leveraged ChatGPT’s capabilities to improve social engineering attacks, resulting in malicious phishing emails increasing by 1,265% in the year following its release. Fact checkers are scrambling to address the deluge of LLM-created misinformation. ChatGPT can produce dangerous content, like recommending self-harm or providing instructions on how to commit crimes: A man committed suicide after allegedly being encouraged to do so by ChatGPT, and a Vice reporter learned how to make crack cocaine and smuggle it into Europe. And it can enable the discovery and deployment of new chemical and biological weapons, as when ChatGPT gave amateurs step-by-step instructions on how to cause a pandemic.
If a suit based on any of these harms is evaluated under the reasonable person standard, OpenAI may face a lot of liability. These risks were all foreseeable. In fact, a March 2023 OpenAI publication details its identification of and attempts to mitigate these categories of harms. The question would be whether those efforts were reasonable—and a jury could find that the mitigations were insufficient or that the technology shouldn’t have been released as widely as it was. If, however, a claim is evaluated under a professional standard of care, OpenAI has a credible argument that they are doing far more than most similarly-situated companies to mitigate harms—certainly more than custom requires, given the lack of responsible release norms—and therefore cannot be held liable for resulting unintended harms.
Talk about incentives.
(This is often where some folks default to talking points about the need to promote innovation and protect innovators from liability—and that is a valid consideration, but let’s say the silent corollary out loud. Harms have been created. If AI producers are not liable for the harms they cause and there are no alternative means of redress, the full costs of those harms are borne by the public, individually and collectively.)
To answer the question of which standard should be employed, Choi employs a framework he previously developed for assessing when the professional standard should be applied to a particular industry. One might think that certain professions set their own standard of care because they have particular traits—required degrees, licensing professions, and codes of ethics. But Choi argues that this is historically inaccurate, as doctors were evaluated under the professional standard long before they were “professionals” as we think of them today. Rather, he suggests that the professional standard is applied in situations where judges have good reason to not trust jury instincts and sentiments, as things can go terribly wrong even when a defendant doctor or lawyer does everything right.
Choi’s professional standard framework requires evaluating three factors: (1) “whether the core elements of the work involve substantial uncertainties in knowledge, and therefore require latitude for discretionary judgment”; (2) “whether there are serious harms that are statistically unavoidable because of the lack of scientific precision or control”; and (3) whether “[the defendants] perform an essential societal service even when their customary practices cause harm.”
One thing I love about this piece is that Choi’s exploration of the first two factors provides an impressively detailed and nuanced yet succinct description of what AI development entails. In describing the core elements of the work and where judgement is exercised, Choi clarifies where the relevant design decisions occur, the benefits and risks associated with different choices, and common sources of bias and error.
After reviewing the AI development process in his evaluation of the first factor, Choi concludes that its status as more of an ‘art’ than a ‘science’ weighs in favor of judges applying a professional standard. He notes that, while “[m]uch of the work involved in training neural networks is either menial or guided by well-established mathematical principles,” there is still “an important component [that] involves subjective judgements that are guided by customary practices derived from trial-and-error.”
In his analysis of the second factor, Choi creates a useful 3×2 typology matrix of potential harms. Along one axis, he distinguishes accidental harms, intended harms, and foreseeable misuses; along the other, he distinguishes between harms resulting from inaccurate and incomplete data from harms resulting from the accurate perpetuation of historical bias in data sets. He determines that “harmful outcomes are an expected feature even of competent AI modeling work,” which in his view also weighs in favor of a professional standard.
Compared with the first two factors, Choi’s analysis of the third is slightly perfunctory: He quickly states that AI developers do not (yet) perform an essential societal service, which favors the reasonable care standard. He concludes that, given the nascency of the field, strict liability is the appropriate current standard; however, once “AI becomes an ordinary fixture of everyday society”—at which point AI modelers may be providing more of an essential service—courts will need to wrestle with what negligence standard should be applied.
Jack Balkin has observed that legal analysis has a fractal nature, as the resolution of one question raises a host of others. Deciding to evaluate AI liability under negligence raises the “which standard” question; Choi’s exploration of the “which standard” question raises further ones.
First, each of Choi’s factors include major wiggle words. When is an uncertainty sufficiently “substantial”? When are harms sufficiently “serious”? At what point does a tool become a “societal service”—and when does that service become “essential”? As any despairing 1L will tell you, wiggle words pervade the negligence analysis—what, after all, constitutes “reasonable” care?—but resolving the squishy questions is usually left to the jury and can vary dramatically based on the facts of a case. In contrast, the selection of a standard is the judge’s role, which more easily becomes precedential—and precedent, once set, can be difficult to shift. How should law approach the risk of inapt legal lock-in in this context?
Second, need this be an all-or-nothing determination? Just as AI companies might have certain acts evaluated under strict liability or negligence regimes, it is possible to apply a professional standard to some types of AI development and deployment choices and an ordinary care standard to others. Even doctors’ acts are sometimes evaluated under the reasonable person standard, such as when a pharmacist inadvertently misfills a prescription.
As a teacher, I would recommend the piece for the introduction-to-AI-development inherent in Choi’s analyses alone. It would make a great opener for any AI and the Law course. As a scholar, I’m grateful to Choi for his thought-provoking exploration of a critical second-order question in the AI liability conversation.
