Ann Bartow

Something Rotten in the State of Legal Citation trumpets an important alarm for the entire legal profession, warning us that given current modes of citing websites in judicial cases create a very real risk that opinion-supporting citations by courts as important as the United States Supreme Court will disappear, making them inaccessible to future scholars. The authors of this important and disquieting article, Raizel Liebler and June Liebert, both have librarianship backgrounds, and they effectively leverage their expertise to explicate four core premises: Legal citations are important; web based legal citations can and do disappear without notice or reason; disappearing legal citations are particularly problematic in judicial opinions; and finally, to this reader’s vast relief, there are solutions to this problem, if only the appropriate entities would care enough to implement them.

Denoting the disappearing citation phenomenon with the vivid appellation “link rot,” Liebler and Liebert explain that the crucial ability to check and verify citations is badly compromised by link rot, and then demonstrate this with frankly shocking empirical evidence. According to their research:

[T]he Supreme Court appears to have a vast problem with link rot, the condition of internet links no longer working. We found that number of websites that are no longer working cited to by Supreme Court opinions is alarmingly high, almost one-third (29%). Our research in Supreme Court cases also found that the rate of disappearance is not affected by the type of online document (pdf, html, etc) or the sources of links (government or non-government) in terms of what links are now dead. We cannot predict what links will rot, even within Supreme Court cases. (P.278).

They warn that without significant changes to current practices, the information in citations within judicial opinions will be known solely from those citations. When citations lack lengthy parentheticals or detailed explanatory text, it might not even be clear to future readers, critics or researchers why a document was cited, no less the nature of the support or clarifications it offered.

Liebler and Libert acknowledge that the Internet has improved legal research in many ways, opening up information conduits that had not been easily available before, and that in many respects website citations were an exciting development for the Supreme Court. They note that Justice Souter was the first Justice to cite the Internet in 1996, in a concurrence, and “then in 1998, Justice Ginsburg used the Internet for sources to demonstrate different meanings of the word “carry” in her dissent.” (P. 279). By 2006, all of the Justices then serving had cited at least one website. Internet based citations continued to blossom, and Liebler and Liebert’s research establishes that between 1996 and 2010, 114 majority opinions of the Supreme Court included links, but that almost one third of them are no longer working. Link rot at the Supreme Court is extant, widespread, and perfidious. Among several arresting examples they offer is the following:

In Scott v. Harris, a video with a dead link was cited extensively by both the majority and minority opinions, serving as the focal point of a serious disagreement in the case. The majority opinion states, “We are happy to allow the videotape to speak for itself.” Additionally, the majority used the citation to the video to disagree with the dissent, stating that “Justice Stevens suggests that our reaction to the videotape is somehow idiosyncratic, and seems to believe we are misrepresenting its contents.” (P. 282).

Even when information cited by the Court remains available on the Supreme Court website, it is often relocated; the old links are not amended to point to the new location, so they are as good as dead if that is what researchers quite reasonably assume them to be. Liebler and Liebert’s findings affirm research which has charted extensive link rot in many other contexts such as law review articles. Even more disturbingly, this research is in accord with “a study of federal appellate opinions [which] found that in 2002, 84.6% of Internet citations in cases from 1997 were inaccessible; moreover, 34% of citations in cases from 2001 were already inaccessible by 2002.” (P. 290-91).

Liebler and Liebert’s stunning revelations are a simple matter to confirm in the context of any subject area. For example, one of the most important copyright cases the Supreme Court has ever decided was Sony Corp. v. Universal Studios in 1984. The long and not particularly well written majority opinion set the balance between content owners like Universal Studios, and companies like Sony that produced new and innovative technologies (in this case the Betamax videocassette recorder) with respect to secondary copyright infringement. Under Sony, a new technology that was capable of substantial non-infringing uses could not be enjoined from distribution on the grounds that it contributed to copyright infringement. Sony was controversial and its convoluted drafting gave lawyers and judges the opportunity to read it into a multitude of meanings. As a copyright law geek of long standing, this author has seen that majority opinion in Sony parsed, diced, sliced by lower courts, and ultimately repackaged as a shadow of its former self by a unanimous Supreme Court in 2005 in MGM Studios v. Grokster. But at least link rot was not a worry. The same cannot be said of Grokster, wherein Justice Breyer’s concurrence has links and some of the links have already rotted. In fairness he notes that “all Internet materials … are available in Clerk of Court’s case file,” but it is not at all clear how easy it might be for a researcher to access this now, or especially five years from now. According to Liebler and Liebert, the case files are only available to those with sufficient means to go to Washington, DC, and visit the office of the Clerk of the Supreme Court. (P.300).

Another thing one learns from Something Rotten in the State of Legal Citation is that the Supreme Court often does its own web-based fact-finding. Liebler and Liebert inform readers that Allison Orr Larsen conducted a study of fifteen years of Supreme Court opinions, and “found that of the over one hundred “most important Supreme Court cases” from 2000 to 2010, 56% include mentions of facts the Justices did not find in the record and instead found independently.” (P. 278). Liebler and Liebert quote her stunning finds as follows:

[I]t was quite common for Justices to demonstrate the prevalence of a practice through statistics they found themselves. And, at a fairly high rate these statistics were supported by citations to websites—I found seventy-two such citations in my non-exhaustive search. Importantly, statistics ere independently gathered from websites with widely ranging indicia of reliability.¹

While it is sort of amusing to picture the Justices surreptitiously googling themselves when they get bored during oral arguments, it’s a little disconcerting to think of them relying even briefly on misinformation-ridden sites like Yahoo Answers. Yahoo has not cornered the market on dumb, because the Internet does not have corners, but Yahoo Answers is rather infamous for exchanges such as:²

Question: Is it wrong to hate a certain race?

Answer: No, because if you are only used to running a 5k, doing a 10k with your jogging group is going to take too long. I hate 10ks myself for this very reason.

Question: Why doesn’t the Earth fall down?

Answer: Because it can fly.

Question: I plan on starting a business selling dognuts, any advice?

Answer: If you want people to eat them, I would call them doughnuts.

Question: Does vodka kill bees and wasps?

Answer: Yes, over time it will destroy their tiny livers, but it is the disruption to the home life that really takes its toll.

One wonders about the quality of the information that the Justices are finding online, and this practice is even more dangerous if link rot means that citations to the Justices’ independent research cannot be assessed or verified. And if Supreme Court Justices are engaging in the dubious practice of doing their own online research about cases before them, one has to assume lower court judges are doing so as well.

Liebler and Lieber conclude their outstanding article by recommending possible solutions to the link rot problem. “Ideally,” they say, “every court should digitally archive all materials cited within an opinion, regardless of the format.” (P. 299) They observe that:

In 2009, the Judicial Conference of the United States created a report titled Internet Materials in Opinions: Citations and Hyperlinking that recommended two primary solutions to the broken Internet link problem: Clerks should download any cited Internet resources and include them with the opinions. The downloaded Internet resources should be included as attachments on a non-fee basis in each court’s Case Management/Electronic Case Files System, such as PACER. (P. 301).

PACER is not without its drawbacks, but there are other alternatives as well, including using the Internet Archive or other internet archiving organizations, or permanent URLs. The main takeaway from this valuable article is that something needs to be done about link rot, and the problem needs to be addressed quickly and expansively. Liebler and Liebert have done a great service to the entire legal profession by bringing link rot to our attention and mapping the gigantic contours of the problem so compellingly.

1. See Allison Orr Larsen, Confronting Supreme Court Fact Finding, 98 Va. L. Rev. 1255, 1288 (2012) (including discussion of the Justices’ use of websites to conduct research during oral argument and for opinions).
2. These are representative, edited versions of Yahoo Answers, screen grabs of which are on file with the author.