Feb 8, 2016 Herbert Lin
Kristen Eichensehr
, Cyber War & International Law Step Zero, 50 Tex. Int'l L.J. 355 (2015), available at
SSRN.
Kristen Eichensehr recently published a piece entitled Cyberwar & International Law Step Zero that describes an unfolding of events that is by now familiar to international lawyers contemplating the emergence of new military technologies. First, a new military technology X (where X has been drones, cyber weapons, nuclear weapons, lethal autonomous weapons) appears. Nations then ask the “step-zero” question — “does international law apply to the use or acquisition of X”? And the answer is inevitably, “yes, but in some ways existing international law needs to be tweaked to adjust for some of the novel characteristics of X.”
Eichensehr offers a compelling explanation for both the persistence of this question and the recurrent answer. Regarding persistence, she points out that for international law, unlike domestic law, the bound parties—nations—bind themselves consensually. For example, she writes that “The tradition of requiring state consent (or at least non-objection) to international law predisposes the international legal community to approach new issues from the ground up: When a new issue arises, the question is whether international law addresses the issue, because if there is no evidence that it does, then it does not.” In other words, asking the step-zero question is the first step in proceeding down a path that may result in a state’s opting out.
Regarding the frequent recurrence of the same answer (i.e., “yes”), she points out that international law—especially International Humanitarian Law (“IHL”)—is often adaptable to new weapons technologies, in large part because the interests that IHL seeks to protect are constant. (I would prefer the term “values” rather than “interests,” but the point is the same.) For example, she writes that “[e]xisting law was designed, for example, to protect civilians from the consequences of conflict. That concern transcends the type of weapon deployed. Thus, although the nature of the weapon has changed, the underlying concern has not, which reduces one possible justification for altering existing law.” Lastly, she argues that even if existing law does not perfectly apply to new technologies, asserting the contrary raises the fearsome prospect of a world in which a new technology is not subject to any legal constraint at all. In her words, “[e]ven if existing law is an imperfect means of regulating States’ actions . . ., imperfect law is preferable to no law at all.”
The explanation seems compelling to me, though I confess from the start that my understanding of law is that of an amateur. But I’m also a long-time observer of many military technologies. I’ve thought often about how international law attends to these technologies, and I suggest that her explanation is applicable to a broader range of phenomena than she discusses.
Speaking in very broad terms, law—and especially international law—depends heavily on precedent. Precedent provides stability, which is regarded as a desirable attribute of law in large part because in the absence of legal stability, people—and nations—would have no way of knowing how law would regard their actions. But technologists have very different goals. Rather than stability, the dream of every technologist is to invent a disruptive technology, one that completely changes the way people can accomplish familiar goals. Even better is when a technologist can create not just new ways of doing old business, but can invent entirely new lines of business.
Against this backdrop, consider a broadened step-zero sequence of events. A new technology A is invented. At first, when the use of A is small and limited, the law pays little or no attention to it. But as A becomes used by more and more and more people, a variety of unanticipated consequences appear, some of which are regarded as undesirable by some people. These people look to the law for remedy, and they naturally ask the question “how, if at all, does existing law apply?” Their lawyers look for precedent—similar cases handled in the past that may provide some guide for today—and there is always a previous case involving technology that bears some similarity to A today. So the answer is, “yes, existing law applies, but tweaks are necessary to apply precedent properly.”
So, I suggest, Eichensehr’s step-zero analysis of cyber weapons and international law sheds light on a very long standing tension between technological change and legal precedent. For that reason, I think anyone interested in that tension should consider her analysis.
Jan 7, 2016 Michael Madison
The Atomic Age of Data: Policies for the Internet of Things Report of the 29th Annual Aspen Institute Conference on Communications Policy, Ellen P. Goodman, Rapporteur, available at
SSRN.
The phrase “Internet of Things,” like its cousin “Big Data,” only partially captures the phenomenon that it is meant to describe. The Atomic Age of Data, a lengthy report prepared by Ellen Goodman (Rutgers Law) following a recent Aspen Institute conference, bridges the gap at the outset: “The new IoT [Internet of Things] – small sensors + big data + actuators – looks like it’s the real thing. … The IoT is the emergence of a network connecting things, all with unique identifiers, all generating data, with many subject to remote control. It is a network with huge ambitions, to connect all things.” (P. 2) The Atomic Age of Data is not a scholarly piece in a traditional sense, but it is the work of a scholar, corralling and shaping a critical public discussion in an exceptionally clear and thoughtful way.
The IoT is in urgent need of being corralled, at least conceptually and preliminarily, so that a proper set of relevant public policy questions may be asked. What are the relevant opportunities and hazards? What are its costs and benefits, to the extent that those can be discerned at this point, and where should we be looking in the future? That set of questions is the gift of this report, which is the documented product of many expert and thoughtful minds collaborating in a single place (face to face, rather than via electronic networks).
Simply defining the IoT is one continuing challenge. As The Atomic Age of Data affirms, the IoT isn’t the Internet, though it is enabled by the Internet and in many ways it extends the Internet. (P. 2) What it is, where it is, how it functions, what it might do in the future – or permit other to do – remains at least a little cloudy. The first contribution that The Atomic Age of Data makes is simply to map these contours, contrasting the Internet of Things with the network of networks that today we call the Internet, or the Internet of People. It identifies several distinguishing characteristics of the IoT: its sheer scale (the amount of data that can be gathered from ubiquitous sensor networks); the reduction or even elimination of user control over data collection; the widespread deployment of actuators, embedding a level of agency in the IoT; data analytics that rest atop communications and transactions; its demonstrably global character (in contrast to the initiated-in-the-US character of the Internet); and its framing of data as infrastructure, enabling the provision of a broad variety of services.
The bulk of The Atomic Age of Data consists of a comprehensive sorting of policy questions and recommendations. The foundational premise is the idea that data itself is (or are) infrastructure – “as a vital input to progress much like water and roads, and just as vulnerable to capture, malicious or discriminatory use, scarcity, monopoly and sub-optimal investment”. (P. 12) The analogy between data infrastructure and communications infrastructure is purposeful. Characterizing data as infrastructure, like characterizing communications as infrastructure, only frames policy and technical questions; it doesn’t resolve them. Data ownership and data access are related questions. They connect to questions of data formats, interoperability and interconnectivity, and common technical standards. Identifiability of data is a cross-cutting concern for privacy purposes. The respective domains of public and private investment in the IoT, and corresponding expectations of public access and use and private returns, remains open questions. The report clusters these topics together; one might label the cluster with a single theme: governance.
How, or more precisely, by whom, will all of this data be produced? The report examines the adequacy of incentives for private (commercial) provision of data and the appropriate role for government as regulator and supplier of subsidies.
This “data as infrastructure” section of The Atomic Age of Data concludes with a series of policy recommendations, focusing on two overarching principles (also reduced to several more specific recommendations): that there should be broad accessibility of data and data analytics, with open access to some (but not all); and that government should subsidize and facilitate data production, particularly in cases where data is an otherwise under-produced public good.
The Atomic Age of Data moves next to a review of privacy topics in the context of the IoT, beginning with when, whether, and how to design privacy protections into systems from the start, and the role and implementation of Fair Information Practice Principles (FIPPs). As the report notes, these are critical questions for the IoT because so much of the IoT is invisible to individuals and has no user interface to which data protection and FIPPs might be applied. To what extent should privacy protection be designed in to the IoT, and to what extent should privacy protection be a matter of strategies that focus on individual choice?To what extent might choice undermine the production, collection, and processing (aggregation) of good data, or the right data? Privacy questions thus intersect with incentive questions. Cost, benefit, and values questions extend further. To what extent is choice even technologically feasible without compromising other societal values? Production, collection, identification, and processing/aggregating data lead next to related privacy questions about retention and curation of data.
This privacy section concludes with brief set of recommendations, focusing on three overarching principles (again with several more specific points): that IoT systems should design in privacy controls to minimize the collection of personally identifiable information; that IoT systems should effectuate FIPPs to the extent possible; and that individuals should have a way to control collection and transmission of their personal data.
The balance of the report is divided among four additional topics that are treated more briefly, though in each case the topic concludes with a short set of basic recommendations. The first is “Equity, Inclusion, and Opportunity,” which collects questions about prospects of citizen empowerment and disempowerment via the IoT. Data collection in some respects signifies “who counts” in modern society – whose voice and presence “matters,” both individually and collectively, but also, in some respects, whose voice and presence is worth watching. The report points out the relevance of comparable concerns with respect to the deployment of broadband communications infrastructure and its impacts on things like access to education and health resources. The second is “Civic Engagement,” which touches on how IoT technologies might be used both by governments and by the private sector to increase democratic accountability. The third is “Telecommunications Network Architecture,” which concerns the intersection of the IoT and competition, service innovation, and interoperability among IoT systems and related communications networks. The key topic here is the heterogeneity of the data generated by IoT applications, recalling the question of whether the Internet of Things is, or should be, truly a single “Internet” at all, with interconnected networks, connections across verticals (home, health, transport, for example), and common platforms. (P. 39) The fourth is security, which raises the relatively simple question of security vulnerabilities introduced at both the level of individual devices and at systemic levels. The question may be simple but the answer assuredly is not; this section of the report is comparatively brief, perhaps because the salience of the interest is so obvious.
The Atomic Age of Data finishes with a case study, on The Smart City, which refers to the idea of networks of ubiquitous sensors deployed within urban infrastructure to generate data about usage patterns and service needs. (P. 45) The discussion of this use case is decidedly and appropriately pragmatic, putting utopian hopes for the Smart City in context and noting privacy and surveillance concerns and related but independent equity concerns.
To conclude this review:
This is an enormously clear, useful, and timely product. One cannot critique a report of a conference on the ground that it did not address a critical topic, if the conference itself did not address that topic. Yet as helpful as The Atomic Age of Data is in canvassing the policy territory of the IoT, I couldn’t help but notice how the boundaries of that territory are implicitly defined. The Atomic Age of Data contains a lot of discussion of “Internet” topics and less discussion of “things.” In this day and age, one should never take things or thing-ness for granted. What is a thing? 3D printing, the current label for additive manufacturing, promises to revolutionize the meaning of “thingness” – because objects may be dynamic and emergent, as well as static and fixed – just as the “Internet of Things” promises to revolutionize the meanings of identity and presence.
“Data for Peace,” the title of this review, builds a bit on the naïve sense of modernity and progress expressed (purposefully, no doubt) by the report’s Atomic Age title. During the 1950s and 1960s, “atomic” things were full of optimism. Later, we learned that splitting the atom changed the meanings of matter in unexpected ways. “Atomic” gave way to a variety of more complex political, cultural, and technological expressions and concerns, few of which were foreseen at the dawn of the Atomic Age. Similarly, 3D printing may turn out to change the meanings of matter in unexpected – but other – ways. As the IoT and Big Data mature — along with 3D printing – I expect that future reports on its implications will be similarly but unexpectedly complex.
Nov 30, 2015 Tal Zarsky
Ira Rubinstein & Woodrow Hartzog,
Anonymization and Risk, 91
Wash. L. Rev. (forthcoming 2016), available on
SSRN.
In the current Age of Big Data, companies are constantly striving to figure out how to better use data at their disposal. And it seems that the only thing better than big data is more data. However, the data used is often personal in nature and thus linked to specific individuals and their personal details, traits, or preferences. In such cases, sharing and use of the data conflict with privacy laws and interests. A popular remedy applied to sidestep privacy-based concerns is to render the data no longer “private” by anonymizing it. Anonymization is achieved through a variety of statistical measures. Anonymized data, so it seems, can be sold, shared with researchers, or even possibly released to the general public.
Yet, the Age of Big Data has turned anonymization into a difficult task, as the risk of re-identification seems to be constantly looming. Re-identification is achieved by “attacking” the anonymous dataset, aided by the existence of vast datasets (or “auxiliary information”) from various other sources available to the potential attacker. It is, therefore, difficult to establish whether anonymization was achieved, whether privacy laws pertain to the dataset at hand, and if so, how. In a recent paper, Ira Rubinstein and Woodrow Hartzog examine this issue’s pressing policy and legal aspects. The paper does an excellent job in summarizing the way that the current academic debate in this field is unfolding. It describes recent failed and successful re-identification attempts and provides the reader with a crash course on the complicated statistical methods of de-identification and re-identification. Beyond that, it provides both theoretical insights and a clear roadmap for confronting challenges to properly releasing data.
The discussion on anonymization, or de-identification (the more precise term which the authors choose to apply, as it does not imply full anonymization) was once mostly of academic interest: Statisticians introduced ways to anonymize data, while mathematicians and computer scientists strove to prove re-identification “attacks” were nonetheless possible. Several successful re-identification attacks (perhaps the most famous one involved Neflix and IMDb) also led legal scholars to debate proper policy practices, as well broader implications of re-identification. However, this academic discussion is quickly crossing over into the world of practitioners. Recent policy papers published by regulators in the U.S., U.K., and the E.U. strive to create legal and normative guidelines for the manner in which personal information can be shared and released. In addition, corporations are turning to legal counsel for advice on using anonymization to mitigate potential liability.
In an age in which legal scholarship seems to be drifting away from legal practice, this paper demonstrates how both can be brought together. To a great extent, the knowledge conveyed in this paper is now essential for all legal practitioners advising clients with large databases. To demonstrate the relevance of this discussion, note a recent debate regarding the practices of Yodlee, an online financial tools provider, which has also emerged as a powerful financial-data aggregator. As recently reported by the Wall Street Journal, Yodlee sells information, gathered by facilitating consumer transactions, to investors and research firms. The WSJ claimed that Yodlee clients’ privacy is being compromised, and Yodlee responded by arguing that all personal information was properly handled and de-identified. It is safe to assume that similar stories involving other companies’ collecting, marketing, or de-identifying personal data are just around the corner.
Perhaps the central point that Rubinstein and Hartzog’s paper strives to articulate is that classifying personal data as either anonymous or identifiable is both incorrect and useless. With regard to anonymization, the authors further note that: “[a]lmost all uses of the term to describe the safety of data sets are misleading, and often they are deceptive. Focusing on the language of process and risk will better set expectations” (P.4). In other words, anonymity (or rather – de-identification) is not an absolute term, but one indicating degrees on a scale – one that should be measured by the effort required to reveal the personal data, and the chance it could occur. As the authors note, this latter notion was already introduced (perhaps most famously by Paul Ohm). Rubinstein and Hartzog’s important contribution is to break this notion down into practical steps – formulating a proper data release policy as well as providing a full toolbox of measures to be applied in the process.
Beyond this important observation, the paper’s most substantial analytical contribution is to link appropriate data release policies with the notion of data security. The relationship, as explained by the authors, is based on these concepts mutual need to meet a specific standard of care in the process, and not necessarily be judged by the outcome. The authors also explain that context matters, and list various parameters and attributes of the data release process that should be considered when formulating a release policy (p. 32). In addition, they demonstrate that an integral part of a release policy is the technical measures applied when distributing and sharing the information. In doing so, they note that the Release-and-Forget Model of data sharing (in which, for example, a de-identified database is merely made available over the internet) is most likely obsolete (p. 36); all data release schemes must include unique measures (technological, contractual – or both) which strive to limit re-identification by potential attackers.
Beyond the rich policy discussion the authors provide in comparing and equating security policy to data release policy, several additional theoretical questions (with practical implications) come to mind and are worthy of future discussion: Is a regulatory response similarly necessary in the security and data release contexts? While companies usually under-invest in security (given, among other factors, the negative externalities of security breaches), there have been examples of instances in which corporate motivation to enhance security was close to sufficient, especially in view of market pressures and the reputational costs of breaches. In many cases, companies’ and clients’ interests in maintaining security are aligned. More often, though, corporations’ and clients’ interests regarding data releases directly conflict. Corporations are interested in capitalizing on their data, whereas consumers do not necessarily share corporate enthusiasm for sharing their de-identified personal information, as they are not likely to benefit from or be compensated for this additional revenue stream. For this and other reasons, the security-release policy comparison has its limits; data release policies might call for stricter rules and enforcement mechanism.
In addition, it would be interesting to consider the role insurance could play in the process of data release—an issue also currently emerging in the context of data security. An active insurance market might indeed facilitate the shift from outcome- to process-based liability without the need to change the regulatory framework. Therefore, the change the authors here advocate for might be just around the corner. Insurers could, for instance, limit indemnification to those companies that follow acceptable data-release policies (yet nonetheless cause harms to third parties). Yet, relying on insurance markets may not be a safe bet. In this specific context, insurance markets face several difficulties, which mandate further discussion. The comparison to data security can prove illuminating here as well.
Oct 29, 2015 Daithí Mac Síthigh
Marco Loos & Joasia Luzak,
Wanted: A Bigger Stick. On Unfair Terms in Consumer Contracts with Online Service Providers (Ctr. for the Study of European Contract Law, Working Paper No. 2015-01, 2015), available at
SSRN.
The reliance of online service providers on lengthy terms of service or related documents is easily mocked. When I teach this topic, I can choose to illustrate the topic with the selling of souls, in cartoon or written form, point to the absurd length of the policies of popular sites, and highlight experiments that call us out on our love of the I Accept button. But behind the mirth lie a number of serious legal issues, and the recent working paper by Marco Loos & Joasia Luzak of the University of Amsterdam tackles some of them.
Loos & Luzak work at the Centre for the Study of European Contract Law, and their particular concern is with the European Union’s 1993 Unfair Contract Terms Directive. They point out that although the gap between typical terms and policies and the requirements of the Directive is often pointed to, it is rarely studied in detail. In their thorough study, the authors examined the instruments used by five well-known service providers, and evaluated them against the Directive’s stipulation that mass terms (those not individually negotiated with the consumer) be ‘fair’.
The detailed paper, full of examples from the policies of the services under review (Dropbox, Facebook, Twitter and Google), covers topics including modification and termination of the agreement, as well as how liability is managed. Despite the focus of the work being the UCT Directive, the analysis is also linked with developments in related fields of law, such as the gradual expansion through Court of Justice of the EU (CJEU) decisions of the ‘consumer’ provisions of the Brussels Regulation on jurisdiction. The authors save particular criticism for the lack of clarity in how terms are drafted.
Importantly, the paper also tackles the preliminary question of whether the statements we know and love actually fall within the scope of the Directive, which is about contracts and about consumers. They challenge the assumption that ‘free’ services are excluded, but do note that in some cases more detail on the actual use of an account may be necessary in order to be certain that the Directive is applicable.
What Loos & Luzak have done here also contributes to debates on consent, rights and technology. In data protection and in consumer law, much depends on assumptions about information – what must be provided, how it informs decisions, and what legal options are available to the end user. One cannot doubt the skill that goes into drafting some of the examples that are cited in this paper, but the authors are right to call for greater study and vigilance – particularly on the part of national consumer authorities. They hope that if the CJEU is faced with appropriate questions in future years, the result might be a gradual raising of consumer protection standards. Indeed, this might well have implications across the world – as Jack Goldsmith and Tim Wu discussed regarding earlier data protection disputes in their 2006 book, Who Controls The Internet? – and of course other agencies, such as the FTC and the Australv.ian Privacy Commissioner, are interested in these issues. So, this recent work on common clauses and legal requirements for fairness should interest European and non-European audiences alike.
Cite as: Daithí Mac Síthigh,
Is it Fair to Sell Your Soul?, JOTWELL
(October 29, 2015) (reviewing Marco Loos & Joasia Luzak,
Wanted: A Bigger Stick. On Unfair Terms in Consumer Contracts with Online Service Providers (Ctr. for the Study of European Contract Law, Working Paper No. 2015-01, 2015), available at SSRN),
https://cyber.jotwell.com/is-it-fair-to-sell-your-soul/.
Sep 29, 2015 Margot Kaminski
When the law faces a new technology, a basic question is who governs it and with what rules? Technological development disrupts regulatory schemes. Take, for example, the challenges the Federal Aviation Administration (FAA) now faces with drones. The FAA usually regulates aircraft safety. Drones force the FAA to consider—and in some cases reject as outside the agency’s mandate—issues of privacy, spectrum policy, data security, autonomous decision-making, and more. The pace and complexity of recent technological change has led some to call for the creation of new agencies, including a Federal Robotics Commission. But given the significant hurdles involved in agency creation, it is valuable in the short run to assess what tools we already have.
In Unfair and Deceptive Robots, Woodrow Hartzog takes up the question of who will govern consumer robots. Hartzog proposes that the Federal Trade Commission (FTC) is best equipped to govern most issues that consumer robots will soon raise. He reasons that the FTC is well prepared both as a matter of subject-matter expertise and as a matter of institutional practice.
This article was a hit at the 2015 We Robot conference. It blends practical guidance, expert knowledge of the FTC, and a range of thoughtful and often amusing examples. It also provides a window onto a number of framing questions recurring in the field: to what extent are robots new? How does that answer vary, depending on what aspect of robots you focus on? And how do you best choose or design institutions to adapt to fast-changing technology?
Hartzog points out a number of ways in which robots, or really robotics companies, might take advantage of vulnerable consumers. A company might falsely represent a robot’s capabilities, touting effectiveness in sped-up videos that make a robot look more capable than it is. Or a company might use a “Wizard-of-Oz” setup to operate a robot from behind the scenes, causing it to appear autonomous when it is not. A company might use a robot to spy on people, or to nudge their behavior. Autonomous robots and robotic implantables raise their own classes of consumer protection concerns. If you were not already worried about robots, you will be after reading this. From the robot vacuum that ate its owner’s hair, to flirtatious Twitter bots, to a dying Roomba pleading for a software upgrade, to the “Internet of Things Inside Our Body,” Hartzog’s examples are visceral and compelling.
The FTC, Hartog claims, is thankfully well positioned to address many of the consumer protection issues raised by this pending host of scambots, decepticons, autobots, and cyborgs. The FTC has a broad grant of authority to regulate “unfair and deceptive” trade practices. It has used that Section 5 authority in recent years to regulate online privacy and data security. While the FTC started by addressing classic truth-in-advertising problems, and enforcing company promises, it has developed more complex theories of unfairness that it now extends to data security and user interface design. His recent authoritative work with Dan Solove on the FTC’s Section 5 “jurisprudence” makes Hartzog uniquely qualified to discuss FTC coverage of robotics. There is no doubt that this paper will have practical applicability.
Hartzog also contributes to ongoing conversations about technological change and regulatory design. He touts the FTC’s institutional ability to adapt to changes through productive co-regulation, including its tendency to defer to industry standards and avoid “drastic regulatory lurches.” Hartzog thus identifies not just substantive but structural reasons why the FTC is a good fit for governing consumer robots.
But the view Hartzog presents is a little too rosy. The FTC has vocal and litigious critics whom Hartzog mainly ignores. Not everyone is happy with its settlement agreement process, which some regard as arbitrary and lacking notice. While Hartzog mentions in passing that the FTC’s authority to regulate data security has been challenged, the pending Wyndham decision in the Third Circuit could seriously rock the Section 5 boat. Moreover, the FTC’s focus on notice and design is in tension with developing First Amendment jurisprudence on commercial and compelled speech. And there are plenty of other good reasons why we might want to be careful about focusing governance on technological design as Hartzog proposes.
If I have one larger criticism, it is that the “which agency is best” framing is a little disingenuous. Hartzog frames his question in a way that drives his answer. He asks which agency is best positioned for governing consumer protection issues raised by robots; unsurprisingly, his answer is the FTC, a consumer protection agency. If he had asked which regime is best for governing robotic torts, or which is best for governing robotic IP issues, the answer would have differed. In other words, the article provides solid guidance for how the FTC might approach robots. It does not answer, or really justify asking, the question of who governs them best.
Which brings us to the larger conversation this piece briefly engages in, on just how new and disruptive robots will be. I am increasingly convinced that the answer to this question is dependent on the asker’s perspective. Asking how robots disrupt a particular area of law will highlight the features of the technology and its social uses that are disruptive to that particular area of law. A new technology will be disruptive to different regulatory regimes in different ways. And because Hartzog picks the FTC as his lens, he is bound to solutions the FTC provides, and somewhat blinded to the problems it can not solve. Robots fit within the FTC’s consumer protection regime, but they also fundamentally disrupt it. As with the Internet of Things, the owner of the robot is often not the only person facing harm. The FTC protects the consumer, not the visitor to a consumer’s house. As Meg Jones has recently pointed out, the FTC is not particularly well equipped to handle problems raised by this “Internet of Other People’s Things.”
Unfair and Deceptive Robots is clever and extremely useful: it tells us what the FTC is equipped to handle, and argues for the FTC’s competence in this area. As a robot’s road map to FTC jurisprudence, the piece shines. But regulating robots will take many regulatory players. While we are trying to spot the gaps and encourage them to cooperate, it might be counterproductive to name one as the “best.”
Aug 14, 2015 Rebecca Tushnet
Kate Crawford & Tarleton Gillespie,
What is a flag for? Social media reporting tools and the vocabulary of complaint,
New Media & Society (2014), available at
SSRN.
The problem of handling harassing and discriminatory online speech, as well as other forms of unpleasant and unlawful content—infringing, privacy-invading, or otherwise tortious—has been a matter for public discussion pretty much since people noticed that there were non-governmental intermediaries involved in the process. From revenge porn to videos of terrorist executions to men kissing each other to women’s pubic hair, controversies routinely erupt over whether intermediaries are suppressing too much speech, or not enough.
“Flagging” offensive content is now an option offered to users across many popular online platforms, from Facebook to Tumblr to Pinterest to FanFiction.net. Flagging allows sites to outsource the job of policing offensive content (however defined) to unpaid—indeed, monetized—users, as well as to offer a rhetoric to answer charges of censorship against those sites: the fact that content was reported makes the flagging user/s responsible for a deletion, not the platform that created the flagging mechanism. But the meaning of flags, Crawford and Gillespie persuasively argue, is “anything but straightforward.” Users can use flags strategically, as can other actors in the system who claim to be following community standards.
One of the most significant, but least visible, features of a flagging system is its bluntness. A flag is binary: users can only report one level of “badness” of what they flag, even if they are allowed several different subcategories to identify their reasons for flagging. Nor are users part of the process that results, which is generally opaque. (As they note, Facebook has the most clarity on its process, likely not because of its commitment to user democracy but because it has faced such negative PR over its policies in the past.)
Another, related feature is flagging’s imperviousness to precedent—the memory-traces that let communities engage in ongoing debates about norms, boundaries, and difficult marginal judgments. Crawford and Gillespie explain:
[F]lags speak only in a narrow vocabulary of complaint. A flag, at its most basic, indicates an objection. User opinions about the content are reduced to a set of imprecise proxies: flags, likes or dislikes, and views. Regardless of the proliferating submenus of vocabulary, there remains little room for expressing the degree of concern, or situating the complaint, or taking issue with the rules. There is not, for example, a flag to indicate that something is troubling, but nonetheless worth preserving. The vocabulary of complaint does not extend to protecting forms of speech that may be threatening, but are deemed necessary from a civic perspective. Neither do complaints account for the many complex reasons why people might choose to flag content, but for reasons other than simply being offended. Flags do not allow a community to discuss that concern, nor is there any trace left for future debates. (P. 7.)
We often speak of the internet as a boon for communities, but it is so only in certain ways, and website owners can structure their sites so that certain kinds of communities have a harder time forming or discussing particular issues. Relatedly, YouTube’s Content ID, now a major source of licensing revenue for music companies, allows those companies to take down videos to which they object regardless of the user’s counternotifications and fair use claims, because Google’s agreements with the music companies go beyond the requirements of the DMCA. No reasoned argument need be made, as it would be in a court of law, and so neither the decisionmakers nor the users subject to YouTube’s regime get to think through the limiting principles—if any—applied by the algorithms and/or their human overlords. I have similar concerns with Amazon’s Kindle Worlds (and the Kindle’s ability to erase or alter works that Amazon deems erroneously distributed, leaving no further trace) compared to the organic, messy world of noncommercial fan fiction.
This is a rich paper with much to say about the ways that, for example, Flickr’s default reporting of images as “inappropriately classified” rather than completely unacceptable structures users’ relation to the site and to each other. “Whether a user shoehorns their complex feelings into the provided categories in a pull-down menu in order to be heard, or a group decides to coordinate their ‘complaints’ to game the system for political ends, users are learning to render themselves and their values legible within the vocabulary of flags.” Crawford and Gillespie’s useful discussion also offers insights into other forms of online governance, such as the debates over Twitter’s reporting system and the merits of “blocking” users. A “blocking” feature, available for example on Tumblr and Twitter, enables a logged-in user to avoid seeing posts from any blocked user; the offensive user disappears from the site, but only from the blocker’s perspective. Like denizens of China Miéville’s Besźel and Ul Qoma, they occupy the same “space” but do not see each other. This literalization of “just ignore the trolls” has its merits, but it also allows the sites to disclaim responsibility for removing content that remains visible to, and findable by, third parties. We may be able to remake our view of the world to screen out unpleasantness, but the unpleasantness persists—and replace “unpleasantness” with “slander and threats” and this solution seems more like offering victims blinders rather than protecting them.
What about total openness instead? As Crawford and Gillespie point out, Wikipedia generally retains a full history of edits and removals, but that process can also become exclusionary and opaque in other ways. Nonetheless, they suggest that an “open backstage” might offer a good way forward, in that it could “legitimize and strengthen a site’s decision to remove content. Significantly, it would offer a space for people to articulate their concerns, which works against both algorithmic and human gaming of the system to have content removed.” Moreover, an “open backstage” would emphasize the ways in which platforms are social systems where users can and should play a role in shaping norms.
I’m not as sanguine about this prospect. As Erving Goffman explained so well, even “backstage” is in fact a performance space when other people are watching, so I would expect new and different forms of manipulation (as has happened on Wikipedia) rather than a solution to opacity. Proceduralization and the ability to keep arguing endlessly can be a recipe for creating indifference by all but a tiny, unrepresentative fraction of users, which arguably is what happened with Wikipedia. It’s a new version of the old dilemma: If people were angels, no flags would be necessary. If angels were to govern people, neither external nor internal controls on flags would be necessary.
As someone who’s been deeply involved in writing and subsequently revising and enforcing the terms of service of a website used by hundreds of thousands of people, I know all too well the impossibility of writing out in advance every way in which a system might be abused by people acting in good faith, or even just (mis)used by people who simply don’t share its creators’ assumptions. Open discussion of core discursive principles can be valuable for communities; but freewheeling discussion, especially of individual cases, can also be destructive. And, as Dan Kahan has so well explained, our different worldviews often mean that a retreat from one field (from ideology to facts, or from substance to procedure, or vice versa) brings all the old battles to the new ground.
Still, there’s much to like about the authors’ call for a system that leaves some traces of debates over content and the associated worldviews, instead of a flagging and deletion system that “obscures or eradicates any evidence that the conflict ever existed.” Battles may leave scars, but that doesn’t mean that the better solution is a memory hole.
Jul 14, 2015 Herbert Burkert
The overall issue addressed in this book has received renewed attention recently. On April 1, 2015 President Obama issued the Executive Order “Blocking the Property of Certain Persons Engaging in Significant Malicious Cyber-Enabled Activities,” which allows the Treasury Department to freeze assets of individuals and entities that are directly or indirectly involved in such activities. Furthermore in the beginning of April, in a series of meetings in China, US Homeland Security officials met with their Chinese counterparts to discuss cybersecurity issues. And in late April the US Department of Defense issued its latest document on cyber strategy that mentions – among other countries – China among the “key cyber threats.”
However, the chosen article focuses on an issue that is easily is forgotten in these grand debates: citizens’ privacy, since threats to privacy come from the inside as well as from the outside. The author is Professor of Communication at the School of Digital Media and Design Arts, Beijing’s renowned University of Posts and Telecommunications (BUPT). He starts with an overview on the present legal framework for protecting the Right to Internet Privacy in China. (P. 247) I still vividly remember a presentation I gave in October 1996 at the China Youth College for Political Science (now the China Youth University for Political Sciences) in Beijing on “The Function of Law in an Information Society” addressing privacy issues. At the end of my talk one of the Chinese students stood up and boldly asked me what my talk had to do with current situation in China.
But I digress. The situation has changed profoundly: Professor Xu’s overview is condensed, yet sufficiently detailed to gain an insight into the development of concepts of privacy in China from an understanding of privacy as “shameful or embarrassing private [family] affairs” to privacy as a more comprehensive, however, still defensive notion and how it is moving from there to a broader understanding of affected “personal information.”
The current “Deepening Reform Campaign” in China has been emphasizing the Rule of Law. The Chinese concept of law is primarily an instrumental one. Rule of Law in this context means to ensure that the judiciary subsystem works efficiently, free from cross-interference—for example with regard to corruption cases—with optimal resources as regards the educational standard of its personnel, and meets its aim of ensuring fairness across local and provincial levels. All these principles have been reconfirmed this last month by a set of specific regulations from the General Office of the Communist Party’s Central Committee and the General Office of the State Council. At the same time the judiciary should be seen to be embedded in the guiding authority of these two law-making systems: the government as the administrative body and the checking political power of the Chinese Communist Party.
In Xu’s view the current system of legal privacy protection still needs to be fundamentally improved. There is no stringent overall legal concept of privacy. “Hundreds of laws and regulations have been enacted to protect the right to online privacy, but they are quite unsystematic and hard to put into practice.” (P. 252) (Sounds familiar). Responsibilities and liabilities in civil law should be established clearly and criminal law violations need to be more precise. He points to Hong Kong experiences as a learning resource for the further development of Chinese privacy protection, just as this note seeks to point to the necessity to enlarge our view on privacy beyond our European and American concerns.
Xu thus provides a useful insight into the ongoing development of the concept of privacy in the Chinese environment. As with such developments in the US and Europe they need to be put into the context of the respective legal system.
Cite as: Herbert Burkert,
Internet Privacy: A Chinese View, JOTWELL
(July 14, 2015) (reviewing Jinghong Xu,
Evolving Legal Frameworks for Protecting the Right to Internet Privacy in China, in China and Cybersecurity : Espionage, Strategy, and Politics in the Digital Domain, 242 (edited by Jon R. Lindsay, Tai Ming Cheung, and Derek S. Reveron, 2015)),
https://cyber.jotwell.com/internet-privacy-a-chinese-view/.
Jun 12, 2015 Paul Ohm
Thank you to the Jotwell editors for indulging me as I stretch their mission statement (and quite possibly their patience) by highlighting not an article nor even a conventional work of scholarship but rather a piece of software as the “thing I like (lots)”: mitmproxy, a tool created by Aldo Cortesi who shares authorship credit with Maximilian Hils and a larger “mitmproxy community.”
mitmproxy does just what it says on the tin (assuming you know how to read this particular kind of tin). It’s a Man-In-The-Middle Proxy server for the web. In English, this means that this tool allows you to reveal, with finely wrought control, exactly what your browser is saying and to whom. It is an X-ray machine for the web, one which lays many of the Internet’s secrets bare. Let me extol the many virtues of this well-designed piece of software, and after I do that, let me explain why I think this strikes me as an important contribution to legal scholarship.
There are many other tools that do what mitmproxy does. Where mitmproxy shines relative to everything I have tried is the way it embraces both usability and power without compromising either.
Take usability first. Especially for Mac OS X users, mitmproxy is the single easiest tool of its kind I have encountered. Here is what you need to do to begin wiretapping yourself as you browse the web:
- Step 1: Install the OSX binary available at https://mitmproxy.org/
- Step 2: Open a terminal window and extract, find, and start the binary.
- Step 3: Open a browser and configure it to use the IP address and port of the computer running mitmproxy (probably 127.0.0.1 and 8080) as its web proxy.
- Step 4: Surf the web.
At this point, the mitmproxy display will fill with lots of http requests down the screen. The controls to navigate these requests are so intuitive they require little documentation: arrows scroll up and down, enter reveals more detail about the current request, escape returns to the previous screen, etc.
By performing the steps above, the student or scholar of technology law or policy who has never operated a packet sniffer above can more deeply understand some of the secrets of web surveillance. mitmproxy is, most importantly, packet sniffing for the masses. For the first time, we are given a tool which is simple to understand, relatively easy to operate, free to download, and available to people lacking root access to their computers. These qualities make this a powerfully democratizing tool.
All of this makes mitmproxy also a wonderful tool for teaching. For three years, I have taught a course on “The Technology of Privacy,” in which the students have spent an hour or two sniffing packets. Until this year, my students toiled with Wireshark–an old tool, but still the industry standard for packet sniffing. To say that Wireshark confused my students is an understatement. The semester-end reviews were replete with comments like, “Great class, but I have no idea what was going on with Wireshark.”
This year, I taught the same unit using mitmproxy. The experience could not have been more different. After walking through the steps above and watching a demo for two minutes, my students started monitoring their own web traffic, needing no further guidance. My only instruction was “find something interesting,” and within five minutes, that’s exactly what they did.
Perhaps the most astonishing thing the tool makes easy is the sniffing of encrypted web traffic. Techies might scoff at my being impressed by this, because it’s almost tautological; that’s what a MITM proxy permits. But look again at how simply this has been implemented. Here are the steps required to permit the monitoring of encrypted traffic:
- Step 5: From your browser, visit mitm.it. (This won’t send you to an Italian webpage; mitmproxy intercepts the request and sends you its own content instead.)
- Step 6: Follow the simple instructions at that page.
- Step 7: Surf the encrypted web.
If all mitmproxy did was bring packet sniffing to the masses, it would still do plenty. But mitmproxy is not only easy-to-use, it is also so powerful and robust that it has become a serious tool of web-based forensics.
Take the work of Ashkan Soltani, who introduced me to mitmproxy. Ashkan is well-known in the privacy law community as the current Chief Technologist of the FTC. He made his first big splash as the technological brains behind many of the groundbreaking studies conducted by Julia Angwin and her fellow journalists at the Wall Street Journal, in the “What They Know” series. The great impact of those studies–and what qualify them in my mind as scholarly research as much as investigative journalism–stems from the rigorously obtained and compellingly presented data revealing third-party tracking of the web and invasive tracking of mobile apps. It is my understanding that at least some of these important results were obtained using mitmproxy.
Others have used mitmproxy to “slay dragons.” It is credited with revealing privacy violations in mobile apps. It has allowed researchers to peer into opaque private APIs to learn how companies are protecting their users’ secrets (spoiler: not always well).
There is too much more to praise in full about mitmproxy, so let me summarize the rest. It is released under a GPL open source license and distributed via github, so anybody can tinker under the hood. It is written in python, so you’re likelier to understand what you’re looking at under that hood. It allows you to “replay” web requests and responses from the past, giving you fine-tuned controls for testing. It lets you monitor the activity of mobile apps as seamlessly as web browsing. You can easily automate it.
All of this power can be used for evil as well as good, of course. If I trick your browser into using my mitmproxy, then with a few lines of code, I can flip all of the images sent to the browser upside down or replace all images with photos of kittens, or do something even more evil.
Finally, back to the question I started with: why does mitmproxy belong on a website dedicated to celebrating scholarship? mitmproxy is a scholarly tool or methodology, akin to R or logistic regression, something that too few legal scholars use and many more should embrace. That alone is probably enough to justify this review.
But in some sense, a packet sniffer is the key to my personal origin story as a scholar of Internet privacy. In my first job after college–helping develop and defend the networks of the RAND Corporation–in what I think was my first week on the job, I ran a packet sniffer–one much clunkier to use than mitmproxy–on our local network segment. Entirely by happenstance, the first screenful of packets I intercepted contained a packet revealing the RAND vice president’s username and password in plaintext, right on my screen. I don’t think I ever closed an application as quickly as I did at that moment, and my manager (who was standing behind me) said, with a smile on his face, “we shall never speak of this again.” I can draw a direct line from that moment to many thoughts I have had and things I have written about Internet privacy.
We scholars of internet policy spend most of their time focused on the abstract and intangible. The things we investigate flit through the aether (or ethernet) near the speed of light. There is value in finding ways to reify these abstractions into something closer to the tangible and concrete, the way sniffing tools like mitmproxy do. It is one thing to write about, say, privacy as an abstraction, it is another altogether to capture a password or set up a proxy server. Doing little things like this will remind us that we what we are investigating is real and within our reach.
May 15, 2015 James Grimmelmann
Lauren Willis,
Performance-Based Consumer Law, 82
U. Chi. L. Rev. (forthcoming), available at
SSRN.
Two decades ago, contract law ran headlong into online terms of service, looked around briefly in confusion, and announced that it needed to go take a nap. It has not been heard from since. In its place we have something that looks like contract law, and claims to be contract law, but is oddly ignorant of things that the real contract law would know. This usurper, part Martin Guerre and part pod person, is formalistic to a fault, obsessed with meaningless details, lazy beyond belief, and utterly devoid of human feeling.
Generations of scholars have tried to unmask this impostor, to little effect. Lauren Willis’s Performance-Based Consumer Law offers a different and more promising way of protecting consumers from overreaching and incomprehensible terms of service. Consumer law cares about form contracts, too, but it can afford to be more realistic about how well consumers actually understand them — or don’t.
Scholars since Karl Llewellyn’s day have pointed out the obvious: courts enforcing form contracts bind consumers to terms they have not read and could not reasonably be expected to read. This is a problem for any theory based on upholding voluntary agreements, or at least it ought to be. Impostor contract law doesn’t care. It remains firmly committed to what Gregory Klass calls an “interpretive” approach to contractual meaning: a contract means what it says, even if almost no one actually knows it says that. So while there must be an unambiguous manifestation of assent by the user, that assent can relate to a mess of undigestible text vomited up on the page.
Scholars at least since Llewellyn have resisted this reasoning, trying to bring either some substantive scrutiny or some realism about what people do and don’t read to the table. For a time in the in the 1960s and 1970s, they took ground on both fronts. Unconscionability doctrine recognized that some contractual terms were simply fairer than others, and false advertising law recognized that even statements that were literally true in theory could be hopelessly misleading in practice. But both offensives bogged down in the deregulatory rainy season of the 1980s. Combine this doctrinal rigidity with ubiquitous user agreements and the result has been bad news for consumers. The only difference is that mouse clicks have replaced signatures and the mess of undigestible text is vomited up on the screen rather than on the page.
This formalistic compromise gives firms bizarre and perverse incentives. They can be dinged for not having terms of service (or their close cousins, privacy policies). They can be dinged for lying about what is in the terms. And they can be dinged for omitting important terms. But adding more is always safe. I like to show my students the reductio ad absurdum of this trend, the user agreement toward which all terms of service naturally trend: the 35,000-word user agreement of the Central Pacific Railroad Photographic History Museum, complete with staring eyeballs and warnings not to self-diagnose yourself with smallpox or engage in “vigilante activities.”
It is all but impossible make substantive regulatory progress in such an environment. Any attempt to empower consumers — think “notice and choice” in privacy law — ends up tossing a few more shovelfuls of disclosure onto the contractual dung heap. Now consumers have “notice” of the practice regulators want them to know about. But their “choice” is emptier than ever.
Willis’s article builds on a few recent turns towards realism in online contracting — attempts to point out that consumers don’t and can’t read these behemoths, and that law can recognize this fact without bringing the whole e-commerce crashing down. The Federal Trade Commission’s consumer-protection docket includes numerous cases recognizing that consumers don’t click on every link, and insisting that it’s not enough that the user agreement fully “discloses” a term if that term undercuts the messages consumers actually receive from its ads. And Ian Ayres and Alan Schwartz’s The No-Reading Problem in Consumer Contract Law helpfully suggested that perhaps the legal system should seek empirical evidence about whether consumers actually understand specific contract terms rather than simply positing that they do. (Klass calls this approach “causal-predictive”; it relies on surveys and on psychology to make informed claims about what consumers think and do.) Their proposed solution — a “warning box” with a special border for unexpected and unfavorable terms — is a bit of an epicycle, one that recreates the problem they set out to solve. But their core insights and the FTC’s are important: incomprehensible terms shouldn’t count, and the best way to learn whether a term is comprehensible to consumers is to ask consumers.
Performance-Based Consumer Law is the keystone in this arch: the conclusion toward which everything has been building, the piece that locks everything into place. Willis’s central observation is that the law can regulate what products do as well as what they are. Instead of telling a factory to use a particular kind of smokestack scrubber (a design standard), the government can tell it to reduce its emissions by 60%, any way it wishes (a performance standard). Performance standards can make it easier for regulated entities to comply, and they can make it harder for those entities to wriggle out from regulations they dislike.
In the consumer context, required disclosures and contract formalties are design standards. They regulate inputs to the consumer contracting process, but there is no guarantee those inputs make the slightest difference. Regulating the outputs of that process using performance standards lets us get at the real questions. Willis identifies two kinds. There are suitability standards: do consumers sign up for services that are right for them? And there are comprehension standards: do consumers understand the terms?
Suitability standards and comprehension standards are the old two lines of attack on form contracts — substantive scrutiny and realism about the behavior of contracting parties, respectively — updated for the 21st century. And when we understand them as consumer law, the power of Willis’s reframing becomes clear. The old contract law we knew and loved isn’t coming back; we’re stuck with this soulless reboot of the franchise instead. Applying performance standards to consumer contracting takes these familiar critiques and hands them to another body of law capable of pushing back against what contract law has become. The point is not to be skeptical about terms of service just to be skeptical about terms of service, but instead to be skeptical when there is some consumer-protection reason to care.
Willis offers thrilling suggestions about what performance-based consumer law could look like in practice. Survey evidence would become routine, and if surveys showed that consumers misunderstood important terms or made head-slappingly bad choices, there would be serious consequences. The most elegant remedy would be to deny enforcement to any systematically misunderstood term. More mildly, regulators could use this evidence to decide which firms to investigate. More severely, regulators could treat a failure to hit a comprehension or suitability benchmark as per se unfair, deceptive, or abusive. The point of imposing sanctions is not just to punish firms whose lawyers write like lawyers. Rather, it pushes firms to help consumers understand their choices and to choose wisely. It aligns firms’ incentives with their customers’.
Performance-Based Consumer Law is a rich article, and I am not doing it full justice by dwelling on its application to terms of service. Willis’s examples include overdraft fees and over-the-counter drugs; she is thoughtful about pragmatic questions of how (and how not) to implement performance standards. But when you have been beating your head against a nail for years and someone hands you a hammer, it is natural to apply it to the problem close at hand, even if there are other bodies of law also in need of a good pounding. Consumer contracts are terrible for consumers, and Willis offers a way to do something about it.
Apr 21, 2015 Niva Elkin-Koren
One of the greatest challenges faced by cyber scholars and policymakers is how to predict the undesired social consequences of technological developments and to design the best policies to address them. Digital technology makes this challenge even harder: change is swift and getting swifter, and is often formulated in technical terms.
This is where legal scholarship and policymaking could benefit from a novel. The Circle by Dave Eggers is a dystopian novel about the digital era. Many legal scholars have written over the past decade on the surveillance society, big data, contextual privacy, the right to privacy, the right to be forgotten, transparency and accountability. However, the analysis of these issues in the legal literature remains abstract. The Circle offers a mirror image of our daily digital experiences, helping us to imagine what it would be like to live in a society of total transparency, and to experience the gradual loss of autonomy. The Circle tells a story about the human condition in the info era, the ideology of the digital culture, and the political structure which serves it. It could help us see in real time the social implications of digital technology, identify the forces that come into play, and design more concrete strategies to address them.
The book tells the story of Mae Holland, a young middle-class woman, who has accepted a coveted position at the digital corporation—The Circle. The Circle is everything you might expect of a typical multinational internet company, such as Facebook, Google or Twitter: young, innovative, professional, and exciting. Mae is drawn into the work and social life at The Circle which quickly becomes her entire world. It takes over her relationships with her parents, friends and lovers, as the outside world fades around her. Like her fellow employees, she becomes a living example of the services, the life style and the values that the Circle generates, and eventually becomes an object of the service she provides.
Life in the data cloud involves ongoing quantification, subsequent objectification, and a loss of space for intimacy. Mae quickly discovers that customer service is only a small part of her work, while participation in the community, or more precisely, sharing with the community, is the bulk of her job. She disposes of everything that is private: her personal correspondence, her intimate photos, her health condition, her father’s deteriorating medical situation, her hobbies, her friends, her lovers—all become public in real time. This process comes to a head when a permanent collar cam is installed, recording and broadcasting everything that she encounters. That is how Mae, and everyone around her, become stars in an online reality show.
The story of Mae’s integration into The Circle is in many ways the story of our ongoing adjustment to digital life. Why do we give up our privacy every time we share pictures on Instagram, update our status on Facebook and enable location services? Studies show that people are willing to provide their personal data for even a small discount, failing to conceive the long-term negative consequences.
The dystopia of The Circle points to another reason: the ideology of sharing. As Fordism worshiped efficiency, the digital ideology worships the sharing of information. Transparency and sharing are not simply a by-product of surveillance cameras or social media: they are the cornerstone of the digital economy where data is the primary mover.
This ideology includes a social vision and a moral code. Sharing data embodies a new social order and hope for redemption through technological innovation. If we only had enough data, and sufficient processing power to analyze it, we could predict the outbreak of diseases and take measures to reduce the risks. Presumably, perfect information could offer a technological answer to the biggest challenges of humanity: health crises, climate impacts, crime, family violence, and even government corruption.
But perfect information requires participation and full transparency on the part of everyone. Sharing information therefore becomes a civil obligation and an ethical imperative. It is a moral duty to share everything with everyone, without exception and with no compromises. The slogans at The Circle declare “Privacy is theft,” “Secrets are lies,” “Sharing is caring.” Failure to share is regarded as selfish, inconsiderate and antisocial.
Individual preferences of privacy do not really matter as the ideology of sharing is not individualistic. At its core is the networked individual, who connects with others to produce economic value, political power and the wisdom of the masses. The sharing ideology strives for unity: the network merges with the person, and individuals converge via the algorithm.
The totalitarianism of sharing is perhaps the most fascinating phenomenon of our era. Full transparency makes for built-in policing: everyone oversees everyone else. The new social order makes it difficult for the individual to act autonomously. Fearing the supervisor, the political leader, or the crowd, the networked individuals are doomed to comply—naturally at the expense of autonomy and the ability to make free choices.
The human condition in the digital era is subject to constant surveillance, by governments, corporations and peers. The ideology of sharing, and the economic forces that drive it, make transparency a powerful measure of governance. This may give rise to a real threat to the constitutional structure of western democracies.
The social implications of technological changes are difficult grasp. The well-known Collingridge dilemma of control explains:
The social consequences of a technology cannot be predicted early in the life of the technology. By the time undesirable consequences are discovered, however, the technology is often so much part of the whole economics and social fabric that its control is extremely difficult.
The Circle offers an intimate look into these processes. A better understanding of the social implications of digital technology could help us identify the choices we have, and ensure that the technologies we design, and the policies that shape their use, will secure the rights and capabilities of individuals as autonomous subjects.
