This essay originally appeared in ‘Knowledge and Information — Perspectives from Engelsberg Seminar, 2018’, Bokförlaget Stolpe, in collaboration with the Axel and Margaret Ax:son Johnson Foundation.

Over the last couple of centuries, the relationship between technology and the individual freedom essential to democracy has been premised on what we might call the ‘Winston Smith Principle’. Smith, you’ll remember, was the fictional model of resistance to totalitarianism in George Orwell’s dystopian 1948 novel Nineteen Eighty Four. He represented the human antidote to a nightmarish political system combining the most disturbing elements of Bolshevism and Nazism. Orwell presents Smith as the final hope, the last man, in the face of a regime that spied on everyone and everything. He is the human resistance, the democratic alternative to Big Brother.

In Nineteen Eighty-Four, technology was presented as the key weapon of a secret police state intent on destroying privacy and demanding the complete obedience of its citizens. Spying technology was ubiquitous. There were microphones and cameras in every room, watching and listening to everything we did and everywhere we went. Society had been transformed into a giant inspection house — a place of radical transparency not unlike the ‘panopticon’, that ‘simple idea of architecture’ imagined by the late eighteenth-century utilitarian thinker Jeremy Bentham. In Orwell’s Room 101, the hellhole in the Ministry of Truth where ‘the worst thing in the world could be found’, there was even technology seeking to invade our own consciousness.

Or, at least, technology was almost everywhere in Nineteen Eighty-Four. For all the nightmarish qualities of Room 101, the one place it couldn’t quite occupy was our own heads. Winston Smith’s own head was the resistance. Big Brother never colonised what has always seemed to be the essentially private realm of our own minds.

Over the last 200 years, this Winston Smith Principle has had a profound influence on democratic theory. In John Stuart Mill’s 1859 book On Liberty, for example, arguably the most important nineteenth-century work setting out the limits of state authority over the individual, Mill argues that it’s the free-thinking person — the Winston Smith in all of us — who determines social progress. And so, for Mill, whose On Liberty was partially an attack on the utilitarianism of his godfather Jeremy Bentham, democracy needed to be built around a defence of the sanctity of our own thoughts. A defence of privacy, for Mill, therefore, becomes one of the key components of democracy.

Mill’s ideas about democracy were developed more fully by the American jurists Samuel Warren and Louis Brandeis in their 1890 Harvard Law Review piece, ‘The Right to Privacy’. Writing in response to the nascent mass media technologies of photography and newspapers, Brandeis and Warren argued that privacy meant the ‘right of the individual to be let alone’.

In the face of these potentially intrusive late nineteenth-century technologies, Brandeis and Warren argued, in 1890, that ‘solitude and privacy have become more essential to the individual.’ ‘The right to be let alone’, they explained, was a general right to ‘the immunity of the person… the right to one’s personality.’

Calibrating the right balance between technology and individual autonomy from the state, Warren and Brandeis thus suggest, is essential to democracy. Privacy matters. Without it, there can be no real individual freedom.

More than 125 years after the publication of ‘The Right to Privacy’ nothing much has changed in terms of the importance we give to the ideal of the private self. As the contemporary American technology writer Nicholas Carr notes, ‘we human beings are not just social creatures; we’re also private creatures. What we don’t share is as important as what we do share.’

And yet while nothing has changed in terms of our ideas about democracy, everything has changed in terms of the new technologies of our networked age. The digital revolution, driven by the creation of both the internet and the world wide web over the last half century, has created a very different informational world to Samuel Warren and Louis Brandeis’ nineteenth-century world. Today, the industrial technologies of photography and print have been replaced by the digital technologies of blogs, tweets, Instagram photos, YouTube videos, Facebook log-ins, WhatsApp conversations and Google searches.

On the internet, those ubiquitous microphones and cameras in Nineteen Eighty-Four have been turned on ourselves. We are increasingly living in an age of big data — a vast digital panopticon which Facebook CEO Mark Zuckerberg once fondly described as a ‘well-lit dorm room’. The scale of this data economy is mind-boggling. Three-and-a-half billion internet users around the world create 2.5 quintillion bytes of data each day. In every minute of every day of 2016, for example, we made 2.4 million Google searches, watched 2.78 million videos, entered 701,389 Facebook log-ins, added 36,194 new posts to Instagram, and exchanged 2.8 million messages on WhatsApp.

The problem, of course, is that much of this data is transparent to everyone. In our digital age, individual privacy is in crisis. As the National Security Agency whistleblower Edward Snowden revealed, the internet is increasingly a place dominated by a ubiquitous government surveillance. And as many critics of the new internet economy — with its supposedly ‘free’ services and platforms like YouTube, Google, Facebook and Instagram — have argued, this economy isn’t really very new. It’s actually a surveillance-style economic system in which we are all collectively being watched in everything we do and say by big data companies like Google, Facebook and Twitter.

Privacy, in other words, is in crisis. Orwell’s dystopia has been resurrected in digital form. Only in today’s networked economy, we are all Winston Smith — struggling to protect our identities in the face of forces intent on knowing everything we think and everywhere we go.

Most troublingly, we are still in the very earliest stages of a revolution which is producing technologies that reveal our innermost thoughts — the data adding up to what Samuel Warren and Louis Brandeis described as ‘our personality’. Artificial intelligence (AI) companies are developing algorithms, for example, which can determine our voting or consumer habits by just crunching our public data. And we are inventing facial recognition technologies that can identify our sexuality without knowing anything else about us. Indeed, the Chinese government is already developing a facial recognition policy known as ‘Sharp Eyes’ designed to verify people’s identities through their faces.

You’ll remember that in Nineteen Eighty-Four’s Room 101 there was technology seeking to invade our own consciousness. But in an age of big data, with increasingly sophisticated AI that can determine our thoughts, Orwell’s ‘worst place in the world’ is, unfortunately, becoming a reality.

In political terms, this digital dystopia is manifesting itself most troublingly in contemporary China where Orwell’s twentieth-century warnings about a Nineteen Eighty-Four-style intrusive ideological dictatorship have acquired a twenty-first-century digital form. The idea began in 2010 with local communist party officials in Suining County in the Jiangsu Province north of the Chinese city of Shanghai. The local government began to award people points for good civic behaviour and deduct points for everything from traffic tickets to ‘illegally petitioning higher authorities for help’. The points were then tallied, and people were placed into reputational tables to determine their civic reliability. If you rated highly, you would qualify for fast-track promotions at work or for access to public housing. If not, you were unlikely to be promoted, find a new home, or even qualify for social security support.

In late 2016, the Chinese authorities published a plan to establish a much more ambitious social credit system that, in the Ministry of Truthstyle language of the communist party, would build a culture of what the party calls ‘sincerity’ and a ‘harmonious socialist society’. Launched in three dozen local governments across China, as well as with eight private technology companies, and aided by increasingly sophisticated facial recognition technology, this new initiative, called Internet Plus, is designed to collect the information of the 730 million Chinese internet users into regional databases that will determine their individual trustworthiness. The official goal is to unite all these different databases by 2020, creating a national social rating system that will rank individuals according to their online data.

Presumably Internet Plus, like the 2010 Suining County pilot project, will reward loyal citizens and punish those who are deemed, by China’s intended second brain, to be politically troublesome or unreliable. The data-engineered caste system in the China of 2020 will, no doubt, be made up of two groups: the trustworthy and the untrustworthy. As Chinese officials put it, by 2020 this system will ‘allow the trustworthy to roam everywhere under heaven while making it hard for the discredited to take a single step’. The underclass, to remix Marx, will have nothing to lose but their bad ranking.

The logic behind China’s Internet Plus scheme, of course, is to tighten the undemocratic grip of the communist party on society. In October 2016 President Xi Jinping called for innovation in ‘social governance that would, in his words, ‘heighten the capacity to forecast and prevent all manner of risks’. Rather than the creation of trust, the point of Internet Plus — which The Economist describes as a ‘digital totalitarian state’ — is to punish ‘untrustworthiness’. To add to the Orwellian nature of this networked dystopia, individuals will be able to enhance their trust scores by informing on the untrustworthiness of others. Thus Internet Plus will, according to China’s elite State Council, ‘forge a public opinion environment that trust-keeping is glorious’ and, at the same time, ‘reward those who report acts of breach of trust’. The most trusted, then, in this surreal system, will be the most untrustworthy. No wonder one Hong Kong human rights activist told the Wall Street Journal: ‘It’s just like 1984.’

So how can we make the digital future unlike Nineteen Eighty-Four? How can we protect individual rights and, by extension, democracy in an age of big data?

The obvious solution is to try to replicate Samuel Warren and Louis Brandeis’ attempts to legislate in defence of individual privacy. This regulatory strategy against the big data economy is most actively being pursued in the European Union through its General Data Protection Regulation (GDPR).

Adopted in April 2016 by the European Parliament, the European Council, and the European Commission after four years of negotiation, the GDPR came into force in May 2018 and has been designed as a single set of rules for member countries to ensure that privacy will become ‘the norm’ in European networked society. Offering to guarantee the ‘fundamental right’ to personal data protection for all EU residents, the GDPR is an attempt to turn today’s data equation on its head by enabling individuals to own what technologists call their ‘social graph’. Instead of big data companies owning us, we will not only own our data, we will be able to delete it or take it with us wherever we go on the internet. Google will no longer be allowed, in the words of Gary Reback, a prominent Silicon Valley anti-trust lawyer, to unilaterally ‘build profiles of everyone’. Some are calling for a similar ‘Social Graph Portability Act’ to be created by the US Congress. They would be wise to use the GDPR Act in the EU as a model for this congressional legislation.

The GDPR’s ‘privacy as the norm’ legislation offers an entirely new ecosystem for privacy and data in our age of surveillance capitalism controlled by Silicon Valley monopolies. The GDPR is crystal clear about who owns our data on Instagram, Facebook and Google. It’s ours. Ours alone. The internet is reimagined as a place where individual privacy is not only the norm but also the highest priority. While the current absence of clear laws puts all the burden on individuals to figure out how their data is being exploited, the GDPR puts all the burden on the big data companies in terms of their accountability for the fate of online data. In the language of the European Parliament, the legislation ‘puts the citizen back in the driving seat’.

The EU legislation enshrines the so-called right-to-be-forgotten ruling that gives individuals the right to have their personal data erased from the internet. A person must provide ‘clear and affirmative consent’ if data companies are going to process his or her private data. The law requires companies to inform individuals if their data has been hacked or tampered with, and it gives people the right to switch their personal data to another service provider. It also warns that companies face fines of up to 4 per cent of their global revenue for breaking these new laws.

While the GDPR focuses on constraining private companies, it does little to address the big data relationship between governments and its citizens. But what about the transparency of this relationship? How can we protect citizens from the prying digital eye of the government so that China’s Orwellian Internet Plus program isn’t also established in Western democracies?

What is needed is a new social contract between citizens and government in our big data age. Unlike in the industrial age, individual privacy can’t simply be protected by the kind of legislation encouraged by industrial age thinkers like Samuel Warren and Louis Brandeis. Instead, what is needed is new contract of trust between citizens and government in our increasingly transparent digital age.

This new relationship, a forging of a kind of new digital democracy, is being pioneered in countries as diverse as Estonia and Singapore. Both these small, digitally advanced states are experimenting with comprehensive digital identity systems that make both citizens and government fully accountable for their actions. In Singapore, this is known as the ‘Smart Nation’ initiative; in Estonia, it is called the ‘National ID System’.

The thinking behind these new social contracts is that, in our big data age, individual privacy is becoming an archaic ideal. For better or worse, we are all living in public in our networked society. So attempting to protect privacy might indeed be a form of misguided nostalgia for a world that can no longer be recreated.

One of the leading thinkers in the establishment of democracy in this post privacy age is Toomas Hendrik Ilves, the former two-term President of Estonia. ‘Our obsession with privacy is misguided,’ Ilves told me, in explaining the thinking behind the pioneering Estonian model of digital democracy. ‘The real issue is data integrity.’

It’s not that Ilves is completely dismissing the significance of privacy or diminishing its importance as a component of individual freedom. But the real issue for government in an internet world, where data about everything and everyone is readily accessible on Google, he believes, is refereeing a system where that data is authentic. Ilves is saying that there’s nothing more important than data integrity in a digital twenty-first-century, where everything — including ourselves — is turned into information. And so the role of government is to create a trustworthy informational exchange system that is secure. If our personal data is, indeed, the new currency of the networked age, then it requires what he calls a ‘sovereign guarantee’. Like currency, it has value only if it has this official stamp of authenticity.

‘Somebody knowing my blood type isn’t a big deal. But if they could change the data on my blood type — that could kill me,’ Ilves explained. ‘The real problem is when somebody starts fiddling with the data.’ The spectre of online surveillance, therefore, worries Ilves much less than that of data corruption. And this explains why so much time and so many resources in Estonia have been invested in its second brain, the online ID system. This platform creates ways to exchange data that are completely secure.

‘Your data is yours,’ another of the digital designers of Estonia’s ID system assured me. But the information is really yours in Estonia only if the government is ensuring that nobody is tampering with it. The greatest service, then, of what Estonians call government-as-a-service is establishing data integrity.

The role of the sovereign in the digital twenty-first-century, Ilves insists, is to guarantee our identity. He calls this a ‘Lockean contract’ and describes it as the ‘new social contract for digital times’.

Given that the original Lockean contract underpinning Anglo-American representative democracy operates on a series of mutual obligations between the government and its citizens, I asked Ilves about the equivalent obligations for our digital age. ‘So what’s our responsibility in this new social contract?’ I inquired. ‘If the government guarantees the integrity of the data, what do citizens need to guarantee in exchange?’

‘It’s a fully transparent system,’ Ilves responded. Just as it keeps government honest, he says, so it also keeps us honest. The government can, if it needs to, examine our data. So we all have to take responsibility for our own online behaviour.

In the information-rich democracy being constructed in Estonia, he suggests, there can be no digital anonymity. Everything people do — from paying taxes online to ordering medicine to posting opinions to driving cars — is done under their own names. So, for example, Estonian newspapers are connecting the ID system to their bulletin boards, making it impossible to comment anonymously. This new social contract does away with the trolls who have made the internet such a barbaric place. And it makes people accountable for the spreading of fake news, racism and sexism, spiteful rumours, and the other anti-social and often undemocratic behaviour seemingly endemic in digital culture.

‘Our goal is to make it impossible to do bad things without consequences. We want to teach people to be good on the internet, to use it responsibly,’ Ilves concluded.

In a way, of course, this is all very chilling — especially to people like Edward Snowden, who lionise individual privacy. The Estonian ID operating system, with its guarantee of identity, is, in contrast, a mutually transparent system. What Ilves describes as a new social contract is based on the rights of both government and citizens to observe each other. The watching is done with full transparency, within a comprehensive legal framework that requires the authorities to alert people if they look at their data.

It’s an architecture of trust designed for what Andreas Weigend, the former chief scientist of Amazon, calls our ‘post-privacy’ world. And that, indeed, might be the fate of democracy in the digital twenty-first-century. For better or worse, it may be time to retire the Winston Smith Principle. Rather than protecting individual privacy, the fate of democracy in our networked age might depend on establishing a new, radically transparent contract of trust between government and citizens.