When I look over all the books I read last year, one stands out as having reformed my world view more than any other: “The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power” by Shoshanna Zuboff.
It is not your grand-father’s internet any more. What we are living through is a brand new kind of technology, and a brand new kind of business built on and for that. It astoundingly hard to grasp the nature of a paradigm shift while it is happening. Shoshanna Zuboff puts it all together into a single book: the history, the discovery, the development, from the Google taking the responsibility to find the right place to put the ad, to predicting behavior from digital exhaust, to the surprising technique of guaranteed outcomes.
You already know that as users of Google search, we are not the customer. But we are not the product either. We are merely the source of something else: behavioral surplus. Google, and Facebook, and Amazon vacuum up that surplus which consists of old search queries, logs of mouse clicks, records of sites visited, and most importantly: “like” button clicks. None of this is of any value to us, but when aggregated it can tell a powerful story about each and every one of us. That behavioral surplus can be used to predict what we will do. If it was only prediction it would not be a particular problem. But this is where it turns dark: if you have a model that can predict what a person will do, then you can use that model to make people do things.
Our democracy and capitalism is built on the idea of a free market. Adam Smith described the “invisible hand” of the market, that while millions of people made independent decisions for our own personal welfare, the market was structured to find a nearly optimal distribution of goods ans services. The hand is invisible because it was impossible to know everything the everyone was doing. The aggregate behavior of all people was unknowable.
Google and Facebook have detailed profiles on most of the people in the country: some thorough and some not, but it is all getting much more complete as new devices emit ever more digital exhaust to be collected. It is surveillance-as-a-service. We can be quite sure the 2016 Brexit vote and the Trump victory were due in no small part to manipulation with the tools of surveillance capitalism. What happens to democracy when a company can sell guaranteed outcomes for a price? What happens to the free market when the invisible hand disappears and is replaced by plutocrats who simply manipulate the public will to their favor?
We don’t know the answers to these questions, and Zuboff doesn’t either. It is too new. It is all just forming. Most of our legislators have only the faintest grasp of what is going on, much less what to do about it. But this is clearly the most important development of our time. If you want to understand it, this book is a monumental resource: well organized, well researched, well written.
This is a timely book, not too soon, and not too late. On a five-star scale, I am giving it a six. It is that important.
Wait … we have had mass propaganda before that persuaded people and in some sense controlled them. Why is it any different this time? The difference is that through the collection of behavioral surplus, a very detailed model of every citizen can be constructed. From this model, algorithms can fashion a detailed plan for exactly what to display unique to each person. But it does not stop there: your behavior is further monitored to see which things you reacted to and which you didn’t. This is used to further tune the model. Advertisements, memes, opinions, political statements, images are all thrust into the sidebars of the web pages. No two people get the exact same mix. Your response on every one can be captured and categorized independently. There has never in history been the ability to customize propaganda campaigns for individuals and be able to measure exactly how each individual responds. We have no precedence for this, and we have no idea where it is going or even how to change course if we see something amiss.
Zuboff gives us detailed example after example of how this came about, how it is being used, and what the legal implications are. Even Roger McNamee who has just published a similarly themed book called “Zucked” admits that the big difference is that Zuboff has all the data. She compare this emergence to that of Totalitarianism just 100 years ago. Totalitarianism enveloped the individual in a state-wide system of total control. We are not to fear that this is all going toward totalitarianism — that was last time, and this time is different. She coins the term “Instrumentarianism” to describe the way that unknown algorithms are at the root of collecting the data, building the models, and crafting the plan of persuasion.
I guarantee in the next 5 to 10 years, you will be hearing a lot about Surveillance Capitalism and Instrumentarianism. We will need these new terms, and many other presented concepts just to understand what is happening. Only then, can we hope to navigate these stormy seas to the (hopefully) safe harbor once we know how to tame this new beast.