A few months ago, I was driving through Los Angeles when the Waze app on my phone told me to take the Stadium Way exit off the 110 freeway. About five other cars peeled off with me, and we became a caravan, snaking through side streets and back onto the freeway a few miles later. I knew Waze had to be in charge of us, since Waze is the navigation app of choice in Los Angeles, and it was beyond coincidence that all these cars took the same wild maze run through streets only locals knew well.
What was Waze up to here, besides offering its users (or a subset of them) a way around a jam? Was it optimizing traffic by taking some cars off the highway and leaving others on? Running an experiment only some AI understood? There was no way to tell. I doubt anyone at Waze could say exactly what was going on either. Algorithms are like that. So are the large and constantly changing data sets informing algorithms most of us with mobile devices depend on every day.
[...]
Surveillance of people is now the norm for nearly every website and app that harvests personal data for use by machines. Privacy, as we’ve understood it in the physical world since the invention of the loincloth and the door latch, doesn’t yet exist. Instead, all we have are the “privacy policies” of corporate entities participating in the data extraction marketplace, plus terms and conditions they compel us to sign, either of which they can change on a whim. Most of the time our only choice is to deny ourselves the convenience of these companies’ services or live our lives offline.
[...]
But what happens when our senses extend beyond the metal carapace we wear when we drive a car—outward through the unseen systems guiding our selves and every other car on the road? In this state we are not GPS satellites and Google data centers, but rather puppets at the ends of digital strings pulled by AI puppeteers.
We surely appreciate and rely on what they provide us, but we also yield agency in the process of blurring between our automotive selves and a vast system of dependencies, which even if they are doing good things for us, make us less than human—or, in Brett and Evans’ words, “simple machines under the control and influence of those in control of technologies”. Inevitably, they also say, “traffic engineers will assume the role of social planners.” But the traffic engineers they’re talking about are not the human kind working for highway departments, but machines run by companies making navigation apps, all of which have purposes beyond providing personal and civic goods.
The challenge Brett and Evan pose in their concluding chapter is “how to sustain the freedom to be off, to be free from techno-social engineering, to live and develop within undetermined techno-social environment”.