Eli Pariser - The Filter Bubble

Stanford Law professor Ryan Calo thinks a lot about robots, but he doesn’t spend much time musing about a future of cyborgs and androids. He’s more interested in Roombas, the little robotic vacuum cleaners currently on the market. Roomba owners name their machines like pets. They delight in watching the little bumbling devices wander around the room. Roombas provoke an emotional response, even a sense of relationship. And in the next few years, they’ll be joined by a small army of consumer-electronic brethren.

The increasing prevalence of humanlike machines in everyday life presents us with new dilemmas in personalization and privacy. The emotions provoked by “humanness,” both virtually (advertars) and in reality (humanlike robots) are powerful. And when people begin to relate to machines as we do to humans, we can be convinced to reveal implicit information that we would never directly give away.

For one thing, the presence of humanoid faces changes behavior, compelling people to behave more like they’re in public. The Chinese experiment with Jingjing and Chacha, the cartoon Internet police, is one example of this power. On the one hand, Calo points out, people are much less likely to volunteer private information when being interrogated by a virtual agent than when simply filling out a form. This is part of why the intelligent-agent craze didn’t work out the first time around: In many cases, it’s easier to get people to share personal information if they feel as though they’re privately entering it into an impersonal machine rather than sharing it with people.

On the other hand, when Harvard researchers Terence Burnham and Brian Hare asked volunteers to play a game in which they could choose to donate money or keep it, a picture of the friendly looking robot Kismet increased donations by 30 percent. Humanlike agents tend to make us clam up on the intimate details of our lives, because they make us feel as if we’re actually around other people. For elderly folks living alone or a child recovering in a hospital, a virtual or robotic friend can be a great relief from loneliness and boredom.

This is all to the good. But humanlike agents also have a great deal of power to shape our behavior. “Computers programmed to be polite, or to evidence certain personalities,” Calo writes, “have profound effects on the politeness, acceptance, and other behavior of test subjects.” And because they engage with people, they can pull out implicit information that we’d never intend to divulge. A flirty robot, for example, might be able to read subconscious cues—eye contact, body language—to quickly identify personality traits of its interlocutor.

The challenge, Calo says, is that it’s hard to remember that humanlike software and hardware aren’t human at all. Advertars or robotic assistants may have access to the whole set of personal data that exists online—they may know more about you, more precisely, than your best friend. And as persuasion and personality profiling get better, they’ll develop an increasingly nuanced sense of how to shift your behaviors.

Which brings us back to the advertar. In an attention-limited world, lifelike, and especially humanlike, signals stand out—we’re hardwired to pay attention to them. It’s far easier to ignore a billboard than an attractive person calling your name. And as a result, advertisers may well decide to invest in technology that allows them to insert human advertisements into social spaces. The next attractive man or woman who friends you on Facebook could turn out to be an ad for a bag of chips.

As Calo puts it, “people are not evolved to twentieth-century technology. The human brain evolved in a world in which only humans exhibited rich social behaviors, and a world in which all perceived objects were real physical objects.” Now all that’s shifting.

The future of personalization is driven by a simple economic calculation. Signals about our personal behavior and the computing power necessary to crunch through them are becoming cheaper than ever to acquire. And as that cost collapses, strange new possibilities come within reach.

Take facial recognition. Using MORIS, a $3,000 iPhone app, the police in Brockton, Massachusetts, can snap a photo of a suspect and check his or her identity and criminal record in seconds. Tag a few pictures with Picasa, Google’s photo-management tool, and the software can already pick out who’s who in a collection of photos. And according to Eric Schmidt, the same is true of Google’s cache of images from the entire Web. “Give us 14 images of you,” he told a crowd of technologists at the Techonomy Conference in 2010, “and we can find other images of you with ninety-five percent accuracy.”

As of the end of 2010, however, this feature isn’t available in Google Image Search. Face.com, an Israeli start-up, may offer the service before the search giant does. It’s not every day that a company develops a highly useful and world-changing technology and then waits for a competitor to launch it first. But Google has good reason to be concerned: The ability to search by face will shatter many of our cultural illusions about privacy and anonymity.

Many of us will be caught in flagrante delicto. It’s not just that your friends (and enemies) will be able to easily find pictures other people have taken of you—as if the whole Internet has been tagged on Facebook. They will also be able to find pictures other people took of other people, in which you happen to be walking by or smoking a cigarette in the background.

After the data has been crunched, the rest is easy. Want to search for two people—say your boyfriend and that overly friendly intern you suspect him of dallying with, or your employee and that executive who’s been trying to woo him away? Easy. Want to build a Facebook-style social graph by looking at who appears most often with whom? A cinch. Want to see which of your coworkers posted profiles on anonymous dating sites—or, for that matter, photos of themselves in various states of undress? Want to see what your new friend used to look like in his drugged out days? Want to find mobsters in the Witness Protection program, or spies in deep cover? The possibilities are nearly limitless.

To be sure, doing face recognition right takes an immense amount of computing power. The tool in Picasa is slow—on my laptop, it crunches for minutes. So for the time being, it may be too expensive to do it well for the whole Web. But face recognition has Moore’s law, one of the most powerful laws in computing, on its side: Every year, as processor speed per dollar doubles, it’ll get twice as cheap to do. Sooner or later, mass face recognition—perhaps even in real time, which would allow for recognition on security and video feeds—will roll out.

Facial recognition is especially significant because it’ll create a kind of privacy discontinuity. We’re used to a public semianonymity—while we know we may be spotted in a club or on the street, it’s unlikely that we will be. But as security-camera and camera-phone pictures become searchable by face, that expectation will slip away. Shops with cameras facing the doors—and aisles—will be able to watch precisely where individual customers wander, what they pick up, and how this correlates with the data already collected about them by firms like Acxiom. And this powerful set of data—where you go and what you do, as indicated by where your face shows up in the bitstream—can be used to provide ever more custom-tailored experiences.

It’s not just people that will be easier than ever to track. It’s also individual objects—what some researchers are calling the “Internet of things.”