Ted Chiang - The Lifecycle of Software Objects

“Well, thank you for coming out to meet me,” says Ana.

“We’ve been looking forward to it,” says Brauer. “A digient whose cumulative running time is longer than the lifespan of most operating systems? You don’t see that very often.”

“No, you don’t.” Ana realizes that they came more for nostalgia’s sake than to seriously entertain a business proposal. Well, so be it, as long as they’re here.

Ana introduces them to the digients, who then give little demonstrations of projects they’ve been working on. Jax shows a virtual contraption he’s built, a kind of music synthesizer that he plays by dancing. Marco gives an explanation of a puzzle game he’s designed, one that can be played cooperatively or competitively. Brauer is particularly interested in Lolly, who shows them a program she’s been writing; unlike Jax and Marco, who built their projects using toolkits, Lolly is writing actual code. Brauer’s disappointment is evident when it becomes clear that Lolly is just like any other novice programmer; it’s clear he was hoping her digient nature had given her a special aptitude for the subject.

After they’ve talked with the digients for a while, Ana and the visitors from Exponential log out of Data Earth and switch to videoconferencing.

“They’re terrific,” says Brauer. “I used to have one, but he never got much beyond baby talk.”

“You used to have a Neuroblast digient?”

“Sure, I bought one as soon as they came out. He was an instance of the Jax mascot, like yours. I named him Fitz, kept him going for a year.”

This man had a baby Jax once, she thinks. Somewhere in storage is a baby version of Jax that knows this man as his owner. Aloud, she says, “Did you get bored with him?”

“Not so much bored as aware of his limitations. I could see that the Neuroblast genome was the wrong approach. Sure Fitz was smart, but it would take forever before he could do any useful work. I’ve got to hand it to you for sticking with Jax for so long. What you’ve achieved is impressive.” He makes it sound like she’s built the world’s largest toothpick sculpture.

“Do you still think Neuroblast was the wrong approach? You’ve seen for yourself what Jax is capable of. Do you have anything comparable at Exponential?” It comes out more sharply than she intended.

Brauer’s reaction is mild. “We’re not looking for human-level AI; we’re looking for superhuman AI.”

“And you don’t think that human-level AI is a step in that direction?”

“Not if it’s the sort that your digients demonstrate,” says Brauer.

“You can’t be sure that Jax will ever be employable, let alone become a genius at programming. For all you know, he’s reached his maximum.”

“I don’t think he has–”

“But you don’t know for certain.”

“I know that if the Neuroblast genome can produce a digient like him, it can produce one as smart as you’re looking for. The Alan Turing of Neuroblast digients is just waiting to be born.”

“Fine, let’s suppose you’re right,” says Brauer; he’s clearly indulging her. “How many years would it take to find him? It’s already taken you so long to raise the first generation that the platform they run on has become obsolete. How many generations before you come up with a Turing?”

“We won’t always be restricted to running them in real time. At some point there’ll be enough digients to form a self-sufficient population, and then they won’t be dependent on human interaction. We could run a society of them at hothouse speeds without any risk of them going feral, and see what they produce.” Ana’s actually far from confident that this scenario would produce a Turing, but she’s practiced this argument enough times to sound like she believes it.

Brauer isn’t convinced, though. “Talk about a risky investment. You’re showing us a handful of teenagers and asking us to pay for their education in the hopes that when they’re adults, they’ll found a nation that will produce geniuses. Pardon me if I think there are better ways we could spend our money.”

“But think about what you’re getting. The other owners and I have devoted years of our attention to raising these digients. Porting Neuroblast is cheap compared to what it’d cost to hire people to do that for another genome. And the potential payoff is exactly what your company’s been looking for: programming geniuses working at high speed, bootstrapping themselves to superhuman intelligence. If these digients can invent games now, just imagine what their descendants could do. And you’d make money off every one of them.”

Brauer is about to reply when Pearson interjects. “Is that why you want Neuroblast ported? To see what superintelligent digients might invent one day?”

Ana sees Pearson scrutinizing her, and decides there’s no point in trying to lie. “No,” she says. “What I want is for Jax to have a chance at a fuller life.”

Pearson nods.”You’d like Jax to be a corporation one day, right? Have some sort of legal personhood?”

“Yes, I would.”

“And I’ll bet Jax wants the same thing, right? To be incorporated?”

“For the most part, yes.”

Pearson nods again, his suspicions confirmed. “That’s a deal-breaker for us. It’s nice that they’re fun to talk to, but all the attention you’ve given your digients has encouraged them to think of themselves as persons.”

“Why is that a deal-breaker?” But she knows the answer already.

“We aren’t looking for superintelligent employees, we’re looking for superintelligent products. You’re offering us the former, and I can’t blame you; no one can spend as many years as you have teaching a digient and still think of it as a product. But our business isn’t based on that kind of sentiment.”

Ana has been pretending it wasn’t there, but now Pearson has stated it baldly: the fundamental incompatibility between Exponential’s goals and hers. They want something that responds like a person, but isn’t owed the same obligations as a person, and that’s something she can’t give them.

No one can give it to them, because it’s an impossibility. The years she spent raising Jax didn’t just make him fun to talk to, didn’t just provide him with hobbies and a sense of humor. It was what gave him all the attributes Exponential was looking for: fluency at navigating the real world, creativity at solving new problems, judgment you could entrust an important decision to. Every quality that made a person more valuable than a database was a product of experience.

She wants to tell them that Blue Gamma was righter than it knew: experience isn’t merely the best teacher, it’s the only teacher. If she’s learned anything raising Jax, it’s that there are no shortcuts; if you want to create the common sense that comes from twenty years of being in the world, you need to devote twenty years to the task. You can’t assemble an equivalent collection of heuristics in less time; experience is algorithmically incompressible.

And even though it’s possible to take a snapshot of all that experience and duplicate it ad infinitum, even though it’s possible to sell copies cheaply or give them away for free, each of the resulting digients would still have lived a lifetime. Each one would have once seen the world with new eyes, have had hopes fulfilled and hopes dashed, have learned how it felt to tell a lie and how it felt to be told one.

Which means each one would deserve some respect. Respect that Exponential can’t afford to give.

Ana makes one final attempt. “These digients could still make money for you as employees. You could–”

Pearson shakes his head. “I appreciate what you’re trying to do, and I wish you the best of luck, but it’s not a good match for Exponential. If these digients were going to be products, the potential profits might be worth the risk. But if all they’re going to be is employees, that’s a different situation; we can’t justify such a large investment for so little return.”