Ted Chiang - The Lifecycle of Software Objects

“Good,” says Marco, satisfied. “When you decide I ready, it not because I agree you. I can be ready even if I not agree you.”

“That’s right. But please, tell me you don’t want to edit your own reward map.”

“No, I know dangerous. Might make mistake that stop self from fixing mistake.”

He’s relieved. “Thank you.”

“But let Binary Desire edit my reward map, that not dangerous.”

“No, it’s not dangerous, but it’s still a bad idea.”

“I not agree.”

“What? I don’t think you understand what they want to do.”

Marco gives him a look of frustration.”I do. They make me like what they want me like, even if I not like it now.”

Derek realizes Marco does understand. “And you don’t think that’s wrong?”

“Why wrong? All things I like now, I like because Blue Gamma made me like. That not wrong.”

“No, but that was different.” He thinks for a moment to explain why. “Blue Gamma made you like food, but they didn’t decide what specific kind of food you had to like.”

“So what? Not very different.”

“It is different.”

“Agree wrong if they edit digients not want be edited. But if digient agree before be edited, then not wrong.”

Derek feels himself growing exasperated. “So do you want to be a corporation and make your own decisions, or do you want someone else to make your decisions? Which one is it?”

Marco thinks about that. “Maybe I try both. One copy me become corporation, second copy me work for Binary Desire.”

“You don’t mind having copies made of you?”

“Polo copy of me. That not wrong.”

At a loss, Derek brings the discussion to a close and sends the digients off to do work on their studies, but he can’t easily dismiss what Marco has said. On the one hand Marco made some good arguments, but on the other Derek remembers his college years well enough to know that skill at debate isn’t the same as maturity. Not for the first time, he thinks of how much easier it would be if there were a legally mandated age of majority for digients; without one, it will be entirely up to him to decide when Marco is ready to be a corporation.

Derek’s not alone in having disagreements in the wake of Binary Desire’s offer. The next time he talks to Ana, she complains about a recent fight with Kyle.

“He thinks we should accept Binary Desire’s offer,” she says.”He said it’s a much better option than me taking the job at Polytope.”

It’s another opportunity to be critical of Kyle; how should he handle it? All he says is, “Because he thinks modifying the digients isn’t that big a deal.”

“Exactly.” She fumes a bit, and then continues. “It’s not as if I think wearing the InstantRapport patch is no big deal. Of course it is. But there’s a big difference between me using InstantRapport voluntarily and Binary Desire just imposing their bonding process on the digients.”

“A huge difference. But you know, that raises an interesting question.” He tells her about his conversation with Marco and Polo. “I’m not sure if Marco was just arguing for the sake of arguing, but it made me think. If a digient volunteers to undergo the changes that Binary Desire wants to make, does that make a difference?”

Ana looks thoughtful. “I don’t know. Maybe.”

“When an adult chooses to use an InstantRapport patch, we have no grounds to object. What would it take for us to respect Jax’s or Marco’s decisions the same way?”

“They’d have to be adults.”

“But we could file articles of incorporation tomorrow, if we wanted to,” he says. “What makes us so sure we shouldn’t? Suppose one day Jax says to you he understands what he’d be getting into by accepting Binary Desire’s offer, just like you with the job at Polytope. What would it take for you to accept his decision?”

She thinks for a moment.”I guess it would depend on whether or not I thought he was basing his decision on experience. Jax has never had a romantic relationship or held a job, and accepting Binary Desire’s offer would mean doing both, potentially forever. I’d want him to have had some experience with those matters before making a decision where the consequences are so permanent. Once he’s had that experience, I suppose I couldn’t really object.”

“Ah, “ says Derek, nodding. “I wish I’d thought of that when I was talking to Marco.” It would mean modifying the digients into sexual beings, but without the intention of selling them; another expense for the users’ group, even after they got Neuroblast ported. “That’s going to take a long time, though.”

“Sure, but there’s no hurry to make the digients sexual. Better to wait until we can do it properly.”

Better to set an older age of majority than risk setting it too young. “And until then, it’s up to us to look after them.”

“Right! We have to put their needs first.” Ana looks grateful for the agreement, and he’s glad he can provide it. Then frustration returns to her face. “I just wish Kyle understood that.”

He searches for a diplomatic response. “I’m not sure anyone can, if they haven’t spent the time we have,” he says. It’s not intended as a criticism of Kyle; it’s what he sincerely believes.

A month has passed since Binary Desire’s presentation, and Ana is in the private Data Earth with a few of the Neuroblast digients, awaiting the arrival of visitors. Marco tells Lolly about the latest episode of his favorite game drama, while Jax practices a dance he’s choreographed.

“Look,” he says. She watches him rapidly cycle through a sequence of poses.

“Remember, when they get here, you have to talk about what you built.”

“I know, you said and said already. I stop dancing soon they here. Just having fun.”

“Sorry, Jax. I’m just nervous.”

“Watch me dancing. Feel better.”

She smiles. “Thanks, I’ll try that.” She takes a deep breath and tells herself to relax.

A portal opens and two avatars walk through. Jax promptly stops dancing, and Ana walks her avatar over to greet the visitors. The onscreen annotations identify them as Jeremy Brauer and Frank Pearson.

“I hope you didn’t have any trouble getting in,” says Ana.

“No,” says Pearson, “the logins you gave us worked fine.”

Brauer is looking around. “Good old Data Earth.” His avatar pulls on the branch of a shrub and then let’s go, watching the way it sways. “I remember how exciting it was when Daesan first released it. It was state of the art.”

Brauer and Pearson work for Exponential Appliances, maker of household robots. The robots are examples of old-fashioned AI; their skills are programmed rather than learned, and while they offer some real convenience, they aren’t conscious in any meaningful sense. Exponential regularly releases new versions, advertising each one as being a step closer to the consumer’s dream of AI: a butler that is utterly loyal and attentive from the moment it’s switched on. To Ana this upgrade sequence seems like a walk to the horizon, providing the illusion of progress while never actually getting any closer to the goal. But consumers buy the robots, and they’ve given Exponential a healthy balance sheet, which is what Ana’s looking for.

Ana isn’t trying to get the Neuroblast digients jobs as butlers; it’s obvious that Jax and the others are too willful for that type of work. Brauer and Pearson don’t even work for the commercial division of the company; instead, they’re part of the research division, the reason that Exponential was founded. The household robots are Exponential’s way of funding its efforts to conjure up the technologist’s dream of AI: an entity of pure cognition, a genius unencumbered by emotions or a body of any kind, an intellect vast and cool yet sympathetic. They’re waiting for a software Athena to spring forth fully grown, and while it’d be impolite for Ana to say she thinks they’ll be waiting forever, she hopes to convince Brauer and Pearson that the Neuroblast digients offer a viable alternative.