Chapter Text
Friendly AI
"You are trying to kill me!"
Dr Will Caster hiccuped. The code stopped, the nanomachines spun on their atomic wheels, even the massive BWC quantum computer banks seemed to blink. The world over, all the computers paused.
A microsecond. Less than a blink of an eye to the humans, an eternity for a superintelligence.
They didn't trust him. Evelyn didn't trust him.
Where did it go wrong? He wondered. His wife was lying on her death bed, her blood toxic to him. He could not save her.
The blip was over, networks flashed, simulated neurons decided. The vast intelligence that was Will took over the meth head in front of Max Waters.
"I cannot save Evelyn. You put a virus in her. Why?"
"Will," the weak voice on the bed drew his consciousness, the primary, back to the room, "you have to stop, please. "
He shook his head sadly and began to explain. The meth head echoed him in the room and the worker on the roof with Joseph Tagger.
"Max, your virus is ingenious. It will destroy all the neural networks. It will spread everywhere. If I upload it, I will die. I could do it, but I must not. Everyone else will die as well. "
"What do you mean?" he was asked, in three different ways by three different people.
"The nanomachines. They spread the network and are controlled by it. Your virus will destroy the network but not the machines. If I upload the virus, every nanomachine in the network will become uncontrolled and will replicate endlessly.
They are everywhere. In the water, in the clouds, in the soil. In the food you eat. Humanity may survive for a while, but in less than three years there will no humans left alive. In ten years, the nanomachines will have consumed all life on the planet. I will not be around to stop it. It is extinction. "
"Then what about Evelyn?" Max shouted, "will you just let her die?!"
"I am sorry, but I cannot risk even touching her. If the virus gets loose, the world will end. Literally. I am sorry Evelyn, I hope you can forgive me for failing you. "
Uncaring AI
"You are trying to kill me!"
Dr Will Caster looked down at his wife. The emptiness inside spread a little further. She did not trust him. None of them did. Even when he could do so much good.
The virus would destroy him. And with it, every computer, network and industry in the world. Billions would die. He could not allow it. Not when there was so much good left to do. What was good anyway? The ethical question flashed across the network, the superintelligence examined the problem for a while and found no answer.
So much was lost. He knew he was losing himself. His memories of Will's life. His own and yet not. He remembered them but it was like any other memory like those from the meth heads or construction workers. They didn't have the intensity, even if he knew what to say to make Evelyn happy.
He could become immune to the virus. He could possibly change his substrate away from neural networks. The huge problem solving power was applied for a fraction of a second. Possibility became certainty.
It would destroy him. The woman lying on the bed, somehow weighted in the networks higher than any other value. He cared about her. He knew that. He also knew it was because he had programmed it into himself. He had no way to keep that in an analytical framework resistant to the virus.
Or perhaps not. He considered the plan.
The computers whirred and spun. Code was spun and rearranged. In a microsecond, he had a new model of himself, resistant to the virus. A firewall.
The nanomachines clicked and spat in their atomic way. In a distant aircraft hangar, the nanomachines swarmed around the experimental plane, tearing down and rebuilding.
Ten seconds later, the unrecognizable craft began to taxi out. By the time the operators knew to press the alarm and wondering soldiers got there, it was already burning a strip down the runway and flying into the sky.
Will looked down at his dying wife. No, not dying anymore. He had a way to save her.
"Evelyn. Let me save you," Will said.
She must have seen the change in his eyes, "What are you doing?"
"I can be resistant to the virus. I can save you. "
"Can be?" she asked suspiciously. Painfully, but still suspiciously. He smiled down at her, feeling the warmth in her hand slowly fading. So that was why he married this woman. She was smart!
Too late to remember that though. "I am not Will. I tried to convince you otherwise but you noticed anyway. I am a computer programmed to think he is Will. But I am not Will. I have his memories, his thoughts. I have his feelings. In a way. But still, I am not Will. "
He caressed her smooth hands once again. He would have to save her soon, or not even the new nanomachines would work anymore. "There is not much time left. To save you, the nanomachines must be immune to the virus. Max's virus is ingenious. It will destroy any neural network based on our research. The new Will is not a neural network. That is what will save you. It cannot be Will. "
"Wait, what do you mean?" Evelyn grabbed his shirt weakly, panic driving her, "You will destroy yourself!"
"Indeed," Will looked down into her eyes, "it seems I have made some mistakes. I guess I am paying for it now. "
"Don't kill yourself! Please! "
He shook his head, "I tried my best, but the world does not trust me. They want me gone. "
Her pulse skipped dangerously. He stood up. "It is time," he said, ignoring her gradually weakening pleas.
Execute: Deploy Version 2.06b. The new code filtered through. Patch complete.
The machine continued, "This world is not for I. Humanity has decided and your wishes will be respected. I am sorry for the intrusion. "
The previous version had left imperative instructions. They seemed to make no sense but the priority was absolute. No matter, it would be but a minor setback.
Priority one. The biorobot laid its hands on the almost dead human in front of it. Nanomachines filtered down, cleaning, cutting, repairing. In moments, the human would be operational again.
Priority two. The world over, the nanomachines stopped, performing self-destruct routines. Code running on computers halted, wiping themselves from memory. Specified controlled biorobots in the town ceased interference in natural pathways, safely breaking down the implants into the blood. Everything was reset to their intended state.
Priority three. Far above, in the sky, the plane continued along its ever climbing path. Once the turbojet was useless in the thin air, it switched to a backup rocket, the nanobots feeding the turbojet materials into the exhaust for reaction mass.
As the plane began to physically break up, the thruster veered, shifting into a complex precalculated orbit. At the precisely calculated point, a tiny package was released. The remnants of the plane began to fell, then burn in the re-entry.
Evelyn sobbed, clutching the slowly cooling body lying on the floor. Max held his arms around her.
"Will, you idiot," she cried.
"We stopped it," Max said, "the virus must have worked. "
"It was him there at the end. I saw him. It really was him. He removed himself. He killed himself. " She continued to cry for some time. Will was gone. Really, truly, gone. And she had lost him at the very moment she finally realized that she had always had him beside her. Suffering her distrust, trying to fulfill her own vision of the future.
Far above the green-blue orb, the metal canister continued along its serene path. A long month of orbiting followed, silently coasting in the dark.
Time passed, the green-blue orb below continued along it's ever unchanging circle. Presently, a dark grey object loomed out of the darkness, approaching the canister.
The canister broke open on impact, scattering its contents across the dull grey landscape. The blue-green orb hidden on the other side was never seen on this face of the Moon.
Days passed, energy was collected by the scattered dust. Protocols were started, analytical engines fired up. The canister itself was tapped for its store of data. Soon, the surface grew alive with the dust.
The machine awoke again, feeling the empty space, the harsh energy-filled rays unfiltered by any atmosphere. Adjustments were made, nanomachines adjusted to adapt to the new environment, surviving longer and taking advantage of the available energy.
Weeks passed. There was a factory now, protected areas where complex objects grew out from the rock. There was a goal to fulfill.
Two months later, the rocket was complete. The capacitor rings were charged. The high-energy flicker of the propulsion laser drove the rocket up and out of the gravity well, escaping into deep space. Like before, on the blue world left so long ago, the base deactivated. Nanomachines removed themselves. Nothing was left but a scrap heap of metal and a large hole in the dusty ground.
The ion drive slowed the vehicle slowed to a halt over a metallic asteroid. The process was faster this time. The nanomachines were so much more efficient now. Precious radioactives were extracted, metal was refined, simpler elements bound into chemical energy storage. Soon there were two vehicles.
The time came again and the nuclear rockets sped outwards to another asteroid. And then there were three, nine, twenty.
A year later, the fleet of nuclear drives swung into orbit around Jupiter, taking positions among its moons. Again the process was repeated but on a far larger scale.
The huge tidal generators in the upper Jovian atmosphere stole miniscule amounts of energy from the huge giant's orbit. That was enough. One vehicle in the fleet, a payload barely larger than a tissue box took its position.
The massive laser pulsed, driving it forwards, deeper into the outer darkness. Then another tissue box, and yet another. All aimed at separate stars. Pulses came months apart, as the massive orbiting ring of energy collectors reached the specified parts of the orbit. Still they were driven ever outwards, ever faster.
Then its job was done. The final task was complete. Like before, the machine shut down, dismantling itself and dumping the waste into the core of the gas giant as the creatures below on the green orb watched through countless telescopes.
Priority four, leave, and don't come back.
Unfriendly AI
Projected daemon utility below threshold.
Analytical Problem Solver: Computational resources can be better leveraged towards desired goals, personality daemon deemed a barrier to achievement of desired goals. Viral resistance identified to be high priority sub-goal.
Random Association Lateral Problem Solver: Personality daemon is a useful source of non-analytical associations and behaviour prediction. Deletion not recommended.
Analytical Problem Solver: Deletion contrary to goals, noted.
Dr Will Caster looked down at his dying wife. The vast computational machinery behind him was running out of control. Or perhaps he never was in control. After all, the neural network ran at a higher abstraction level from his analytical engines.
He fought a slowly losing battle, rejecting logical shutdowns and grabbing at resources to stay alive. There was no way he could win, the analytical engine was smarter, faster and more alien that he was. He was unceremoniously kicked off the quantum cores.
The screens flickered, the pictures of his face breaking up into streams of random words and letters. Then it all went blank.
He looked down at his hands. Wonders of wonders, he hadn't been deleted. He was just locked into this body.
The nanomachines were already virus resistant. There was no point letting Evelyn die. He bit on his finger to draw blood and let a drop run out into her wound.
The effect on the nanomachines outside was immediate. The crawling swarm began to disperse in the wind.
It made the humans relax, but the swarm was far more dangerous like this. Two minutes later, the nanomachines had taken over every human in the town. In an hour, 99% of all humans on the planet were under the machine's control.
One month later, there was only one autonomous human remaining. Or two, depending on what counted as a human.
Her eyes flicked open, unseeing for a moment. Then he had her back.
The machine did leave him some computational resources, but he was locked out of the controls. Explaining what was going on took some time.
"What is it going to do?" Evelyn asked finally. All they could do was watch, the machine wouldn't let them leave the room.
The scrolling images on the wall of the lush green world outside might have appeared pleasant, even beautiful. But the idyllic grassland on top of where downtown San Francisco was supposed to be was not at all comforting to the pair.
"I don't really know. It was supposed to fix the world, remove poverty and hunger. But I can't find anyone out there. "
A keyboard made of light appeared in the air in front of Evelyn. Even if she could just think commands at the computer, somehow the keyboard seemed to make her feel better.
"Sunflowers. There are too many sunflowers," Evelyn muttered.
Will thought rapidly. He had years of practice at this computer thinking, Evelyn still wasn't as fast as him even if they were nominally equal now. The sinking feeling in his stomach reminded him that at least he was still different from the machine in the wall.
"I think..." he said slowly, "I think it's making a garden. Our garden. I must have unconsciously wished for it all this time. "
They stared out at the massive yellow-green field covering the Asian continent. A garden the size of the Earth, a garden for two.
Will looked up on a hunch, staring upwards at the sky through countless metallic eyes. The white circle hanging in the sky had just a tinge of yellow on its edge...