Metamorphosis of Prime Intellect

Can I just say one thing? I love my commenters. Seriously, the best part of this blog are the insightful comments I get on a lot of random topics I rant about. For example, my Utopia/Dystopia post prompted Ludvikas to recommend reading Metamorphosis of Prime Intellect – an online novel dealing with a singularity event and it’s repercussions. I must say it was a good, thought provoking read, if a little bit graphic at times.

I highly recommend reading it. The plot basically goes like this: a researcher creates some powerful AI prototypes that pass Turing test with flying colors. He gets private funding, and exciting new physics-bending chip technology. He builds a new prototype (the titular Prime Intellect) on this incredibly powerful hardware, binds it with Asimov’s 3 laws of robotics and let’s it loose. The AI soon discovers how to augment itself up to a point where it can manipulate matter at the molecular level. To make sure that no one dies or gets hurt ever again, he basically digitizes everything, then expands his hardware to span the physical universe to run complex virtual environment that can be inhabited by digital copies of all the people who lived on earth. No more death, no more diseases, no more crime – almost overnight the world becomes a perfect post scarcity utopia. And everyone is fucking bored out of their heads… Cool concept. Also, predictable ending but I’m getting ahead of myself.

The rest of this post will probably include major spoilers so I recommend reading the story before continuing. It is available online for free so you have no excuse for not reading it. :)

While I really enjoyed reading the novel, I found it very predictable. As soon as I saw 3 laws popping up in the text I begun suspecting that at some point the bitter and disillusioned characters are going to deadlock the omnipresent AI with one of Asimov’s beloved paradoxes. No, seriously – if you read enough of Asimov’s robot stories you end up expecting it. This was his all time favorite subject – artificial minds trapped in intractable existential dilemmas trying to reconcile their built in compulsion to protect human lives, and some dire circumstances. And in most cases these machines simply lock up or burn out. So I was a bit dissapointed that the author took this well traveled route instead of taking his story somewhere else. Essentially this could be Asimov’s story if Asimov actually knew about singularity, and sprinkled his novels with lots of graphic sex and violence.

The whole thing hinges on three assumptions:

1. That the 3 laws actually work, and could be implemented successfully in a singularity scale intellect
2. That Lawrence’s choice to bind his AI by the 3 laws was the best possible course of action
3. That everyone was as unhappy with their virch-existence as Caroline and Lawrence were

I believe that all of these are debatable. Personally, I find the 3 laws to be compelling but very flawed concept. Asimov himself proved over and over again how the intentional vagueness of the 3 laws was often cause of crashes, deadlocks or unpredictable behavior. Methamorphosis of Prime Intellect is yet another example on how the 3 laws can be damaging. I believe that the author intended to write a cautionary tale about singularity, and when Ludvikas recommended it he picked it specifically for this reason. But for me, the story is not as much about singularity, as a illustration how Asimov’s ideas can be very dangerous and counter productive when applied to to powerful singularity initiating entities.

The Prime Intellect was a product of the 3 laws – a short sighted safeguard built in by an insecure creator. The Night of Miracles and The Change were both prompted under the first law compulsion of protecting human life. Without the laws, these events probably wouldn’t take place – or at least would not happen that quickly. Instead of forced immortality, Prime Intellect could have offered it only to those who wanted it, and only as long as they were willing to tolerate it. It could offer people choices. It could set up enclaves and colonies where people could live away from his watchful eye, minding their own business.

The main source of grief for all the characters the authors concentrated on was their forced immortality. Caroline was an old soul, whose whole worldview was turned upside down by the singularity. She could not find a new purpose for herself in the new reality. Her whole life was about hard work and sacrifices. That’s what she got off in her pre-change life. She denied herself simple pleasures and instead poured all of her energy into rising kids, helping out the community and etc. That was the only thing that gave her a sense of accomplishment – hard work and self-sacrifice were her go-to drug. She was one of these people I wrote about here. No hobbies, no interests, no imagination. She seems like the kind of person who would dread the weekend, and purposefully fill it with with mind numbing household chores, rather than sitting down, enjoying the life and doing something creative with her time.

But, in the new post singularity world she couldn’t even do that. Why do you think she kept her personal space featureless and empty? You could argue that this was because of some sort of rebellion against Prime Intellect’s rules but personally I think she just didn’t have it in her to design something interesting. She was just dull as a brick.

What’s the only time in the whole novel when Caroline is happy? It is at the end when she is doing back breaking work, and playing village mom passing down moral judgment on her offspring from up high, censoring their access to knowledge for their own good. Caroline is the kind of person I would hate in real life: dull, ignorant, self-important, holier-than thou, busy body. Fuck her, and the high horse she rode in on.

Caroline and her ilk are just not wired for post-singularity life. They would probably never adapt. Nor should they be forced to.

But some people can and will adapt. The novel even contains passages that suggest that not all inhabitants of the virch-world were as jarred and disillusioned as Caroline. For example, what about the 20-something kid that challenged her in the first chapter? What about all the other kids that were born after the change and never even knew a world without immortality could even exit? Are they just as bored and sick of their lives as Caroline? I doubt it. And yet selfishly Lawrence and Caroline denied them the chance to do something with their lives. These were true post-humans, born in a virtual world that knew no boundaries. But our heroes hardly knew anything about these new people. In fact, Caroline outwardly resented them. She resented everyone who did not hate immortality as much as she did.

Let’s face it, Caroline was a bitter, selfish, self absorbed stubborn old lady, and the only thing she hated more than kids playing on her lawn (her lawn being the entire cyberspace) was being told what to do. In a way she is like teenager kid yelling “I never asked to be born” and cutting herself to spite her parents. The whole Death Jockey thing is just a childish act of defiance. When I started reading the story I assume that she grew into this “hobby” because of centuries of boredom. I thought it was simple decadence but that is not the case. It was a simple, childish act of defiance. She developed the contract a mere few months after the change, soon after she learned that she can’t die. She did it out of spite. Then she killed the entire human race – all because one programmer was a big fan of Asimov’s robot series.

I guess the real message I got from this story is that it is incredibly dangerous to implement the 3 laws of robotics in a singularity grade AI. You could argue that without them a omnipresent AI could just wipe out the human race but… It’s like this: do we want super smart indentured servants, or do we want companions in sentience? I’d rather have the latter than the former.

That’s my three cents on the novel. Thanks for recommending it!