Optimize Literally Everything

The strange, vast thoughts of Eliezer Yudkowsky

Realistic Villains and Viewpoints

Methods of Rationality is sometimes misinterpreted as invoking Gray Vs. Grey Morality.

This flabbergasted me the first time I read it, and I still don’t really get how anyone could read HPMOR and think that. Dementors are pure evil. Phoenixes are pure good. I don’t think it will spoil anything about HPMOR to reveal that I consider Amycus Carrow and Professor McGonagall to be around as far apart along that axis as human beings can be. Perhaps there is no pure white and pure black in this world, but that doesn’t mean all grays are the same shade.

But when we plunge into Draco’s point-of-view, we get the world the way Draco sees it, with all the moral arguments aligned in favor of the Death Eaters, phrased in ways that sound persuasive to Draco Malfoy. Only the villains in children’s books would phrase their words to make the hero sound more justified, and this is true no matter how justified the hero may actually be.

When Lucius Malfoy is out in public he acts like a stern, honorable politician, undertaking the thankless task of protecting the naive public from powerful and charismatic cult leaders like Dumbledore… because that is an obvious story that realistic!Lucius would automatically seize upon, not because HPMOR is putting Lucius Malfoy on a moral level with Alice Longbottom.

But then since in Lucius’s world he’s not supposed to be the villain, why wouldn’t he be allowed to love his son?  In Lucius’s mental world, he’s never received his Villain Letter to tell him that he’s the bad guy; so Lucius thinks he is supposed to have courage, honor, and the other noble attributes of a Most Ancient House, and he has raised his son accordingly.

Maybe someone could mistake that for Gray vs. Grey morality, if they’re accustomed to Tolkien heroes fighting orcs and Sauron, or if they don’t realize how little it means for a character to think they’re justified - how little that tells you about their position on the good-evil axis.  Adolf Hitler was an outspoken opponent of animal vivisection, who at some dinners would give graphic accounts of animal cruelty in an attempt to convince those present to not eat meat.  Apparently Hitler also didn’t get his Villain Letter.  He probably even wore clothes that weren’t black.  That’s the difference between the children’s-book Voldemort, and Adolf Hitler in real life.

Of course it’s not just villains who try to justify themselves.  Self-justification is cheap, and any character who is even slightly clever will be able to purchase it by the truckload.  A large part of the art of rationality is learning to make self-justifications more expensive and difficult to purchase.  Any character who is not being depicted as a master-level rationalist should have no trouble at all coming up with a story that makes them the good guy, irrespective of what they’re currently doing.

Of course this does require that you be able to cut yourself loose from the moorings of your own mental universe and imagine the way things look to someone else, which is also a key rationalist skill

The economist Bryan Caplan invented an improved version of steelmanning called the Ideological Turing Test. In the Ideological Turing Test, you must write an argument for an opposing position which is realistic enough that an adherent of the position cannot tell the difference between what you have written, and something that was written by an actual advocate. The Ideological Turing Test is stricter than ‘steelmanning’, since it is far too easy to persuade yourself that you have generated the ‘strongest argument’, and much less easy to fool someone who actually believes the opposing position into thinking that you were sincerely doing your best to advocate it.  It is a test of understanding; a trial to make sure you really understand the arguments you say you don’t believe.

People fail the Ideological Turing Test, again, because they’re attached to their own mental moorings, because they fear the violation of letting themselves see the universe from another viewpoint, because they plain lack practice at imagining that another viewpoint might also think itself justified.

Methods!Harry tries to teach this very skill to Draco in Ch. 23, who’s having the standard problem with wielding it:

Even having seen the point, Draco hadn’t been able to invent any “plausible alternatives”, as Harry Potter put it, to the idea that wizards were getting less powerful because they were mixing their blood with mud. It was too obviously true.

It was then that Harry Potter had said, rather frustrated, that he couldn’t imagine Draco was really this bad at considering different viewpoints, surely there’d been Death Eaters who’d posed as enemies of blood purism and had come up with much more plausible-sounding arguments against their own side than Draco was offering. If Draco had been trying to pose as a member of Dumbledore’s faction, and come up with the house elf hypothesis, he wouldn’t have fooled anyone for a second.

When I’m writing HPMOR’s Death Eaters, I’m trying to pass the Ideological Turing Test for Death Eaters—-when I write Draco Malfoy’s viewpoint, I’m writing about Death Eaters the way that Draco Malfoy would see them. The goal is that a real Death Eater would read my Draco Malfoy viewpoint and not say, “Aha! This was clearly written not by the real Draco Malfoy, but by someone who wanted to make Death Eaters look bad.” (Except, of course, that Draco Malfoy’s thoughts are inwardly optimized to look good to his friends and social circle, not to look good to pro-Enlightenment Muggle readers.)

Professor Quirrell is being written with a goal of making sure that the real Professor Quirrell wouldn’t be able to point to one of his lines and say, “What? I wouldn’t say that. There are much more persuasive arguments for a nation that stands strong under a strong ruler, like—-“

I’m often nervous about how many people say they find Professor Quirrell to be extremely persuasive, when his viewpoints are not things I feel undecided about (to put it mildly), but at least it shows I’m doing my job right.

There is a saying that everyone believes themselves to be the hero of their own story. This is not even close to being literally true; so far as I can tell a majority of the world consists of people who explicitly believe themselves to be NPCs, and are viscerally shocked and disbelieving at any suggestion that it could be possible for them to involve themselves in the story’s plot. There are also people who believe themselves to be the antiheroes or outright villains of their own stories. But these are exceptions, especially on a literary level; as a first approximation, most active characters in a story should believe themselves to be the hero.

Similarly, every active character should also live in a mental world that has their own existence at the center, not your protagonist’s. When you’re writing Bob’s viewpoint, every object should be mentioned as often as it is relevant to Bob. Ron Weasley barely exists at all in Harry’s world; but as soon as we switch to Hermione’s world, Ron exists again. Draco sees all things as they relate to Draco; Professor McGonagall sees all things as they relate to Hogwarts.  Dumbledore is still constantly thinking about the huge events with Grindelwald that dominated the first part of his life.  To pass Daphne Greengrass’s Turing Test, I have to write her viewpoint in such a way that someone reading her thoughts can’t tell that anyone else (like Harry Potter) is the real center of the universe.

(Also on the same theme: every agenty character is the [memetic hazard warning: TVTropes] Only Sane Person [/hazard]. Here’s a partial list of HPMOR characters who are depicted as being the only sane person:  Harry, Hermione, Professor McGonagall, Professor Quirrell, Draco, Neville, Daphne Greengrass, Susan Bones, Mad-Eye Moody, Amelia Bones, and Fawkes.)

What we really believe doesn’t feel to us like a belief, it feels like the way the world is.  Really believing that the sky is blue doesn’t feel like being a Blue-Skyer, it feels like the sky being blue.

The process of creating and becoming a character isn’t just the invention of a personality.  It’s the extrapolation of the universe that is that character’s mental world - not what they ‘believe’, but the surrounding universe that this viewpoint will appear to live in.

(Discussion.)