Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Are Your Enemies Innately Evil?

88 Post author: Eliezer_Yudkowsky 26 June 2007 09:13PM

Followup to:  Correspondence Bias

As previously discussed, we see far too direct a correspondence between others' actions and their inherent dispositions.  We see unusual dispositions that exactly match the unusual behavior, rather than asking after real situations or imagined situations that could explain the behavior.  We hypothesize mutants.

When someone actually offends us—commits an action of which we (rightly or wrongly) disapprove—then, I observe, the correspondence bias redoubles.  There seems to be a very strong tendency to blame evil deeds on the Enemy's mutant, evil disposition.  Not as a moral point, but as a strict question of prior probability, we should ask what the Enemy might believe about their situation which would reduce the seeming bizarrity of their behavior.  This would allow us to hypothesize a less exceptional disposition, and thereby shoulder a lesser burden of improbability.

On September 11th, 2001, nineteen Muslim males hijacked four jet airliners in a deliberately suicidal effort to hurt the United States of America.  Now why do you suppose they might have done that?  Because they saw the USA as a beacon of freedom to the world, but were born with a mutant disposition that made them hate freedom?

Realistically, most people don't construct their life stories with themselves as the villains.  Everyone is the hero of their own story.  The Enemy's story, as seen by the Enemy, is not going to make the Enemy look bad.  If you try to construe motivations that would make the Enemy look bad, you'll end up flat wrong about what actually goes on in the Enemy's mind.

But politics is the mind-killer.  Debate is war; arguments are soldiers.  Once you know which side you're on, you must support all arguments of that side, and attack all arguments that appear to favor the opposing side; otherwise it's like stabbing your soldiers in the back.

If the Enemy did have an evil disposition, that would be an argument in favor of your side.  And any argument that favors your side must be supported, no matter how silly—otherwise you're letting up the pressure somewhere on the battlefront.  Everyone strives to outshine their neighbor in patriotic denunciation, and no one dares to contradict.  Soon the Enemy has horns, bat wings, flaming breath, and fangs that drip corrosive venom.  If you deny any aspect of this on merely factual grounds, you are arguing the Enemy's side; you are a traitor.  Very few people will understand that you aren't defending the Enemy, just defending the truth.

If it took a mutant to do monstrous things, the history of the human species would look very different.  Mutants would be rare.

Or maybe the fear is that understanding will lead to forgiveness.  It's easier to shoot down evil mutants.  It is a more inspiring battle cry to scream, "Die, vicious scum!" instead of "Die, people who could have been just like me but grew up in a different environment!"  You might feel guilty killing people who weren't pure darkness.

This looks to me like the deep-seated yearning for a one-sided policy debate in which the best policy has no drawbacks.  If an army is crossing the border or a lunatic is coming at you with a knife, the policy alternatives are (a) defend yourself (b) lie down and die.  If you defend yourself, you may have to kill.  If you kill someone who could, in another world, have been your friend, that is a tragedy.  And it is a tragedy.  The other option, lying down and dying, is also a tragedy.  Why must there be a non-tragic option?  Who says that the best policy available must have no downside?  If someone has to die, it may as well be the initiator of force, to discourage future violence and thereby minimize the total sum of death.

If the Enemy has an average disposition, and is acting from beliefs about their situation that would make violence a typically human response, then that doesn't mean their beliefs are factually accurate.  It doesn't mean they're justified.  It means you'll have to shoot down someone who is the hero of their own story, and in their novel the protagonist will die on page 80.  That is a tragedy, but it is better than the alternative tragedy.  It is the choice that every police officer makes, every day, to keep our neat little worlds from dissolving into chaos.

When you accurately estimate the Enemy's psychology—when you know what is really in the Enemy's mind—that knowledge won't feel like landing a delicious punch on the opposing side.  It won't give you a warm feeling of righteous indignation.  It won't make you feel good about yourself.  If your estimate makes you feel unbearably sad, you may be seeing the world as it really is.  More rarely, an accurate estimate may send shivers of serious horror down your spine, as when dealing with true psychopaths, or neurologically intact people with beliefs that have utterly destroyed their sanity (Scientologists or Jesus Camp).

So let's come right out and say it—the 9/11 hijackers weren't evil mutants.  They did not hate freedom.  They, too, were the heroes of their own stories, and they died for what they believed was right—truth, justice, and the Islamic way.  If the hijackers saw themselves that way, it doesn't mean their beliefs were true.  If the hijackers saw themselves that way, it doesn't mean that we have to agree that what they did was justified.  If the hijackers saw themselves that way, it doesn't mean that the passengers of United Flight 93 should have stood aside and let it happen.  It does mean that in another world, if they had been raised in a different environment, those hijackers might have been police officers.  And that is indeed a tragedy.  Welcome to Earth.

 

Part of the Politics Is the Mind-Killer subsequence of How To Actually Change Your Mind

Next post: "The Robbers Cave Experiment"

Previous post: "Correspondence Bias"

Comments (133)

Sort By: Old
Comment author: Hopefully_Anonymous2 26 June 2007 11:01:56PM 5 points [-]

An interesting (and in my opinion daring) point, Eliezer, although I'm not sure if it's true or not, because I'm not sure about the degree to which genetics, etc. plays a role in creating "evil mutants". After all, people who commit 9/11 type acts ARE rare. The 9/11 participants in my understanding included people with masters degrees and people with long periods of exposure to the West, and that even enjoyed Western comforts immediately prior to their act. I'm not sure if they're representative of "muslim males" as much as they're representative of people that belong to death cults. Just because they're widely admired in some parts of the world doesn't mean that they'd have many imitators. It defies most forms of "selfish gene" logic to kill onesself prior to procreating, particularly if one is a young healthy male. I do think it's possible that the actual 9/11 participants were deviant in all sorts of ways, rather than representatives of people that grow up culturally non-western and muslim rather than culturally western (muslim or not). However, I think you still make great points about the not-always-utilitarian human bias of picking a side and then supporting all of its arguments, rather than focusing on what mix of policy is actually best.

Comment author: [deleted] 19 July 2009 07:18:31AM 10 points [-]

A single generation of mutation could not create an effect as specific as "die for something". Especially not frequently enough for nineteen of them to emerge closely enough to cooperate.

Comment author: NancyLebovitz 15 August 2010 11:30:34PM 12 points [-]

From what I can gather suicide bombers and the like are pretty normal people. Part of what makes normal people normal is that they're relatively easy to influence.

If you want to find something like evil mutants, try looking at those who recruit suicide bombers. On the other hand, it's probably harder to study them, and even they may not be as alien as we hope.

Comment author: JoshuaZ 15 August 2010 11:44:12PM 15 points [-]

From what I can gather suicide bombers and the like are pretty normal people. Part of what makes normal people normal is that they're relatively easy to influence.

Well, suicide bombers are more likely to have engineering degrees than the general public. There's also some evidence that engineers are surprisingly likely to be creationists. I don't think engineers are evil mutants, but it does suggest that there are certain modes of thinking that are likely to have bad results. To repeat fairly standard speculation in this regard, engineers aren't taught critical thinking and are taught to not tolerate uncertainty. This is not a good combination.

Comment author: simplicio 16 August 2010 12:09:10AM 23 points [-]

To repeat fairly standard speculation in this regard, engineers aren't taught critical thinking and are taught to not tolerate uncertainty. This is not a good combination.

Full disclosure: I am (almost) an engineer.

I don't think that's quite correct (uncertainty is a huge concern of engineers), although it's getting there. I would speculate as follows:

  • We know a lot of science, but it's mostly divorced from its epistemic basis. We don't know how we know.
  • We have just enough "science cred" to feel entitled to have opinions on any & all scientific issues, but are probably not actually educated outside a small area.
Comment author: NancyLebovitz 16 August 2010 08:15:57AM 3 points [-]

Something I've wondered about in re the high proportion of engineers among suicide bombers-- I'd have thought that engineers would be last people in the world to think that you can improve things by giving them a good hard kick. Any theories about what I'm missing?

Comment author: Mitchell_Porter 16 August 2010 08:36:31AM 21 points [-]

It's not that Muslim engineers have a special tendency to become jihadis. But engineers do stuff. They solve problems, they act. So when an engineer does join the jihad, they won't be half-hearted about it, and they'll probably be good at it. And in this regard, the jihad is exactly the same as all modern war: educated people who know something of physics and problem-solving always play a large role. That's my theory.

Comment author: gwern 16 August 2010 09:09:37AM 5 points [-]

http://www.slate.com/id/2240157

Another possible explanation would be that engineers possess technical skills and architectural know-how that makes them attractive recruits for terrorist organizations. But the recent study found that engineers are just as likely to hold leadership roles within these organizations as they are to be working hands-on with explosives. In any case, their technical expertise may not be that useful, since most of the methods employed in terrorist attacks are rudimentary. It's true that eight of the 25 hijackers on 9/11 were engineers, but it was their experience with box cutters and flight school, not fancy degrees, that counted in the end.

Comment author: NancyLebovitz 16 August 2010 02:20:19PM 5 points [-]

And if someone is good at making bombs (which is the role I would have expected for engineers) that's precisely the sort of person a terrorist organization wouldn't want to die.

I think.

One thing I've noticed is that everyone (ok, some huge proportion of people) thinks they're an expert on how to do effective terrorism.

Comment author: gwern 16 August 2010 02:52:45PM *  9 points [-]
Comment author: Ender 18 June 2012 03:42:40PM 6 points [-]

Making perfect, evil plots can be a great conversation starter.

Comment author: gwern 26 May 2011 05:15:25PM 2 points [-]
Comment author: NancyLebovitz 16 August 2010 02:24:20PM 2 points [-]

It's not that Muslim engineers have a special tendency to become jihadis. But engineers do stuff. They solve problems, they act.

My impression of engineers is that they're more apt than the general population to invent, tinker, and adjust, but this is specifically about the sort of physical stuff where they have some knowledge. They aren't especially apt to go into politics.

Comment author: David_Gerard 17 January 2011 12:53:29PM *  1 point [-]

This doesn't stop them. c.f. the nascent RW article on engineers and woo. Stereotypical engineer arrogance comes from assuming one's tested competence in one's chosen field carries through to fields outside one's tested competence. Engineers can get away with all manner of gibbering delusion as long as the stuff they design still works.

Comment author: wedrifid 17 January 2011 01:14:55PM 4 points [-]

Stereotypical engineer arrogance comes from assuming one's tested competence in one's chosen field carries through to fields outside one's tested competence. Engineers can get away with all manner of gibbering delusion as long as the stuff they design still works.

I'd believe that. Systematic overconfidence when it comes to things outside of their field is ubiquitous across experts of nearly every kind. Experts also systematically overestimate the extent to which their expertise happens to be relevant to a given context. (Thankyou Ericsson).

I've got a sneaking suspicion that another factor that contributes to said stereotypical arrogance in engineers is their relatively weak social competence (and orientation) compared to others of equivalent levels of skill. Most of what makes use judge others as arrogant seems to be the force with which they present their position as compared to the level of status that we believe it appropriate for them to claim. Insufficient submission to the social reality makes being perceived as arrogant nearly inevitable. Engineers (and other nerds) tend to do that more due to attitude, ability or a little of both.

Comment author: salymnr 07 September 2012 03:52:12AM 4 points [-]

Working engineer here. A lot of social ineptitude throughout the industry, at least with respect to interaction with non engineers. Certainly can't help the arrogance thing.

However I think assumed confidence outside of one's field is a result of what an engineer should be. Engineers solve problems with incomplete information regularly. So when approaching a field he knows little about, an engineer will not hesitate to be as confident as he is in his normal field. Business as usual. I don't think it would be hard for that to come across as arrogant.

Comment author: Perplexed 17 January 2011 01:17:20PM 0 points [-]

My impression is that we tend not to participate in the political structures and organizations that non-engineers set up. We are more likely to try to reinvent the whole concept of political action. And, in doing so, to draw our inspiration from science fiction.

Comment author: Vladimir_M 16 August 2010 12:40:49AM *  4 points [-]

JoshuaZ:

To repeat fairly standard speculation in this regard, engineers aren't taught critical thinking and are taught to not tolerate uncertainty.

I would be really curious to see the evidence you have for this latter claim. Could you give some concrete examples from engineering education or actual practice where, according to you, intolerance of uncertainty is taken to unsound extremes?

As for "critical thinking," well, that's a highly subjective category. Where you see a scandalous failure of critical thinking, someone else might see a relatively insignificant and excusable human error, and vice versa, even if you're both in complete agreement that the belief in question is factually false.

But in any case, could you point out an example of some actual educational program that teaches critical thinking in ways that engineers supposedly miss? I honestly can't think of what exactly you might have in mind here.

Comment author: JoshuaZ 16 August 2010 12:43:59AM 2 points [-]

It isn't a great way of phrasing things and may just be wrong. Simplicio's description seems like a better guess for what is going on. The article I linked to also suggests a few other possibilities.

Comment author: NancyLebovitz 16 August 2010 08:11:44AM 1 point [-]

I said suicide bombers seem to be normal people, not that they seem to be typical people.

Comment author: stcredzero 20 May 2012 09:44:41PM 1 point [-]

There's also some evidence that engineers are surprisingly likely to be creationists.

Also to be libertarians, I suspect.

Comment author: JohnDavidBustard 26 August 2010 04:07:47PM 5 points [-]

This may be entering into dangerous territory but to what extent does the psychology of a suicide bomber differ from that of say a first world war soldier.

In both cases their death is guaranteed, and in both cases they view the justification as being the protection of their community. Would the outcome of losing such a war be bad enough to justify most men risking their lives? Perhaps what is strange is having a society where killing yourself for a cause is rare?

Comment author: thomblake 26 August 2010 04:19:00PM 4 points [-]

In both cases their death is guaranteed

Not so. The soldier can say, there's still a chance (and indeed, many soldiers survived), but the suicide bomber likely doesn't have that option.

Comment author: NickiH 03 April 2011 03:48:45PM *  10 points [-]

Soldier: The government told me to. They've been elected by us, so they must be right, yeah? Everyone else is doing it - think how my friends would look down on me if I said no! I'm going to be a hero! Heroes get all the girls.

Bomber: My God told me to... can't argue with God, right? My friends are doing it - I don't want to look like a coward! Mmm, virgins. (Or other heavenly reward of choice).

Hmmm... that was originally going to be a list of differences in their viewpoints, but the more I think about it, the more similar they appear. Now I'm not sure what I think any more!

Comment author: TheOtherDave 03 April 2011 04:55:33PM 2 points [-]

Well, one salient difference might have to do with comparing the available mechanisms for calibrating my confidence in the judgment of a government with those for calibrating my confidence in the judgment of a god.

Comment author: NickiH 04 April 2011 08:17:16PM 3 points [-]

Given that people who believe in god tend to really believe in god, and people who trust governments do so usually with a number of reservations, does that mean that the bomber has more justification than the soldier?

Comment author: TheOtherDave 04 April 2011 10:00:31PM 2 points [-]

No. Why would it?

Justification for an act is not something that emerges full-blown out of nothing. My act cannot be justified by of my faith in X if that faith is itself unjustified.

And if I have faith in X within certain constraints and with certain reservations (as I do with governments, for example), that doesn't somehow make that faith less justified than if I "_really believe in" X without constraints or reservations.

And all of that is true whether X is my government, my god, or my grandmother.

Comment author: NickiH 05 April 2011 08:01:58PM 2 points [-]

From the point of view of the bomber, faith in God is not itself unjustified. It is in fact a vital part of his psychology.

The original point was the difference in the psychologies of bombers and soldiers. They are both doing it because they were told to, but their confidence in the judgement of the one telling them to is different. So the one with the higher confidence feels more "justified". That's what I thought you meant, anyway. If it's not, could you please clarify?

Perhaps I should have said "the bomber thinks he has more justification than the soldier".

Comment author: TheOtherDave 05 April 2011 09:03:54PM 3 points [-]

Ah, I see.

If "justification" refers to a feeling, then sure: the person who is really convinced that X is reliable and wants them to do something has more justification for doing that thing than the person who isn't quite sure that X is reliable, or isn't quite sure that X wants them to do it. (Again, whether X is a government, a god, or a grandmother.)

I was thrown off because "justification" in other contexts is often used to mean something different.

Which is fine; I don't mean to turn this into a discussion about the meaning of a word.

Sorry to cause confusion; thanks for clarifying.

Comment author: Biophile 05 October 2012 10:33:48PM *  1 point [-]

Do people who believe in God tend to really believe in God?

Comment author: TheOtherDave 05 October 2012 11:41:20PM 2 points [-]

That's a hard question to answer without defining the terms better.

I grew up among a lot of self-identified religious people. Using as my test for the left-side "believe in God" the willingness to arrange at least some superficial aspects of one's life around those beliefs (e.g., where one lives, sends children to school, eats, etc.), and using as my test for the right-side "believe in God" the willingness to die rather than violate what they understood to be God's law, I'd say I'm .95 confident that fewer than five percent of the folks with LH beliefs had RH beliefs, and .75 confident that fewer than 1one percent did.

Comment author: BlueAjah 12 January 2013 04:11:26PM 1 point [-]

Yes, but dying is against God's law... so they've cleverly got around that problem.

Comment author: BerryPick6 12 January 2013 04:25:29PM 3 points [-]

Not true for every religion.

Judaism has certain specific instances where it is accepted that it would be better for one to die than commit a sin.

Also, martyrdom would not be such a large aspect in Christianity (or, at least, in early Christianity) if dying for God wasn't considered a good thing.

Comment author: TheOtherDave 12 January 2013 09:14:11PM 0 points [-]

Yes, precisely this.

Comment author: [deleted] 03 April 2011 07:08:00PM *  3 points [-]

There are other differences aside from whether or not the attacker will survive. The 9/11 attacks are often compared to the attack on Pearl Harbor. One difference stands out: the 9/11 attacks included attacks on two large buildings packed with thousands of innocent civilians, with no obvious connection to any military installation. This is in contrast with the Japanese attack on Pearl Harbor, which was an attack on a military installation. What the typical WWI soldier did was a lot more like what the Japanese did at Pearl Harbor - i.e., they attacked the enemy's military - and a lot less like what the 9/11 attackers did.

An additional point is that focusing on the suicide bombers may be too narrow a focus. The suicide bombers were, after all, not the only Islamist fighters who targeted innocent civilians for murder. The murder of Daniel Pearl was another deliberate murder of a civilian, not as an unavoidable casualty of an attack on a military base but as the sole target of the murder. As I recall from the murder video that was released, the murderers made a point of mentioning that Pearl was a Jew. The obvious moral comparison here is not to anyone in WWI, nor to the allies in WWII, nor to most of the Axis in WWII, but specifically to the Nazi military who were engaged in a program of the deliberate extermination of the Jewish people.

Comment author: bgaesop 05 April 2011 09:53:01PM 5 points [-]

One difference stands out: the 9/11 attacks included attacks on two large buildings packed with thousands of innocent civilians, with no obvious connection to any military installation

The 9/11 hijackers would no doubt not refer to the inhabitants of the World Trade Center as innocent civilians, but as economic oppressors. There is a reason they targeted both the Pentagon and the World Trade Center, after all.

Comment author: [deleted] 05 April 2011 10:48:29PM 0 points [-]

Yes, obviously the hijackers did indeed see the people in the WTC as sufficiently similar to enemy soldiers to constitute a legitimate target for attack. It is just as you say. But this very fact I think reveals a psychological gulf that lies between them and the WWI soldier on either side - which was what was asked about.

How we classify and identify things can itself be a significant fact about our psychology. A stereotypical example of someone who is mentally abnormal is someone who non-jokingly identifies himself as Napoleon or Jesus.

Comment author: bgaesop 18 April 2011 12:12:25AM 10 points [-]

Sorry for responding so late, but do you really think that this thought:

"My people are being oppressed, primarily economically. I can see that it is mostly Americans doing this. Peaceful protest tends to get me shot at. Clearly these Americans consider their profits more important than my and my people's lives; their actions are causing our suffering and deaths, they are aware of this, yet they continue to do so. Therefore, they are deliberately killing and ravaging my people, and so it is justified for me to kill them. Also, doing so may cause them to strike out in more obvious, militaristic ways, which will weaken their economy (punishing them) and make it more obvious to my fellows that, indeed, America is an extremely evil nation that must be opposed. Better to force them out in the open than let them continue oppressing us by subterfuge. Doing this will be very difficult, and will likely cost me my life, but the organization I just joined has offered to pay a good deal of money to my surviving family when/if I do die, and given that right now they're struggling to buy food because of those fucking Americans and their economic jackassery. Therefore, it is justified and indeed Justice for me to blow up their center of commerce, even at great personal sacrifice."

Is of equivalent sanity to this thought:

"I'm the reincarnation of Napoleon! Hibberty flibberty jibbit!"

Comment author: Jiro 20 July 2014 04:01:32AM 1 point [-]

No, it's the equivalent of trying Sherlock Holmes style reasoning in real life. That's still insane, especially when used to kill people.

Comment author: Eugine_Nier 18 April 2011 01:06:09AM 1 point [-]

The 9/11 hijackers would no doubt not refer to the inhabitants of the World Trade Center as innocent civilians, but as economic oppressors.

I'd think the hijackers would refer to them as infidels.

Piece of advice: just because you see the world in purely Marxist terms, doesn't mean everyone else does.

Comment author: bgaesop 18 April 2011 01:15:06AM 8 points [-]

I'd think the hijackers would refer to them as infidels.

Do you really, truly think that the only motivations in choosing to do an attack against America (heck, picking America as the target in the first place) and picking the WTC and Pentagon as the targets of that attack, was because the attackers were Muslim while the ones being attacked were not? If so, why have they not done similarly to all non-Muslim nations? Why not attack symbols or places of power of religion, rather than economics and the military?

Certainly religion is used as a framing device and recruitment tool; it's a powerful ingroup identifier. Especially when you have people doing the same on the opposite side of your fight.

Piece of advice: just because you see the world in purely Marxist terms, doesn't mean everyone else does.

That's not so much a piece of advice as a snipe at what you perceive to be the dialectic I'm using to interpret this. It seems to me that you didn't say that to enlighten me, but to reduce my status in the eyes of what you (and I) assume is a mostly capitalist readership.

Comment author: Eugine_Nier 18 April 2011 01:39:01AM 4 points [-]

Do you really, truly think that the only motivations in choosing to do an attack against America (heck, picking America as the target in the first place) and picking the WTC and Pentagon as the targets of that attack, was because the attackers were Muslim while the ones being attacked were not?

No, not the only one, but if one were to ask them why they picked the targets they did, they'd describe it religious terms (talking about infidels, jihad and the great Satan) not in Marxist terms (i.e., economic oppression). In fact judging by the fact that most of the hijackers were from wealthy families, I'd guess they didn't really care about the economic dimension except as part of a general attitude that our decadence is sinful and is spreading to the middle east.

Piece of advice: just because you see the world in purely Marxist terms, doesn't mean everyone else does.

That's not so much a piece of advice as a snipe at what you perceive to be the dialectic I'm using to interpret this. It seems to me that you didn't say that to enlighten me, but to reduce my status in the eyes of what you (and I) assume is a mostly capitalist readership.

I stand by my advice as good advice. If you want to successfully model others' behavior, you shouldn't assume they see the world the same way you do.

Comment author: bgaesop 23 April 2011 06:25:32AM *  7 points [-]

No, not the only one, but if one were to ask them why they picked the targets they did, they'd describe it religious terms (talking about infidels, jihad and the great Satan) not in Marxist terms (i.e., economic oppression).

Just as an aside, "economic oppression" isn't a uniquely Marxist term, nor am I even aware of a specific Marxist definition of it. Are you thinking of "economic exploitation", perhaps? The latter means the difference between the amount of wealth generated by labour and the amount that labourer is paid.

I am pretty darn thoroughly convinced (though of course I am open to changing my mind) that the idea "religion made them do it!" is overly simplistic. I used to hold the position you do, but over the course of several years of examining the issue, I have come to the conclusion that the use of religious terminology and phrasing and all the general trappings of Islam are, while perhaps truly believed, are for the most part merely a rhetorical device constructed to take maximum advantage of the society they are recruiting, living, and (typically) acting in. I'm hesitant to say this next sentence, politics being the mind killer and all that, but I shall anyways (I have noticed I am in a hole. Hypothesis: if I dig long enough I'll get to China!). Osama bin Laden talks about "defeating the Great Satan for the glory of Allah and Mohammed (pbuh)" for the same reason George Walker Bush talked about "spreading Freedom and Democracy": because it resonates with his intended audience, convinces them that he has similar thought-processes to them and is representative of their interests, or at the very least their team, not because he (edit: necessarily) believed that that was what he was doing.

In fact judging by the fact that most of the hijackers were from wealthy families, I'd guess they didn't really care about the economic dimension except as part of a general attitude that our decadence is sinful and is spreading to the middle east.

Most people who have had impact in the world have come from wealthy (or at least not working-class-poor) families, including probably every Socialist Revolutionary you've heard of (Marx, Engels, Lenin, Che, et cetera), not to mention almost every politico in general. If anything, being middle class (inasmuch as that term makes sense) makes you more likely to simultaneously see the degradation of the poor and have the education to see what (at least seem to you) like plausible explanations for it. And then if you're an engineer or what have you, you have access to abilities that can actually do something about this (build bombs, fly planes, whatever), or the funds to support yourself while you learn them, or whatever. The point is, being middle class is not likely to make you less politically aware and active than being poor, and it is likely to increase your free time and ability to do things politically, including but not limited to committing acts of terrorism.

I stand by my advice as good advice. If you want to successfully model others' behavior, you shouldn't assume they see the world the same way you do.

When phrased this way it seems much more like actual advice and much less like an insult. I'm not sure how much of this is my inference and how much is your implications, but it's kind of moot. No hard feelings are taken, hopefully none were intended. Friends? I certainly agree that I should not model their minds as being identical to mine, but given that I don't want to kill people, I'm already doing that at least to some degree.

That said, I think that you are being overly simplistic in your model of these people. Again I link to this page. Could you please explain, or link me to someone else who has, what makes you think that your model of their minds and motivations is more accurate than mine?

Comment author: Eugine_Nier 23 April 2011 08:13:44AM *  8 points [-]

Osama bin Laden talks about "defeating the Great Satan for the glory of Allah and Mohammed (pbuh)" for the same reason George Walker Bush talked about "spreading Freedom and Democracy": because it resonates with his intended audience, convinces them that he has similar thought-processes to them and is representative of their interests, or at the very least their team, not because he actually believed that that was what he was doing.

There is a problem with arguments of the form, "The leader of that group clearly doesn't 'really' believe his own rhetoric he's just saying that because it resonates with his followers." This implies that their followers actually believe that stuff, otherwise there would be no point in the leaders' saying it. But you've just admitted that there exist people who really believe that stuff, why is it so absurd for the leader to be one of those people?

I certainly agree that I should not model their minds as being identical to mine, but given that I don't want to kill people, I'm already doing that at least to some degree.

You're still self-anchoring. You observe that they want to kill people, so you try to imagine under what conditions you would be willing to kill people.

That said, I think that you are being overly simplistic in your model of these people.

Well, near as I can tell, your model boils down to "they secretly have to same world-view as I do, and the difference in their rhetoric is because it resonates with their audience".

For the record I should probably mention my model:

They observe that the Islamic world isn't as powerful as it was in its glory days. Furthermore, the West and the United States in particular is influencing their culture in ways they don't like. Solving this problem requires a model of how the world works. Well, the model they turn to is one based on Islam.

There is certainly more that could be added to this model, e.g., a discussion of how feuds work in clan-based societies for starters.

Comment author: wedrifid 23 April 2011 08:32:20AM 5 points [-]

Osama bin Laden talks about "defeating the Great Satan for the glory of Allah and Mohammed (pbuh)" for the same reason George Walker Bush talked about "spreading Freedom and Democracy": because it resonates with his intended audience, convinces them that he has similar thought-processes to them and is representative of their interests, or at the very least their team, not because he actually believed that that was what he was doing.

There is a problem with arguments of the form, "The leader of that group clearly doesn't 'really' believe his own rhetoric he's just saying that because it resonates with his followers." This implies that their followers actually believe that stuff, otherwise there would be no point in the leaders' saying it. But you've just admitted that there exist people who really believe that stuff, why is it so absurd for the leader to be one of those people?

The only part I would leave out of bgaesop's paragraph is the "not because he actually believed that that was what he was doing". All of the previous stuff fits fine when both the leader and the intended audience are sincere homo-hypocrites. That is why he is doing it (or equivalently the fact that they do it so well is what made them the leaders). What they believe about the matter can be orthogonal.

Comment author: bgaesop 23 April 2011 09:21:21AM *  3 points [-]

There is a problem with arguments of the form, "The leader of that group clearly doesn't 'really' believe his own rhetoric he's just saying that because it resonates with his followers." This implies that their followers actually believe that stuff, otherwise there would be no point in the leaders' saying it. But you've just admitted that there exist people who really believe that stuff, why is it so absurd for the leader to be one of those people?

My mistake, wedrifid is correct, I turned my thought into a sentence poorly.

You're still self-anchoring. You observe that they want to kill people, so you try to imagine under what conditions you would be willing to kill people.

I admit to not having considered this bias on this subject. That said, I don't think that this bias is affecting me very significantly here, and I think that because of the direction I approached my current position from: I arrived at it after moving from somewhere near where you are currently. I will consider the possibility that my position is affected by this bias, however. The manner in which I am doing so right now is to reread the wikipedia page that I just linked and follow several of the citations. It seems that the consensus is that perceived western aggression against Muslims and Islam is one of the prime motivators--which would then include what I said, and also perceived aggression against Islam specifically. So a mixture of what we've both been saying.

Well, near as I can tell, your model boils down to "they secretly have to same world-view as I do, and the difference in their rhetoric is because it resonates with their audience".

I don't think that they are attempting to inspire a proletarian revolt across nations. I don't think that they are attempting to engage in a class struggle pitting the poor against the rich. I do think that they perceive themselves and their fellow Muslims as being the victims of exploitation by Westerners, and I think that they perceive a number of dimensions to that exploitation: military, economic, and cultural; perhaps more. Military is fairly obvious. Economic is what I was talking about, I mentioned it specifically because we were discussing the attacks on the World Trade Center. Cultural is what you are talking about. I believe that while it is an important portion of their motivation, it is not the primary piece. Unfortunately their rhetoric focuses on that issue largely (though by no means entirely) which gives an inflated view of its importance.

They observe that the Islamic world isn't as powerful as it was in its glory days. Furthermore, the West and the United States in particular is influencing their culture in ways they don't like. Solving this problem requires a model of how the world works. Well, the model they turn to is one based on Islam.

It might be that we are saying similar things with rather different vocabularies. When you say that the Islamic world isn't as powerful as it was in its glory days, does that include what I talk about when I say they're being economically exploited? For instance, instead of a wealthy semi-equitable (or perhaps merely remembered as such) Caliphate, they are frequently poor or highly segmented populations dependent on natural resource exportation? Where does reaction to the West's military operations fit into your model? That certainly seems to be one of the motivating forces most commonly cited by terrorists themselves.


Out of curiosity, have you been downvoting me? I've been upvoting you. I ask because I notice that every time I post in this thread my karma goes down, and though I do realize it's a silly thing to care about, for some reason I do. Something about human brains enjoying watching numbers go up, I suppose. It's particularly frustrating because I am enjoying the discussion, but seeing that number going down makes me feel like my participation is unwanted (which I am assuming is not the case, but who knows, maybe it is).

Comment author: bgaesop 23 April 2011 10:00:08AM *  0 points [-]

I just remembered the obvious point that I had been forgettig this whole time. Your position seems to me to be basically the position the article we're both commenting on is directly arguing is a silly, untenable one to take.

Comment author: [deleted] 18 April 2011 02:24:50PM 4 points [-]

If so, why have they not done similarly to all non-Muslim nations?

Muslim attacks are a worldwide phenomenon, concentrated in and around the Muslim world. See for example this or this.

Comment author: LCC 20 June 2013 02:31:33PM 0 points [-]

I read somewhere (I wonder if I can still find the source) that the terrorist groups which train the terrorists and provide the logistics etc. reward the family of the attacker generously. The article said that the reward was enough to allow one of the brothers of the attacker to marry – and by extension, to procreate. The "genetic" motive is therefore in my opinion all but irrelevant. That being said, I don't know if that specifically applies to the perpetrators in question.

Comment author: Doug_S.2 26 June 2007 11:30:50PM 4 points [-]

I agree; there may very well be the rare innately evil person, but promoting or implementing an ideology that is based on false premises that turns out to have evil consequences does not require "innate" evil. The 9/11 hijackers might very well be described as "neurologically intact people with beliefs that have utterly destroyed their sanity" but, if the beliefs they had about the state of the world were actually true (which they weren't!) then many value systems would endorse their actions.

If there were a diety that condemns unbelievers to Hell and cannot be caused to do otherwise, it's not hard to argue that it it is morally necessary to kill people who try to persuade people to become unbelievers. Given the existence of such a diety, a utilitarian perspective might easily reduce to something like "Do whatever it takes to minimize the number of souls in Hell."

Comment author: Michael_vassar4 27 June 2007 08:05:52AM 4 points [-]

"Everyone is the hero of their own story." is a popular claim, and may well be true, but I don't know if any evidence for it has been carefully collected.

"The Enemy's story, as seen by the Enemy, is not going to make the Enemy look bad." Some cultures fairly reliably create people who think rap sounds bad, others fairly reliably create people who think rap sounds good. Some cultures fairly reliably create people who think sushi tastes bad, others fairly reliably create people who think sushi tastes good. Some cultures fairly reliably create people who think saying that the best thing in life is "To crush your enemies, see them driven before you, and to hear the lamentation of their women" is bad, others fairly reliably create people who think this, or suttee, or gladiatorial combat, or mortification of the flesh of various types, or honor murder, or military heirarchy, or pederasty or... make people look good.

"If it took a mutant to do monstrous things, the history of the human species would look very different." but if "violence [was] a typically human response" to the global situation as seen by some cultural groups prior to 9/11 then recent history would also look very different. In fact, the number of terrorists in the world is quite small, which I take to be a really really good reason for not killing random Muslims who kind-of-sort-of look the same. Terrorists are much much rarer than Scientologists or Jesus Camp victims, and for this reason one might think them much more likely to "send shivers of serious horror down your spine".

You know that most cops never kill anyone, right?

"in another world, if they had been raised in a different environment" seems like a weird exculpatory justification. One with weirdly much in common with calling a foetus a potential human being, as if almost every human cell did also not contain a recipe for a potential human being and as if every large lump of organic matter didn't have the potential to be one if processed correctly. I mean really, surely we could, if we wanted to, find tissue samples from every victim of violence who we identified and clone them, but would we then rightly feel that the tragedy of them being raised in the wrong environment had been corrected?

Comment author: TGGP3 27 June 2007 09:10:37AM 1 point [-]

A great post, and one of the reasons I promote emotivism. I attribute a recent dissagreement (in which I admit I acted like a dick) to just this. The funny thing is that usually two people argue with each other, convinced the other is evil. In this case I am arguing with someone over just how scary some other people that we both don't care for are.

Comment author: Peter_McCluskey 27 June 2007 07:24:58PM 1 point [-]

It appears counterproductive to use the word mutants to describe how people think of enemies. Most people can easily deny that they've done that, and therefore conclude they don't need to learn from your advice. I think if you were really trying to understand those who accept misleading stereotypes of suicide bombers, you'd see that their stereotype is more like "people who are gullible enough to be brainwashed by the Koran". People using such stereotypes should be encouraged to think about how many people believe themselves to be better than average at overcoming brainwashing.

And for those who think suicide bombers are unusual deviants, I suggest reading Robert Pape's book Dying to Win.

Comment author: TGGP3 27 June 2007 08:04:26PM 0 points [-]

This post starts off talking about school shooters, but I think it could be applied to terrorists as well, although they have a movement and ideology behind them.

Comment author: Asad_Quraishi 29 June 2007 02:05:22PM -1 points [-]

It is interesting that you talk about the "nineteen [who] hijacked four jet airliners in a deliberately suicidal effort to hurt the United States of America". Your article starting in this vain gave me hope. Alas, it was not to be. I have not read all of your posts nor followed all of your links so I realize I have missed many of your thoughts and probably have an incomplete picture, so take what I'll say and put it in that context. If not, then your lack of expansion of this idea to its logical conclusion that these individuals did not typify [virtually any] Muslims and in fact were, in 2001, an incredibly small minority of the Muslim population, is a disappointment. Clearly this is the case. It is also true that the extreme reaction, and in fact what was simply a justification for an already-desired action at the time, of the U.S. has resulted in a far more rapid growth in militants than could have been achieved by any approach except perhaps one explicitly designed to achieve such aims. This is sad. It is sad that a powerful nation takes on a weak one - ruled my a madman but populated by you's and I's - crushes it, breaks it, botches putting it back together and then blames the people of this same nation for their 'lack of support'. Blames... actually I'm not sure who is being blamed... for the rise of a new breed of militant. No that's innaccurate - actually the picture that is being painted is one in which all of the current militants/maniacs(and they are) were already there! Hey, I'm all for revenge - go and blast Al Qaeda in Afghanistan and kill a significant number of innocent civilians (can we agree that even one innocent dead child is significant). I can understand this. I am fairly certain that if someone killed just one of my children I would want revenge - and may even, temporarily, willingly, lose my mind in emotion and rage - though I consider myself a highly civilized being. No one should understand US' actions towards Iraq. All of us should pray that Iran isn't next. I grew up in the 70s and early eighties when the cold war was still going strong and total annihilation seemed a damn good possibility. I thought all of that was over and done with and the world was free to spread its wings and explore as-yet unexplored avenues of humanity. We just might be seeing the creation of a new war, not as cold as the last, and not as predictable or stable. I hope not.

Forgive my minor rant. Someone with a name like mine rarely dares to speak up in this way. While I agreed with the sentiment of your article what remained unsaid screamed at me and I could not, this time, shut it out.

Comment author: ChristianKl 24 May 2012 11:35:00AM 5 points [-]

Eliezer just used 9/11 as an example. His post isn't supposed to say all there is to say about 9/11. It's supposed to make a point about seeing the humanness in people who do bad deeds.

No one should understand US' actions towards Iraq.

Not understanding actions doesn't help.

All of us should pray that Iran isn't next.

Praying doesn't help either. It's rather important to understand why the world is the way it is. If a significant amount of people in a democracy understand the way the world works you get positive political change.

Comment author: TGGP3 23 July 2007 07:35:26PM 3 points [-]

Some Dude, since when is war profitable? It can be extremely expensive, and you can't really have both sides win, yet it is often the case that both sides are eager for it.

Comment author: bgaesop 05 April 2011 09:59:28PM 8 points [-]

Some Dude, since when is war profitable?

Since there existed private military contractors, or before that, since there existed spoils of war?

Comment author: Michael_Zeleny 07 August 2008 07:44:30PM 0 points [-]

An accurate estimate of anyone else’s psychology is a dubious benefit in strategic interactions that depend solely on being able to predict the actions of friend and foe.

In Proposition XXXVII of Part IV of the Ethics, Benedict Spinoza asserts that the good that every man who follows after virtue wants for himself, he also desires for other men; and this Desire is greater as his knowledge of God is greater. After proving his claim, Spinoza observes that the law against killing animals is based more on vain superstition and womanish pity than on sound reason (legem illam de non mactandis brutis magis vana superstitione et muliebri misericordia quam sana ratione fundatam esse). He continues:

The rational principle of seeking our own advantage teaches us the necessity of joining with men, but not with beasts, or with things whose nature is different from human nature; we have the same rights against them as they have against us. Indeed, because the right of each one is defined by his virtue, or power, men have a far greater right against beasts than beasts have against men. Not that I deny that beasts feel. But I do deny that we are therefore not permitted to consider our own advantage, use them at our pleasure, and treat them as is most convenient for us. For they do not agree in nature with us, and their affects are different in nature from human affects.

By parity of reasoning, the rational principle of seeking our own advantage allows us to use our enemies at our pleasure, and treat them as is most convenient for us. For our civic nature is defined by the constitution of our state; and to the extent that foreign subjects do not agree in nature with us, and their affects are different in nature from our affects, we would be ill served by extending our habitual notions of humanity, formed through intercourse with our compatriots, to anyone that does not partake of our social compact. To this effect, in setting our foreign policies for war and peace, we must form alliances without affection and prevail in conflicts without hatred. Erring in either direction can only deter us from attaining our goals.

Comment author: timtyler 16 July 2009 07:01:47PM 0 points [-]

What do you mean by "evil"?

Comment author: ABranco 16 October 2009 04:59:37AM *  3 points [-]

I've come to think that way years ago, and am happy to read such a clear exposition on the point.

What's ironic is that the first time the issue actually called my attention was while reading the famous How to Win Friends and Influence People. Now, the book is about winning, not truth, and the point of the chapter was saying to people "If I were in your position, I would have done the same thing" as a strategy to win their sympathy. Functional and teleological friend-winning strategy, sure. But when afterward Carnegie made the point that saying this wasn't a lie — since if you were born in the same conditions and had had the same life as the other, you'd be just like him —, I couldn't argue against it. So, surprisingly, the winning strategy did have some philosophical depth to back it up.

The kinda cheap, new age self-help book The Four Agreements also got it right with his "Don't Take Anything Personally: Nothing others do is because of you. What others say and do is a projection of their own reality, their own dream."

Don't read the book, though: the twisted reasoning(?) that got the author to this rather sensible conclusion would make you want to scream. Of course, this reaction wouldn't make much sense, 'cause y'know, if you were a 57-year-old Mexican shaman with his exact same genetics and life experiences...'

Comment author: ciphergoth 05 January 2010 11:03:31AM *  1 point [-]

Nothing others do is because of you.

This seems trivially false - clearly the actions of others are influenced by my existence and the things I do in myriad ways. What non-trivially-false thing do you mean to say by it?

Comment author: ABranco 07 January 2010 01:05:57PM 1 point [-]

Influenced, yes.

But not 'because' of you. It's not that personal. The other person carries all her dreams, histories, frustrations, hormones, cognitive biases and -- which is relevant -- a rather inaccurate map of the very territory that you are. Their response is more like an effect of that mix, in which, what concerns you, only a partial map of who you are play a role.

Comment author: ciphergoth 07 January 2010 01:59:30PM 1 point [-]

That seems to me a very different sentiment from the one you quoted!

Comment author: Rain 05 May 2010 05:32:19PM *  17 points [-]

This interview with notorious lawyer Jacques Verges in Spiegel deals with the question of evil in a very thought provoking way.

SPIEGEL: Mr. Verges, are you attracted to evil?

Jacques Verges: Nature is wild, unpredictable and senselessly gruesome. What distinguishes human beings from animals is the ability to speak on behalf of evil. Crime is a symbol of our freedom.

SPIEGEL: That's a cynical worldview.

Verges: A realistic one.

SPIEGEL: You have defended some of the worst mass murderers in recent history, and you have been called the "devil's advocate." Why do you feel so drawn to clients like Carlos and Klaus Barbie?

Verges: I believe that everyone, no matter what he may have done, has the right to a fair trial. The public is always quick to assign the label of "monster." But monsters do not exist, just as there is no such thing as absolute evil. My clients are human beings, people with two eyes, two hands, a gender and emotions. That's what makes them so sinister.

SPIEGEL: What do you mean?

Verges: What was so shocking about Hitler the "monster" was that he loved his dog so much and kissed the hands of his secretaries -- as we know from the literature of the Third Reich and the film "Der Untergang" ("Downfall"). The interesting thing about my clients is discovering what brings them to do these horrific things. My ambition is to illuminate the path that led them to commit these acts. A good trial is like a Shakespeare play, a work of art.

Comment author: simplicio 15 August 2010 11:46:08PM 17 points [-]

Another problem with seeing enemies as innately evil is that it lets us off the hook as to our own capacity for evil (so elegantly demonstrated by the Milgram experiments, although I hope I would do better).

I've lost count of how many times I've heard that Hitler or Stalin or whoever was "just evil," or that the holocaust was the result of some essentially German negative personality trait, or that child abusers of various kinds are "just monsters."

To the extent that these statements mean only "Boo Stalin!" or "Boo paedophiles!" I guess they're not so bad, but I think people actually believe them as propositions to some extent. Certainly, if movies are any guide, the bad guys are usually pure evil - for no readily apparent reason, they just love pain and want to blow up the world.

Which is a big problem, because it leads you to be naive about your own propensity. An acquaintance of mine knew a rapist, through work. This rapist was not a slavering beast, he was an ordinary guy (maybe with some nasty explicit or implicit beliefs about women) until he got drunk and raped somebody. I really don't want to say "it could have been me," and I honestly don't think it could have. But I doubt he thought, say a year before, that it could have been him.

Comment author: nick012000 29 September 2010 09:50:37AM 3 points [-]

More likely is that your inaccurate map of the territory of his mind was sufficiently wrong that it fell under the "normal person" category. As someone who has fantasies about that sort of thing (but would hopefully never actually do it), let me tell you that this isn't the sort of thing that comes out of nowhere. Odds are, he knew where his proclivities lay, and simply decided not to actualise his fantasies until alcohol reduced his inhibitions sufficiently that he decided to go through with them.

Comment author: nick012000 29 September 2010 09:57:13AM *  10 points [-]

Eliezer, terrorists may not be evil mutants, but I'm pretty sure they do hate freedom. Islam translates to "submission to God", and if you look at the history of radical Islam, you'll see that their main opposition has been to freedom and liberalism all along. It all got started with a Muslim university student in the fifties who got disgusted with American immorality, and decided that Islam needed to stand against it, so he tried to overthrow the Egyptian government and establish an Islamic state. It failed, and he and his followers came to believe that it failed because Islam was being corrupted by Western freedoms and immorality.

They might not be evil, but their value structure is incompatible with ours.

Comment author: Mitchell_Porter 29 September 2010 10:05:32AM 17 points [-]

I'm pretty sure they do hate freedom

Here's something that Muslim university student wrote:

When, in a society, the sovereignty belongs to God alone, expressed in its obedience to the Divine Law, only then is every person in that society free from servitude to others, and only then does he taste true freedom. This alone is 'human civilization', as the basis of a human civilization is the complete and true freedom of every person and the full dignity of every individual of the society. On the other hand, in a society in which some people are lords who legislate and some others are slaves who obey them, then there is no freedom in the real sense, nor dignity for each and every individual.

Qutb hates "liberal" freedom, but he considers it internal slavery to animal desires, and it correlates with external slavery to a human hierarchy. Whereas knowledge of Islam humanizes you, and a shared knowledge of Islam allows people to live without dictators, because order comes from an impersonal source - shariah law - rather than the whim of a governing class.

Qutb definitely values a form of freedom, but says it can't exist unless you have Islam first.

Comment author: nick012000 29 September 2010 12:35:39PM 7 points [-]

Yup. Thank you for finding that quote; it pretty much proves my point. He hates the Western version of freedom, and wants to destroy it to replace it with the iron boot on Islamic rule (and seems to have missed that in order to implement sharia law, there have to be people doing the implementation).

Comment author: smijer 21 May 2012 08:24:37PM 5 points [-]

The person who originally claimed that "they hate us for our freedom" was probably referring to a Western, enlightenment notion, called by that name.

The thing that the Muslim university student praises and calls freedom is apparently an Islamic religious idea, corresponding very roughly to the sort of freedom a recovering addict craves from his addictions.

If the words were tabooed, then you would probably see the coherence of both points of view, and I think, could fairly assert that Islamists really do "hate our freedoms" in a sense, so long as you don't allow this approximation to carry more than its fair burden of explanatory weight (as certain former POTUSs have done).

Comment author: Danfly 21 May 2012 08:44:57PM 0 points [-]

When you put it like that, it actually sounds a lot like the Kantian notion of heteronomy versus autonomy.

Comment author: ChristianKl 24 May 2012 11:44:31AM 3 points [-]

Sayyid Qutb is more than a random Muslim university student. He's a central thinker. He was a leader of the Muslim Brotherhood when he wrote Milestones (the work you are citing).

Comment author: AlexanderRM 07 November 2014 07:04:26AM 0 points [-]

I just want to point out that that concept sounds almost exactly like something John Winthrop (the first governor of the Massachusetts colony) said, although I'm not sure if it was his Model of Christian Charity or a different speech (I have it in a book but it's not with me at present).

Although it's actually your interpretation that's almost exactly the same, not so much the original quote. Basically that being free from God's will doesn't make you free, it makes you a slave to your animal desires, and you can only be free by being subservient to God. Rather interesting that the ideas aren't that different. Although it might be that it's not so much a common point of Abrahamic religion, but rather an independent response that very religious people develop in response to ideas of freedom. Also generally interesting to remember how incredibly different Western culture- not German culture or whatever, but English culture- were just a few centuries ago, compared to the way they are now.

Anyway, mostly just found that interesting. Slightly relevant to point out that calling it "the Western version of freedom" isn't quite accurate, as said version goes back less than three centuries. I could furthermore point to all the people who cite Winthrop or the Puritans are America's founders (ex. politicians who use the phrase "City on a Hill"), but that wouldn't really be accurate; they don't really agree with any of their ideals and cite them because they honestly don't know how alien their ideals were to us.

Comment author: PhilGoetz 08 November 2010 05:25:08PM *  2 points [-]

I agree with the post; and I think it would be more applicable to us here on LW if we extend it to cover "stupid" as well as "evil". We see "stupid" and "evil" as not being very different; and we get the same shot of righteous adrenaline from putting down a stupid comment as from putting down an evildoer.

People who work with Steve Jobs said in the 1990s that he assigned everyone a "bozo bit"; and if they disagreed with him a few times, he set their bozo bit to 1, and ignored or derided everything they said from that day onward. That's functionally very similar to deciding someone is evil. Someone may be more likely to be stupid than to be evil; but that doesn't make having an irrational and emotional response to it any better.

Nor are we very good at determining who is stupid and who isn't! I've had many, many disagreements with Eliezer and with Vladimir Nesov. I know neither of them are stupid. But too many of us "solve" these recurring disagreements by setting a bozo bit and ignoring that person in the future.

Comment author: Vladimir_Nesov 08 November 2010 05:35:35PM *  0 points [-]

It's a good general heuristic. It saves time and effort. When a person changes, you might get exposed to overwhelming evidence of such change, and update back only then. It also helps to have topic-specific bozo bits. For me, you are flagged for metaethics and decision theory, but you write good posts on human rationality.

Comment author: Annie 26 February 2011 05:58:24PM 2 points [-]

Thank you so much! I really like this thread, because I've been arguing with people for years about it, and people just don't get it. ;) It's a really interesting topic, as well, trying to think from the bad guy's perspective. Thank you, again.

Comment author: Willami 24 September 2011 11:16:29AM 0 points [-]

To me, it simply comes down to one thing: belief.

If you absolutely believe something, then no matter how implausible it may seem to others with other beliefs, to you, in your mind, it is evident truth, and that therefore is your reality, and anyone who thinks otherwise will often be irritatingly stupid to you.

People with absolute beliefs that just require faith can pretty much rationalise anything to fit them, and are amazingly good at ignoring obvious flaws in their beliefs, and at seeing any, even tiny, counter argument, as being 'evil' and taking the other side, they can seem, and often are, a bit of a threat to those of us with a little more rational.

That's actually OK if that's as far as it goes, or if the 'war' is fought with words or posturing, but when people with absolute beliefs decide to make others conform to those beliefs through force, that's when you start getting the problems that lead to wars and terrorism.

As a general world view of when differing people try to 'police' the others (and not counting direct threat and the need to defend yourself), I believe those who try to enforce their belief upon others through force and threat we can call the bad guys, and those who try to do it through wordy persuasion and their own example that it works, without feeling the need to try to directly enforce it upon others, essentially presenting a counter choice and giving them it as a choice, we can call the good guys, because THAT really works, just a really persuasive argument or excellent proposition or compromise can completely change your perception of others and your particular reality in relation to them, and as such, everyones behaviour, forever! Thinking 'they're too stupid/corrupted to think like us so we'll just have to use force to get what we want' is when you lose (think of the whole IRA terrorist affair as a good example).

Think of it as the difference between teaching your dog to behave because he wants to because he likes you and believes in you as a good and respectable alpha, or teaching it to behave by pure punishment, like taking away it's food then smacking him on the nose with a newspaper whenever he jumps up on the kitchen table looking for scraps, the latter may work while you're about, but the moment you stop directly enforcing these punishments for misbehaviour, that dog will jump on the table anyway, probably with unbridled joy that he now has the freedom to!

I personally can't see that anyone in the whole 'War Against Terror' are particularly blame free, and I include pretty much all that got involved in it in that statement, and I think there was and is too much stereotyping and meddling in others affairs in the world by the super powers, using too much stick and not enough carrot, and using threat and force to try and make other far away people conform to your personal reality rarely ends well, and frankly, it p*sses people off, and all your really doing is creating the 'they'll only behave for as long as you can control them' scenario as set out above, eventually something bad is going to happen when your back is turned, or one day you get too weak to keep a firm grip on that newspaper!

As an aside, on the subject of 'evil wrongdoers' in history and how we decide who was really 'the bad guys' when we review it through our contemporary eyes, I can't help but notice they very often carry a 'good book' and an agenda to use it to qualify their actions, though...

Comment author: kilobug 08 October 2011 12:21:12PM 8 points [-]

I agree with most of the points in this article, but yet it underestimates a fundamental difference between two ways of disagreeing.

Take the typical political debate about "raising taxes on the wealthy to give social help to the poor" vs "giving tax cuts and reducing social help". People can disagree on that topic for two completely different set of reasons.

People can disagree on that topic because, even if they share a more-or-less common utility function, in which having people dying of cold in the street is valued very negatively, they have different expectations about what each policy will do. Some will say that raising taxes on the wealthy and giving the money to the poor will improve the living conditions of the poor, without hurting much the wealthy, and will be good for the economy since it'll increase the demand in construction/good factory/... which is the true motor of economy. Some will say that raising the taxes on the wealthy and giving the money to the poor will lower the incentive for the rich to invest in the economy, and for the poor to find themselves a job, and will at the end damage the whole economy and makes everyone poorer on the long run. I've my own opinion on that (and I may very well be reflected by the way I formulated the two hypothesis, but that's really not my goal here, so please accept my apologizes if it's too obvious). That disagreement is not easy to settle (or it would have been settled since long), but can be argued rationally, looking at history, at different countries, making prediction on what will happen when a country changes it's welfare/tax policy and checking the prediction later on, ... Labeling "evil" someone who share a similar utility function but disagree on the ways to maximize it is indeed a catastrophic mistake.

But people can also disagree on that topic because they have different utility functions. Some do think that homeless are people who deserve to be homeless because they were too lazy and stupid, and give a positive value to the fact that lazy/stupid people are punished. Some others do not include any significant term in their utility function for the homeless, considering they are only a tiny part of the population and not worth considering. Some others do think that having rich people is in itself unfair, and that taxing them highly, even if it doesn't reduce poverty, increases fairness and is therefore valued positively. Disagreements between people having different utility functions like that will be much, much harder to settle. And labeling "evil" someone who has a broadly different utility function than yourself is much more understandable. If we don't for that, well, for what is the word "evil" ?

And it gets even more complicated, because you can have people who claim they share your utility function, while in fact they don't. And you don't always even know for sure if they are or not. And because it's often a bit of both - people who favor high taxes/social help and those who favor tax cuts/no social help usually have both different pondering in their utility function and different expected outcomes for the two policies.

Comment author: DSimon 08 October 2011 01:07:45PM *  3 points [-]

You pose a number of excellent questions in your comment that one might ask about a political opponent. So then the next step is: how does one go about answering those questions? How do you figure out whether one's opponent has different terminal values, or different instrumental values, or both?

Comment author: kilobug 08 October 2011 01:41:32PM 6 points [-]

That's a very difficult question.

One part of the answer lies into understanding the nature of the debate. Basically, I consider there are two kinds of debates :

  1. A debate between two people, who trust each other (at least to a point, like friends or family members) and without witness. In that case, the whole point of the debate is trying to discover the truth, hoping at the end the two will agree (one conceding he was wrong, finding a common ground in the middle, or finding a third option unthought at the beginning), and it's quite easy to ask the other about his real goal (immediate or distance) and to assume he'll be honest about it.

  2. A debate between two people, but who perfectly know they won't convince each other, but they try to convince witness of the debate. That's a typical debate between political candidates in an election. In that kind of debate, you've to be very doubtful about the claimed values (terminal or instrumental) of everyone involved. That kind of debate is very hard to handle in a non-mindkilling way.

Most real life debates are somewhere in between those two archetypes.

So I make that double distinction : between disagreement in expectation and disagreement in utility function, and between debating in order to get closer to the truth, and debating in order to convince third parties. I don't have a magical solution to find out for sure in which case we are, but I'll be glad to hear some tips/hindsight on the topic.

Another way to state it : to me a political debate in front of witnesses is very like a kind of prisoner dilemma. You can cooperate, by being true on your terminal and instrumental values, being honest, pointing to the flaws of your own side when you see them, ... Or you can defect, by hiding your true values, hiding facts, avoiding your weak points, and even lying on facts.

If both cooperate, the debate will go smoothly, and is likely to end up in everyone being closer to the truth than when you started. If both defect, the debate will get dirty, the two debaters will end up more convinced of their own view than initially, but the witnesses will still, on average, be closer to the truth, because I do believe that it's easier to defend something "true" than something "false". But if one cooperates and the other defects, then the one who defects is very likely to convince the witnesses, regardless of him being right or wrong.

So for myself I tend to use a "tit-for-tat with initial cooperation and forgiving", like I do on anything that I identify as an iterated prisoner dilemma : I cooperate initially, if I get the feeling the other is defecting, I'll resort to defecting (but I'll never go as far as openly lying, that's against my ethical code of conduct), but still try to fallback to cooperate every now and then and see if the other cooperates then or not.

Comment author: DSimon 09 October 2011 04:54:32PM *  2 points [-]

I think that debates of the first type are probably even rarer than you estimate; even when two people who trust each other, and who both have a deliberate intent to seek the truth, are arguing alone, political instincts and biases kick in pretty hard.

I do really like your overall strategy; I'll try to remind myself more often in the future to occasionally turn down my politics-face a bit to see if the other is willing to return to a more cooperative state.

Comment author: momothefiddler 24 October 2011 07:35:29PM 5 points [-]

The Jesus Camp link is broken. Does anyone have an alternative? I don't know what Eliezer is referencing there.

Comment author: thomblake 24 October 2011 08:08:53PM 1 point [-]

I believe the link initially pointed to a trailer for the movie Jesus Camp.

Comment author: taelor 24 October 2011 08:14:10PM 5 points [-]

Jesus Camp is a documentary about a camp for fundamentalist Christian youths. The first part can be seen here (check the related videos for the subsequent parts). Alternately, if you don't have time to watch the full movie, this should give you a general idea.

Comment author: Rinon 04 June 2012 03:06:02PM *  2 points [-]

This post needs to be air-dropped over the world's ten largest metropolitan areas. Actually, Harry Potter and the Methods of Rationality needs to be air-dropped, with translations where necessary, because it contains the same truths but is more entertaining. I think the same arguements apply to labeling your enemies as insane mutants, which is a somewhat gentler, more politically correct way of demonizing them. We tend to assume that the enemy is insane because we could not imagine doing such a thing, and are therefore Surprised by Reality. It might make sense to update our idea of "sanity."

Comment author: [deleted] 04 June 2012 03:17:44PM 2 points [-]

Actually, Harry Potter and the Methods of Rationality needs to be air-dropped, with translations where necessary, because it contains the same truths but is more entertaining.

Not many people would want to read such a long book in large metropolitan areas (where people tend to be very busy). And this Cracked article also makes the same point (in Section 1 -- they're numbered backwards; and the other sections also cover topics which were covered in the Sequences).

Comment author: OnTheOtherHandle 13 September 2012 06:51:35AM *  4 points [-]

Ever since I read this post, I've been trying to be as charitable as possible to my opponents, but it's been an uphill battle because emotions flare up quickly.

I recently discovered a nice psychological trick that happens to work on me, which helps me argue more sanely. As soon as I get angry or emotional, I catch myself, look straight at the other person, and repeat in my mind, I want you to prosper. I want you to be happy. I want you to live a fulfilling life. (This is true, for any and all people who disagree with me.)

Personally, I find it easier than trying to fight negativity with more negativity ("You're a Bad Person for thinking your opponent is a bad person; why is this such a horrible argument?"). Explicitly reminding myself that I'm on Team Humanity, not Team Political Party, explicitly reminding myself that the point of this arguing is to find a better way to help people, is usually enough to help me zoom out and erase negative feelings.

This hasn't been extensively tested, but it seems like it would fit the mindset of many people here. It worked on an Objectivist who said he would rather have people in Rwanda and Somalia starve to death than live like "parasites" on private charity. As I type this, I don't have nearly as much of a negative reaction to this person as I would have otherwise. I want this person to have a good life.

Edit: This is probably due to the Ben Franklin effect. I wonder if it would be strengthened by finding something to verbally compliment them on.

Comment author: jooyous 04 January 2013 09:14:38AM *  6 points [-]

I will just pop in here to say that I used to be this huge snob who would look down at people my age who said that Ender's Game is one of their favorite books. I was like "Clearly, they have not read any fancy literature since middle school. Silly noobs!"

And then I read the book again and I realized that actually that book is super-important because it basically captures the contents of this article in a book for children when most books for children are all about taking out the evil bad guy and sorta imply that violence is no big deal.

Comment author: wallowinmaya 03 February 2014 01:19:03PM 0 points [-]

Great post of course.

If it took a mutant to do monstrous things, the history of the human species would look very different. Mutants would be rare.

Maybe I'm missing something, but shouldn't it read: "Mutants would not be rare." ? Many monstrous things happened in human history, so if only mutants could do evil deeds, there would have to be a lot of them. Furthermore, mutants are rare, so no need for the subjunctive "would".

Comment author: TheOtherDave 03 February 2014 05:10:33PM 0 points [-]

We posit a hypothetical alternate universe U where only mutants do monstrous things.

We observe that mutants are rare in our world, and we speculate that the causes of mutant rarity would not be different in U, and therefore we conclude that "mutants would be rare" in U, and therefore we conclude that "the history of the human species would look very different" in U... specifically, that fewer monstrous things would have happened.

Comment author: brazil84 03 February 2014 01:35:46PM 1 point [-]

Realistically, most people don't construct their life stories with themselves as the villains. Everyone is the hero of their own story. The Enemy's story, as seen by the Enemy, is not going to make the Enemy look bad. If you try to construe motivations that would make the Enemy look bad, you'll end up flat wrong about what actually goes on in the Enemy's mind.

Well only at the uppermost level. i.e. most evil people are unaware that they are evil. They misconstrue their own motivations. So if you construe motivations which make the Enemy look bad, there's actually a good chance that you are correct. Of course there is also a good chance you are a hypocrite, i.e. you are misconstruing your own motivations too.

Comment author: [deleted] 03 February 2014 02:51:12PM -2 points [-]

Nah, some of us are entirely well aware of the fact that we're just plain evil. It's just that, at a certain point, you look at the things that the vast supermajority of the human race consider Good and Virtuous, like noble kings who pull swords from stones, hard labor on farms, and their local army. Then you look at the things the vast supermajority of the human race consider Evil and Scummy, like democratically-elected officials, premarital sex, video games, and peace activists.

And then, you realize that judging by the set of things you found appealing, you must be Evil, and you might as well embrace it.

Comment author: blacktrance 03 February 2014 03:36:39PM -1 points [-]

I'm confused. By "evil" do you mean "what the supermajority of the world considers Evil"?

Comment author: [deleted] 03 February 2014 03:47:13PM -1 points [-]

Basically, yes. I am what the supermajority of the world considers Evil: a damn dirty democratic socialist who wants people to stop obsessing over stupid bullshit and have fun together.

The mere fact that this sounds pretty ok when I say it outright just means I have creepy mind-control powers.

Comment author: blacktrance 03 February 2014 03:52:46PM 0 points [-]

But you don't actually think you're evil. You think you're good, you just recognize that people think you're evil.

Comment author: [deleted] 03 February 2014 04:02:24PM -1 points [-]

No, I think the terminology is democratically determined.

Comment author: Lumifer 03 February 2014 09:59:58PM 4 points [-]

LOL. Supporting evidence, quoting Yvain's post:

If you write some very reasonable liberal enlightened essay about how maybe there's some reason to believe some women are such-and-such but we must not jump to conclusions, people will call you a sexist, you'll have to argue that you're not a sexist, and your opponents have spent their entire lives accusing people of sexism and are better at this argument than you are and will win (or at least reduce your entire output to defending yourself). If you're Heartiste, and people call you sexist, you can just raise an eyebrow, say "Well, yeah", and watch people whose only master-level argumentative gambit is accusing people of sexism have no idea what to do.

Comment author: [deleted] 04 February 2014 04:20:14PM *  1 point [-]

While this is indeed exactly what happens when people call me a "communist" or "socialist", I'm not really sure what the relation is. I was mostly just expressing my near-total cynicism regarding people's ability to recognize that they don't live in storybooks.

EDIT: As well as expressing the observation that despite my expressed moral alignment agreeing, if somewhat roughly, with what is commonly found on LessWrong, we are a tiny clique among billions. The expected-value human, chosen at random from throughout the world, thinks and feels in a way that we would deem bizarrely and almost inexplicably hostile. Whatever anyone thinks of my politics aside, much/most of the planet still operates mostly on "survive ethics", in which thinking is expensive and aesthetics suicidal, and would be both shocked and appalled to find out about how we live as people well-off enough to implement even a little "thrive ethics".

Hell, just yesterday I had to suffer through a "debate" with a man who believes that by persecuting gay people and marrying exclusively within our religion, we can cause God to send the Messiah, who will proceed to... harshly enforce religious law, including death penalties, which are then multiplied as spiritual punishments in the afterlife. This is what many/most people actually believe.

Comment author: Lumifer 04 February 2014 04:59:53PM 2 points [-]

The expected-value human, chosen at random from throughout the world, thinks and feels in a way that we would deem bizarrely and almost inexplicably hostile.

I don't think so. I've been to many places in the world, observed people there and talked to some of them. They did not strike me as "bizarrely and almost inexplicably hostile".

thinking is expensive and aesthetics suicidal

Nope. I don' think this is true at all.

This is what many/most people actually believe.

Didn't you mean to say "This is what many/most Orthodox (and, generally, non-Reform) Jews actually believe"? There is a wee bit of a difference here... :-)

Comment author: [deleted] 04 February 2014 06:22:33PM *  3 points [-]

I don't think so. I've been to many places in the world, observed people there and talked to some of them. They did not strike me as "bizarrely and almost inexplicably hostile".

I will admit to a certain cynicism in my worldview. I do, however, wish to note that half the species lives in either China, India, or Africa. Of these, China is generally considered fairly safe and orderly. India and Africa are getting more safe and orderly over time. South America is a step up from India and Africa in some countries and a step down in some few remaining regions. North America and Europe are generally some of the safest, most orderly places on Earth. Eastern Europe and the Caucasus, as I understand them, are often orderly but sometimes unsafe.

The Middle East, where I live, is generally considered the planet's dumping ground for pointless, vicious hatred.

Anyway...

Didn't you mean to say "This is what many/most Orthodox (and, generally, non-Reform) Jews actually believe"? There is a wee bit of a difference here... :-)

Not even. I've met ultra-Orthodox Jews who use their mysticism and woo to justify a very warm and outgoing outlook on the world, and to whom punishment fantasies are regarded as poison for authentic religion. There's a very religious Muslim who works in the same building as me, actually, and we always say hi in the hall. To him, Allah (whom I usually regard as an utter totalitarian) gives the imperative to treat life seriously, consider things, and be ethical. In general, this guy acts like a kinder, more mature person than many of the secular people I see every day.

This isn't to say I agree with either of these two religious views either, just that what we're dealing with is a general psychological outlook rather than a uniform ideology confined to a single specific clade. "Religious/ideological people are violent and nasty, but most people aren't like that" is an easy way for atheists, agnostics, and moderates of all stripes to congratulate themselves, but I don't think it's actually true.

Now that I come to reevaluating my expressed view, I no longer think my original cynicism is even mostly correct, either. However, it took a relatively large step of new, original thinking for me to come up with a theory as to why, and it also required a major evaluation of my sampling set (ie: the set of people I've met as opposed to the set of people who exist).

So anyway, I should probably go exercise, maybe refrain from writing down my Deep Insight so I never get ridiculed for taking so long to figure out what other people just knew the whole time. You get an upvote for challenging my narrow-minded cynicism and making me reevaluate.

Comment author: Colombi 20 February 2014 05:23:26AM -2 points [-]

Eliezer, you are awesome!

Comment author: FiftyTwo 04 October 2014 08:41:23PM 0 points [-]

Does anyone know of any psychological studies showing it is actually the case that people regard their enemies as evil, rather than misguided?