Meteuphoric

It is hard for humans to escape caring what other humans think of them. Arguably, it is hard for humans to escape caring about about what other humans would think of them, were other humans to ever learn the fascinating truth of what they put in their sandwich today, or how stylish the sweatpants they wear to eat it will be.

This attention to ‘what people think’ is usually seen as regrettable but unavoidable, so we encourage one another not to do it, and leave it at that.

Yet how good or bad ‘caring what people think’ is must surely depend a lot on who ‘people’ are.  And I think this actually differs substantially between different self-conscious minds, and can be altered. Which is to say, even if you are beholden to the thoughts of ‘people’, the nature of this is flexible.

In the extreme, arbitrary imaginary observers could applaud any kind of behavior, so it’s flexible in the sense that there isn’t really any behavior this couldn’t lead to. But also in practice, with the particular set of people who exist (and the ones who could, and will, and did), I suspect an individual person can come to quite different conclusions about what ‘people’ think by aggregating the people and their thoughts differently.

Some variation I suspect exists among people’s imaginary observers:

• Real vs. ideal: if everyone in the world prefers policy X but you are confident that a rational agent would prefer policy Y, which do you feel less ashamed carrying out?

• Actual vs. hypothetical: if two people are aware of your behavior and everyone else isn’t, how do you weigh what those two actually think of it relative to what everyone else would think of it if they knew? e.g. How shameful would it feel to partake in bestiality in front of other enthusiasts?

• Local vs. global: do you feel better doing something that the people around you approve of if you think that most people in the world or in time would disagree, or doing something locally unpopular, confident that most people ever would applaud?

• Informed vs. uninformed: if given all of the information P is a good action and Q is a stupid action, but with less information Q looks better, which do you feel less ashamed to do? (Supposing that in practice nobody is looking).

• Sympathetic vs. critical: do you have to really make a strong case to persuade the imaginary onlookers, or are they itching to approve of your actions if you give them any justification?

• Eliteness: do you care what everyone thinks, or how strongly do you weight the views of the ‘best’ people, on some measure of best?

• Specific cultural groups: do you mostly care what atheists think, or American people or liberals?

• Weighting of particular people: are there romantic partners or Gods or heroes or crushes who get a particular place in the jury?

If you are mostly trying to appeal to an ideal hypothetical global informed elite audience, this seems pretty hard to distinguish from just being a really good person. It sort of internalizes the externalities, and and requires you to do your best impression of being reasonable and informed yourself.

If you are performing more for the respect of ten ignorant idiots literally watching you, this could look like all sorts of things, but many of them will be bad. Often even for the observers, since you are incentivised to match their ignorance.

A problem with listening to arguments is that they often fail to include the evidence that provoked them, which can be informative where the argument itself is fatally flawed.

For instance, suppose there is a God. And suppose that people frequently see him, and so feel inclined to believe in him. However they know ‘I saw God!’ will get little interest and much criticism, so they don’t say that. But, feeling more positively inclined toward pro-God arguments, and end up tentatively agreeing with some of them. They come to say ‘how did eyes evolve?’ and ‘where did the universe come from?’, because these are the most compelling-to-them pro-God arguments they came across. And so you—who has never seen God—just see a whole lot of people making bad arguments about God, and then weirdly believing them. But the important evidence—that a large portion of the population has experienced personally meeting God—is hidden from you, though in sum you might have taken it more seriously than you take a flawed argument.

If people feel that arguments are more virtuous than anecdote, you should remember that when people make arguments, they might be doing it in the place of anecdotes, that a) actually changed their mind and b) are actually interesting evidence.

This is especially true in a world where most people can’t argue their way out of a paper bag, and are also more frequently compelled by non-argument phenomena than arguments.

So, an upshot is that if someone makes an argument to you, consider asking the story of how they came to feel disposed toward its conclusion.

A real example:

I remember motivatedly reasoning in the past, and while I expect my arguments were above average, had someone wondered what produced them and asked me, I might have told them that I had been in the forests, and that they were incredible and made me feel different to being in other places, and that I am further offended by people getting their way and destroying value in the name of bad reasoning, and that I had always been basically on the environmentalist side, because everyone I knew said that it was wrong. And even if my arguments had been of no interest to someone, they could infer from my story that the forests were probably amazing to experience, and that local environmental politics was polarized (that the other side seemed frustratingly wrong could probably be guessed). Either of which is evidence on whether the forests should be destroyed, and probably not mentioned a lot in my arguments.

A real possible example where this might help:

Perhaps people sometimes argue that AI will undergo a fast take-off, because “once there is a new feedback loop, who knows how fast it will go?” And you do not find this argument compelling—after all, there are new feedback loops all the time, and they rarely destroy the world. But what caused them to think AI will undergo a fast take-off? One possibility is that their intuitions are taking in many other things about the world, and producing expectations of fast take-off for reasons that the person does not have conscious ability to explain. If so, that would be interesting to know, regardless of the quality of their arguments. Another possibility is that they heard someone argue really compellingly, and they can’t remember the precise argument. Or they trust someone else who claimed it. These might be informative or not, depending on why the person seemed worth listening to.

Related: Chesterton’s Fence in the presence of bull

How much do you think your time is worth?

Let’s say you are pretty unsure, but you guess ‘hardly anything’. So when you see opportunities to turn your time into a little bit of value, you naturally jump at them. You offer to clean your friend’s car for her, because it’s a way of saving ten dollars for like an hour of your time, which would otherwise be put to a worse use. You put a lot of attention into finding good shopping deals and learning how to make cheaper meals. And in between all this, you don’t mind cutting down your work hours to enjoy life, because losing an hour of productivity is worth less than the popcorn. Sometimes you do reassess the value of your time, by thinking about your second best options for spending it. For instance, if you didn’t clean your friend’s car, what would you have been doing? Probably making your own granola, which is worth something, but doesn’t matter much. So you learn that you were right: your time is not worth much.

But let’s say instead that you guess that your time is worth a lot. You then shamelessly skip over any task that isn’t also worth a lot. You jump at opportunities to spend dollars to save minutes, and buy whatever is convenient and doesn’t get in the way of the real work. If a new opportunity appears, you compare it to the next best thing you could be doing, and since that is something awesome (or you wouldn’t be doing it), few opportunities are good enough to take. So it turns out that you were right: your time is worth a lot.

This probably shouldn’t happen to an ideal agent, but I think it probably does happen to humans. I am curious whether this is a familiar phenomenon to others.

You might think, if you guess your time is not worth much, won’t you correct it later if it is, when you see the more amazing opportunities on the table? In practice maybe not, because you (and others) partly assess which things you can do by the current value of your time (e.g. maybe I can see that it is valuable to start an awesome startup but I assume it is not an opportunity for *me* because I am usually peeling carrots or something, and am not even amazing at that). Also you don’t necessarily hear about high value opportunities if you are doing low value activities. Also, maybe you hear about fewer things altogether if you stop at the first opportunity to turn time into peanuts and turn your attention to running that process, instead of looking further. And you just see a whole different set of opportunities if you go down one path instead of another. Like if you decide to start at cleaning jobs, you then hear about other (sometimes somewhat better) cleaning-job-level things, not about important philosophy problems that need solving.

Often groups of people behave in consistent ways for a long time because they share a belief that the consistent way everyone is behaving will cause things to be bad for any individual who deviates from it.

For instance, there is a line in the sandwich shop. From a perspective so naive to our ubiquitous norms that it is hard to imagine, you might wonder why the person standing at the back does so, when the shopkeeper is much more likely to get sandwiches for people at the front. The reason of course is that if he were to position himself in the ample physical space between the person at the front and the shopkeeper, there would be some kind of uproar. Not only would the person at the front be angry, but everyone in the line would back them up, and the shopkeeper probably wouldn’t even grant a sandwich to the line-jumper.

So key to our ubiquitous tendency to stand peacefully in line is the fact that our common behavior is ‘stand in line and get angry with anyone who jumps it’ not just ‘stand in line’ which would be immediately exploited until the norm upholder gave up or died of starvation.

An interesting thing about this extra clause is that it is about our hypothetical behavior in circumstances that rarely happen. If our norms work well enough, we might go for years all peacefully standing in line, without anyone ever trying to push in at the front, because why would they?

An upshot is that if serious norm violations are rare, people might become pragmatically ill-equipped to respond to them. They might forget how to, or they might stop having the right resources to do so, physical or institutional. Or if generations are passing with no violations, the new generation might just fail to ever learn that they are meant to respond to violations, or learn what that would look like, since they never observe it. And maybe nobody notices any of this until norms are being violated and they find they have no response.

For instance, suppose that occasionally people sort of wander toward the front of the line in ambiguous circumstances, hoping to evade punishment by feigning innocent confusion. And those in the line always loudly point out the ‘error’ and the room scowls and the person is virtually always scared into getting in line. But one day someone just blatantly walks up to the front of the line. People point out the ‘error’ but the person says it is not an error: they are skipping the line.

The people in the line have never seen this. They only have experience quietly mentioning that they observe a possible norm violation, because that has always been plenty threatening. Everyone has become so used to believing that there is terrifying weaponry ready to be pulled out if there really were a real norm violation, that nobody has any experience pulling it out.

And perhaps it has been so long since anyone did pull it out that the specific weapons they stashed away for this wouldn’t even work any more. Maybe the threat used to be that everyone watching would gossip to others in the town about how bad you were. But now in a modern sandwich shop in a large city, that isn’t even a threat.

The world is full of sufficiently different people that in the real world, maybe someone would just punch you in the face. But it seems easy to imagine a case where nobody does anything. Where they haven’t been in this situation for so long, they can’t remember whether there is another clause in their shared behavior pattern that says if you punch someone because they got in line in front of you at the sandwich shop that you should be punished too.

Does this sort of erosion of unexercised threats actually happen? I am not sure. I think of it if for instance a politician behaves badly in ways that nobody had even thought of because they are so obviously not what you do; and then get away with it while onlookers are like ‘wait, what?! You can’t do that!’ But I don’t know enough details of such cases to judge whether they are cases of threat erosion.

Another case where I guess people might experience this is in bringing up children, because threats of punishment are often made, and the details of the relationship is changing as the children change, so if you don’t exercise the threats they can cease to be relevant without you noticing.

I probably saw something close to this in looking after my brothers. My brothers were into fighting for fairness and justice, where ‘fairness’ is about one’s right to have stuff someone else has, and ‘justice’ is about meeting all slights with tireless bloodthirsty revenge. So my main task was dealing with fights, and threats were relevant. When my brothers were small, I was physically in control of what they could have or eat or punch, so could uphold threats. Later they were really big and I couldn’t have enforced any threats if they decided to make it a physical conflict. This hadn’t been strongly tested by the time it dawned on me, and I continued to bluff. But there was perhaps an earlier period when I didn’t realize I was now bluffing, where if they had realized first, I would have been left with no response. This isn’t quite a case, because noticing that my threats were decaying didn’t let me strengthen them. But had I been an adult in charge of money and the car and such, things may have been different.

I’m not sure if such a situation would last for long in important cases where a replacement threat is feasible. If violations are still clearly bad for a set of people who have general resources to invest in threats, I’d often expect a new and compelling response to be invented in time.

From an email discussion (lightly edited):

I actually think an important related dynamic, in the world at large more than EA, is people favoring actions that are verifiably useless to themselves over ones that are probably useful to others but also maybe useful to themselves. I blogged about this here a while ago. In short, I see this as a signaling problem. The undesirable action (destroying resources in an evidently useless way) is intended to signal that you are not bad. Bad people (greedy exploiters trying to steal everyone else’s stuff) can make themselves look just like effective good people (both do things that look high leverage and where it is not totally clear what the levers are ultimately pushing). So the bad people do that, because it beats looking bad. Then there is no signal that the effective good people can send to distinguish themselves from the bad people. So people who want to not look bad have to look ineffective instead.

A way something like this might happen in our vicinity e.g. if I genuinely guess that the most effective thing to do might be for me to buy a delicious drink and then sit still in a comfy place for the day and think about human coordination in the abstract. However this is much like what a selfish version of me might do. So if I want to not humiliate myself by seeming like a cheating free-rider liar motivated reasoner in front of the other EAs, or perhaps if I just experience too much doubt about my own motives or even if I just want to make it straightforward for others around to know they can trust me, perhaps I should instead work for a reputable EA org or earn money in an annoying way and give it to someone far away from me.

On this model, the situation would be improved by a way to demonstrate that one is effective-good rather than effective-evil. (As in, a second sense in which it is a signaling problem is that adding a good way to signal would make it better).