To save the world, make sure to go beyond academia

There were several interesting talks at the GoCAS workshop on existential risk to humanity. The one that was maybe the most thought-provoking was the last one, by Seth Baum, who discussed the difficulty of translating the results of academic research into something that actually does save the world.

He gave two examples. First, climate change: apparently in the economics literature, there has been an extended debate about the optimal level of a carbon tax; however, in the US, all of this debate and finding exactly the optimal level is kind of irrelevant, given that there is no carbon tax and there is considerable opposition to creating one. So the practical work that needs to be done, has been various organizations that are working to get support for politicians that care about climate change. Also valuable are some seemingly unrelated efforts like work to stop gerrymandering, because it turns out that if you didn’t have gerrymandering, it would be easier to elect politicians who are willing to implement things like a carbon tax.

His other example was nuclear disarmament. Academia has produced various models of nuclear winter and of how that might be an x-risk; however, in practice this isn’t very relevant, because the people who are in charge of nuclear weapons already know that nuclear war would be terribly bad. For them, the possibility of nuclear winter might make things slightly worse, but the possibility of nuclear war is already so bad that such smaller differences are irrelevant. This is a problem, because nuclear disarmament and reducing the size of the nuclear stockpile could help avert nuclear winter, but the decision-makers are thinking that nuclear war is so bad that we must be sure to prevent it, and one of the ways to prevent it is to have a sufficiently large nuclear arsenal to serve as a deterrent.

His suggestion was that the thing that would actually help in disarmament, would be to make various countries – particularly Russia – feel geopolitically more secure. The US basically doesn’t need a nuclear arsenal for anything else than deterring a nuclear strike by another power; for any other purpose, their conventional military is already strong enough to prevent any attacks. But Russia is a different case: they have a smaller military, smaller population, smaller economy, and they border several countries that they don’t have good relations with. For them, maintaining a nuclear arsenal is an actual guarantee for them not getting invaded. Similarly for Pakistan, maybe Israel. The key for actually getting these countries disarm would be to change their conditions so that they would feel safe in doing so.

He emphasized that he wasn’t saying that academic research was useless, just that the academic research should be focused and it should be used in a way that actually helps achieve change. I’ve been thinking about the usefulness of my x-risk/s-risk research for a while, so this was very thought-provoking, though I don’t yet know what actual updates I should do as a result of the talk.

Debiasing by rationalizing your own motives

Some time back, I saw somebody express an opinion that I disagreed with. Next, my mind quickly came up with emotional motives the other person might have for holding such an opinion, that would let me safely justify dismissing that opinion.

Now, it’s certainly conceivable that they did have such a reason for holding the opinion. People do often have all kinds of psychological, non-truth-tracking reasons for believing in something. So I don’t know whether this guess was correct or not.

But then I recalled something that has stayed with me: a slide from a presentation that Stuart Armstrong held several years back, that showed the way that we tend to think of our own opinions as being based on evidence, reasoning, etc.. And at the same time, we don’t see any of the evidence that caused other people to form their opinion, so instead we think of the opinions of others as being only based on rationalizations and biases.

Yes, it was conceivable that this person I was disagreeing with, held their opinion because of some bias. But given how quickly I was tempted to dismiss their view, it was even more conceivable that I had some similar emotional bias making me want to hold on to my opinion.

And being able to imagine a plausible bias that could explain another person’s position, is a Fully General Counterargument. You can dismiss any position that way.

So I asked myself: okay, I have invented a plausible bias that would explain the person’s commitment to this view. Can I invent some plausible bias that would explain my own commitment to my view?

I could think of several, right there on the spot. And almost as soon as I could, I felt my dismissive attitude towards the other person’s view dissolve, letting me consider their arguments on their own merits.

So, I’ll have to remember this. New cognitive trigger-action plan: if I notice myself inventing a bias that would explain someone else’s view, spend a moment to invent a bias that would explain my opposing view, in order to consider both more objectively.

The muted signal hypothesis of online outrage

Everyone, it sometimes seems, has their own pet theory of why social media and the Internet often seem like so unpleasant and toxic places. Let me add one more.

People want to feel respected, loved, appreciated, etc. When we interact physically, you can easily experience subtle forms of these feelings. For instance, even if you just hang out in the same physical space with a bunch of other people and don’t really interact with them, you often get some positive feelings regardless. Just the fact that other people are comfortable having you around, is a subtle signal that you belong and are accepted.

Similarly, if you’re physically in the same space with someone, there are a lot of subtle nonverbal things that people can do to signal interest and respect. Meeting each other’s gaze, nodding or making small encouraging noises when somebody is talking, generally giving people your attention. This kind of thing tends to happen automatically when we are in each other’s physical presence.

Online, most of these messages are gone: a thousand people might read your message, but if nobody reacts to it, then you don’t get any signal indicating that you were seen. Even getting a hundred likes and a bunch of comments on a status, can feel more abstract and less emotionally salient than just a single person nodding at you and giving you an approving look when you’re talking.

So there’s a combination of two things going on. First, many of the signals that make us feel good “in the physical world” are relatively subtle. Second, online interaction mutes the intensity of signals, so that subtle ones barely even register.

Depending on how sensitive you are, and how good you are generally feeling, you may still feel the positive signals online as well. But if your ability to feel good things is already muted, because of something like depression or just being generally in a bad mood, you may not experience the good things online at all. So if you want to consistently feel anything, you may need to ramp up the intensity of the signals.

Anger and outrage are emotional reactions with a very strong intensity, strong enough that you can actually feel them even in online interactions. They are signals that can consistently get similar-minded people rallied on your side. Anger can also cause people to make sufficiently strongly-worded comments supporting your anger that those comments will register emotionally. A shared sense of outrage isn’t the most pleasant way of getting a sense of belonging, but if you otherwise have none, it’s still better than nothing.

And if it’s the only way of getting that belonging, then the habit of getting enraged will keep reinforcing itself, as it will give all of the haters some of what they’re after: pleasant emotions to fill an emotional void.

So to recap:

When interacting physically, we don’t actually need to do or experience much in order to experience positive feelings. Someone nonverbally acknowledging our presence or indicating that they’re listening to us, already feels good. And we can earn the liking and respect of others, by doing things that are as small as giving them nonverbal signals of liking and respect.

Online, all of that is gone. While things such as “likes” or positive comments serve some of the same function, they often fail to produce much of a reaction. Only sufficiently strong signals can consistently break through and make us feel like others care about us, and outrage is one of the strongest emotional reactions around, so many people will learn to engage in more and more of it.

The parliamentary model as the correct ethical model

In 2009, Nick Bostrom brought up the possibility of dealing with moral uncertainty with a “parliamentary model” of morality. Suppose that you assign (say) 40% probability to some form particular of utilitarianism being correct, and 20% probability to some other form of utilitarianism being correct, and 20% probability to some form of deontology being true. Then in the parliamentary model, you imagine yourself as having a “parliament” that decides on what to do, with the first utilitarian theory having 40% of the delegates, the other form having 20% of the delegates, and the deontological theory having 20% of the delegates. The various delegates then bargain with each other and vote on different decisions. Bostrom explained:

The idea here is that moral theories get more influence the more probable they are; yet even a relatively weak theory can still get its way on some issues that the theory think are extremely important by sacrificing its influence on other issues that other theories deem more important. For example, suppose you assign 10% probability to total utilitarianism and 90% to moral egoism (just to illustrate the principle). Then the Parliament would mostly take actions that maximize egoistic satisfaction; however it would make some concessions to utilitarianism on issues that utilitarianism thinks is especially important. In this example, the person might donate some portion of their income to existential risks research and otherwise live completely selfishly.

As I noted, the model was proposed for dealing with a situation where you’re not sure of which ethical theory is correct. I view this somewhat differently. I lean towards the theory that the parliamentary model itself is the most correct ethical theory, as the brain seems to contain multiple different valuation systems that get activated in different situations, as well as multiple competing subsystems that feed inputs to these higher-level systems. (E.g. there exist both systems that tend to produce more deontological judgments, and systems that tend to produce more consequentialist judgments.)

Over time, I’ve settled upon something like a parliamentary model for my own decision-making. Different parts of me clearly tend towards different kinds of ethical frameworks, and rather than collapse into constant infighting, the best approach seems to go for a compromise where the most dominant parts get their desires most of the time, but less dominant parts also get their desires on issues that they care particularly strongly about. For example, a few days back I was considering the issue of whether I want to have children; several parts of my mind subscribed to various ethical theories which felt that the idea of having them felt a little iffy. But then a part of my mind piped up that clearly cared very strongly about the issue, and which had a strong position of “YES. KIDS”. Given that the remaining parts of my mind only had ambivalent or weak preferences on the issue, they decided to let the part with the strongest preference to have its way, in order to get its support on other issues.

There was a time when I had a strong utilitarian faction in my mind which did not want to follow a democratic process and tried to force its will on all the other factions. This did not work very well, and I’ve felt much better after it was eventually overthrown.