Links for May
In English
What would happen if a superintelligent AI was aligned with your values?
The details here are a little too much in the “superintelligence is magic that can achieve anything” direction to my taste (I don’t think that anything will just be instantly teleported into safety, superintelligent AI or not), but I don’t doubt that the same results could be achieved via more mundane means. And it’s nice to have some more uplifting visions of the future.
The Choose Your Own Adventure Book or Ghost Ship Model of Will
I first put the core idea into words when someone I met at a workshop said she often had trouble being on time for things. She would notice that it was time to go a meeting soon but that she still had three minutes, so she could, keep reading her novel. And then, ten minutes later she’d actually stop and end up late, which she didn’t like. To this, I said something like “Ah. Apparently, your choice points don’t happen very often when you’re reading a novel. So, if you want to avoid being late, maybe you should seize choice points shortly before you need to leave, because you probably won’t get another one in time.”
The metaphor here is that your agency has a structure somewhat more like a choose your own adventure book than the completely free “I can do whatever I want whenever I want” which we often see it as. Of course, the chose your own adventure book metaphor is too constraining; it offers too few choices compared to one’s experience of the real world. Nonetheless, it captures the sporadicalness of the choice points.
Patri Friedman on political views.
In a comment someone implied that one should have “a stable set of political views that are interrelated and coherent to some degree.”
I think I might disagree with this, and thought y’all might find it interesting.
So my claim is that to the degree that political views are describing mechanisms and outcomes in the real world, the real world is so complex that I’m actually not sure that an accurate description would be “interrelated and coherent” to a significant degree.
In fact I’ll hypothesize that most of the time when people choose view B partly based on how related and coherent it is with view A, they are making a worse choice (for building an accurate model of reality) than if they chose view B solely based on how it seems to empirically fit the world.
Coherence is beautiful and appealing, our mind likes simpler models, but except in the few cases where reality has a strong simplicity orientation (laws of physics), generally a move towards simplicity is a move towards a smaller, more impoverished space of models which is thus less likely to be an accurate description of complex reality. You are throwing away degrees of freedom when trying to fit a very irregular curve.
I think the laws of physics & mathematics have spoiled us because they are so universal, present everywhere, extremely important when they apply, and have so much simplicity and elegance. And I feel like I may be becoming (through a combination of reflecting on past idiocy, and getting really into meditation) such a radical empiricist that I view the desire to find simple models for the world as an omnipresent human foolishness.
I will caveat that the legibility & computational tractability of simple models do matter, our brains can only manage a certain size of model, I just think we will generally understand reality better by viewing it with curiosity and openness to it being modeled by incoherent, unrelated sub-models, rather than trying to force it to conform to our current set of (imperfect and incomplete) models.
On Anchor Collapse and Actually Deciding
Say you’re afraid of dogs. You don’t want to be afraid of dogs, of course, because you like dogs and everyone knows that only some dogs are mean. […] and you don’t want to be stupid, so you deny that the other side of the argument even exists. “There’s no reason for it”/”its irrational”/”I have a phobia”. […]
But let’s be real here. Dogs bite. I’ve been bit. If you’re phobic, you’ve probably been bit too. If you give yourself some room to not worry about looking stupid and look at the facts, there’s a reason to be afraid of dogs. You can’t guarantee you won’t get bit again, and getting bit really freaked you out. You really don’t want it to happen again. Once you admit this you can start to frame it as a decision. […]
So you’ve admitted that yes, the dog might bite you, and that would be really bad. But you still want to pet the dog! So you tell me “jimmmy! I want to not be afraid of dogs so I can pet them!”
“So pet the dog”
“But it might bite me!”
“It might”
“But I don’t want it to bite me!
“You don’t. And if it does, it will be real hurty. Have you considered that maybe you shouldn’t pet the dog?”
“But I want to pet the dog!”
“Then pet the dog”
“But it might bite me!”
…And we can go on all day like this. You’re wanting to pet the dog and not be afraid, but you’re also not wanting to get bit. As if there’s anything I can do about it. The risk is part of the territory. […]
And the way people often handle these is to just get sick of the struggle and suppress one side. “Okay, I know its a nice doggy so I’m gonna pretend that I’m okay with risking getting bit when really I’m not and I’ll just suppress that”. Only what they actually say to themselves is more like “I know its safe. I already decided. The fear is irrational and I want it gone.”But that’s not shitting or getting off the pot. That’s not collapsing the anchors. The two desires are still separate, so that’s not actually deciding. […]
But that’s nonsense. Of course you don’t want to get bit. Who wants to get bit? Getting bit is hurty and bad. And you want to pet the doggy. At the same time. Of course you want to pet the doggy. Doggies are cute and nice. And you haven’t let yourself go there because “I can’t have it so I’m not allowed to think it” but you really wish you could pet the dog with no risk of it biting you. It’s the best of both worlds. It would be really nice to pet the dog with no risk of it biting you. […]
The interesting thing is what happens the moment you stop holding the desires apart and experience them both simultaneously. This is collapsing anchors.
And it goes something like this…
I want to pet the doggie, and if I do, I might get bit.
…
…
…
(Seriously, give it a moment. Shit takes time.)
…
Is it worth it?
Am I willing to stick my hand out and pet that dog knowing that there is some chance that the dog is going to bite it?
And then you sigh a bit. And then you’re silent. And you picture not the separate issues of petting (good!) and being bit (bad!) but the combination package of getting to pet the dog but maaaaaybe getting bit. […]
If your answer is yes, then you can say “yes, I want to pet the dog, even knowing that I might get bit. I still want to pet the dog because it’s worth it. I want that package deal where my hand gets bit sometimes.”
Or if your answer is no, then you say “No, I don’t want to pet the dog. It’s not worth the chance of getting bit”. And that’s the end of it. It’s not “but I wish I could pet it and it wouldn’t bite me!” because you know that comes with the territory – it’s a dog and you can’t predict them perfectly. […]
And either way, there’s no conflict. No two separate desires. Just a congruent choice coming from a decision you had not made before.
See also:
- Duncan Sabien’s concept of “Fabricated options“, which is when you don’t collapse the two and insist that of course there has to be a way where you pet the dog and are guaranteed to not get bitten.
- My “Things are allowed to be good and bad at the same time“
- Internal Double Crux, which is a technique for integrating both the “I want to pet the dog” and “I might get bitten”.
Adam Davis on a student with an apparent trauma history.
This semester’s unusual student experience was an office hour in which I heard “I can’t be told I’m wrong. It upsets me, and I freak. I shut down. You have to say things like, ‘there’s another way of looking at it,’ or ‘have you thought about it this way?’”
I tried “You might not know about this… (?)” Nod.
If you were expecting a rant on the lines of “Ach! These young snowflakes today!” keep scrolling. There’s plenty out there.
There is not the least question that this was a person already fragile, damaged, subjected to sustained abuse, who cultivated withdrawal as a firstline coping mechanism. Also not in doubt: without her courageously frank explanation and clear request, I and my colleagues would surely have done something to drive her away. Years ago, when no such conversation would have been imaginable, she’d simply have quit showing up one day, and we’d maybe have wondered, briefly, why.
Oh stop. There’s no prospect of my building her “resilience,” “grit,” whatever word we use to make it feel all right to be callous, by asserting my right — and I do indeed have the right, and the privilege — to say whatever I want to say, however I want to say it. The world is better, not worse, because she declares a need for accommodation, and gets it. We are not weaker. Our precious bodily fluids are just fine.
She’s finishing the semester. That’s big. That’s a real thing.
And here’s another real thing: it occurs to me that what she was asking wasn’t, actually, the least bit unreasonable. In fact, although I can’t specifically recall telling a student, in so many words, “you’re wrong,” mmm, there are lots of things we quite routinely say that come close enough, and there are many fully functional alternatives that get us through the necessaries just fine. Her request makes me reflect on how I communicate with all kinds of people who maybe don’t have the kind of guts and poise and self-understanding she has.
Oh, she is not weak.
Somebody said, “we’re all just walking each other home.” When kindness does not come to us easily or naturally, as it assuredly does not come easily or naturally to me, noop, it’s all the more important to sink effort into it.
There was a lot of debate of this on my Facebook.
Some people were saying that “we should get this person to work on her problem rather than accommodating her in this way”. But actually accommodating her can be an important first step in helping her fix the problem!
Often people with these kinds of issues experience strong shame about it because others send the message that it’s unreasonable/not okay. That shame then makes it harder to deal with the original issue because it’s painful to think about.
If there are people who communicate with their behavior that the person doesn’t need to feel deep shame about their problem, then that actually makes it easier to work on the problem itself.
There was also a bunch of debate about whether this was reasonable, whether it’s even possible to finish a degree without being told you’re wrong, etc.. I think this was a bit ambiguous from the original post. My interpretation was that the specific phrasing of “you’re wrong” was triggering to the student, but she was open to having the same point communicated with a different phrasing, and that e.g. having assignments marked down for problems wouldn’t be an issue. The fact that her lecturer didn’t consider there to be an issue would be in line with this interpretation.
Some people with experience in education also chimed in, pointing out that they’ve never had a reason to say “you’re wrong” to a student – that there’s always a better way of expressing it, and it just seems like common decency not to embarrass a student.
In any case, even if it was the case that “she can’t realistically expect people to accommodate her this way”… if someone has this issue, it’s likely due to something like cPTSD, which can easily take years to recover from. So it’s simultaneously true that it will be a major problem for her until she gets over it, and that getting over it may take very long and require accommodation and external support to get there. That combination of facts sucks, but just saying “she should get over it” isn’t going to solve anything.
How the incels warped my research
I have generally tried to ignore the manosphere. But as an evolutionary psychologist, I’ve found that hard to do. You can hardly read two paragraphs of incel ideology without coming across references to my field.
Louis Bachaud and Sarah Johns recently published a content analysis of manosphere messaging in the journal Evolutionary Human Sciences, explaining the ways in which our research gets appropriated by manosphere circles.
For example, incels maintain a wiki page of scientific citations they claim support their worldview — an annotated bibliography of misogyny. In one case, in a sort of Russian nesting doll of misrepresentation, the incel wiki quotes a paper citing a study of mine as demonstrating that women prefer dominant men — which they further twist into the incel notion that women actually prefer violent men as romantic partners.
Reading this entry, I thought, “That’s odd, I don’t remember ever publishing on dominance preferences — do the incels know my work better than I do?” No. I double-checked: That study didn’t even mention dominance preferences.
Curiously overlooked in this whole wiki section on women’s preferences is the fact that kindness is repeatedly found to be among the most desired qualities in large-scale, cross-cultural studies of mate preferences. […]
Like any biological approach to behavior, evolutionary psychology has always been controversial. In part, this is owing to some truly bad actors in the field. All it takes is some thoughtless tweets or blog posts for the entire field to earn a reputation as a safe space for provocateurs. […] This allows the manosphere to sell its audience a scientific consensus around its ideology that simply does not exist. Its members appropriate and mischaracterize the literature on evolutionary psychology to lend a scientific patina to their hateful, misogynistic, and dangerous ideas.
For instance, incels are obsessed with the “dual mating strategy” hypothesis, a divisive idea that interprets fluctuations in women’s sexual desire as evidence that women have evolved to seek out men with “good genes” at the most fertile point in their menstrual cycle. Incels use this hypothesis to explain, in their eyes, why relationships are doomed: No matter how good a partner you are, women will always be looking to sleep around with someone better.
Part of the problem is that the dual mating strategy hypothesis was indeed a popular idea among evolutionary psychologists until about 2016. After that, it became one of the more prominent epicenters of psychology’s replication crisis, which revealed that large swaths of psychology research were based on unreliable findings. But even before this major setback, the dual mating strategy hypothesis was critiqued by some evolutionary psychologists like my friend and colleague Jim Roney. Nonetheless, Jim’s work gets hardly any play in manosphere circles, and the hypothesis has since morphed into a version quite unlike the one promoted by incels.
At the end of the day, incels attempt to draw from evolutionary theory a power it does not have. Evolution is not destiny. It is a powerful tool for explaining how we came to be who we are today, but it cannot tell us who we should be today or who we can be tomorrow.
“I lost trust”: Why the OpenAI team in charge of safeguarding humanity imploded
Ilya Sutskever and Jan Leike announced their departures from OpenAI, the maker of ChatGPT, on Tuesday. They were the leaders of the company’s superalignment team — the team tasked with ensuring that AI stays aligned with the goals of its makers, rather than acting unpredictably and harming humanity.
They’re not the only ones who’ve left. Since last November — when OpenAI’s board tried to fire CEO Sam Altman only to see him quickly claw his way back to power — at least five more of the company’s most safety-conscious employees have either quit or been pushed out. […]
… the real answer may have less to do with pessimism about technology and more to do with pessimism about humans — and one human in particular: Altman. According to sources familiar with the company, safety-minded employees have lost faith in him.
“It’s a process of trust collapsing bit by bit, like dominoes falling one by one,” a person with inside knowledge of the company told me, speaking on condition of anonymity.
Compare enough statistics, some of them are going to correlate closely just by random chance. For example, the popularity of the first name “Eleanor” in the US closely correlates with the amount of wind power generated in Poland, r=0.993, p<0.01.
Apparently I organised a student protest against a teacher
This is the best thing I’ve read in a long time. Autistic child who has problems understanding social norms reacts to a mean teacher, without realizing it ends up organizing a student revolt and causing the teacher to get replaced.
First, we need to understand what kind of polity the Old Republic – and thus the Empire – is. And here, the phrasing I go to (somewhat imprecise) is that the Republic was a ‘Republic of Princes’ in the same sense that the Holy Roman Empire was an empire of ‘princes’ or more technically ‘imperial states. […]
In short, the Republic was not a democracy of people but a republic of states, the ‘princes’ which in turn governed their own territory internally. These ‘princes’ could be any form of government. And indeed, the imperial states of the Holy Roman Empire could be noble rulers, but also bishops ruling cities (the ‘prince-archbishops’), monks running abbeys (Imperial prelates), grandmasters running holy orders, and even cities governing themselves (free and imperial cities). So too with the Republic, which is why the Trade Federation can sit on the Senate alongside democratic Naboo and monarchic Alderaan. […]
… what I think a historian of this period, looking back would conclude about the Star Wars story would be this: the Clone Wars were essentially a civil war between the princes of the Rim territories against the princes of the core regions (as the later effectively ruled the senate). That civil war produced political momentum among some of the core princes towards centralization, which fuels the career of Palpatine. Palpatine’s reign and the Empire in general is thus understood as a reaction to the Clone Wars primarily aimed at centralizing power at the expense of the princes. […]
[…] now there are simmering tensions which the Imperial Navy is supposed to tamp down. As a result, imperial designers reach for escalation dominance in their designs, aiming to build ships which can, on their own, intimidate the militaries of the princes – because remember, the ‘princes’ (planetary governments of whatever form) all have their own small navies – in order to avoid a conflict. The [Imperial Star Destroyer] is the end result of that design philosophy: a gun-platform powerful enough to be effectively beyond the ability of any planetary princely navy to fight effectively.
Certain days driving home in Hanoi, metis would take me.
The hundreds of bikes and cars moving unpredictably required of me an intense focus in all directions at once, a broad awareness and an intense focus working as one. If I couldn’t track the speed, direction, and distance of every vehicle behind and ahead of me, and stay open to noticing potholes, sudden braking cars, or swerving buses – I might not survive the drive home. I’d seen enough dead bodies and broken bones on the roads to know that traffic was powerful and indifferent to me, the way the ocean is to sailors, the way the jungle is to hunters.
On certain magic days, when the traffic and my focus merged into a liquid exchange, something would happen and I’d be beyond focus and awareness. Beyond my self and my neck-swiveling calculations of swerving trajectories. On those days, I was a fluid entity of sensory intuition – heat on the side of my face and the thick tang of stale diesel exhaust told me without looking that I had a bus to my left. The honks and revs around me, the way each one muffled, or grew shriller, or faded, became an internal picture of the vehicles around me – how they were rushing up on me, turning to a side street, falling behind… The flicker of red reflection off the edge of my glasses told me the car ahead had tapped its brake lights. The sudden drop of the bike’s engine a few feet away told me they were suddenly braking in reaction.
Wordlessly, thoughtlessly, acting simply as an aspect of the situation pouring around and through me, I banked left and revved to get ahead of the bus, before it could block me off from the gap between it and the truck in front of us. I couldn’t see the traffic ahead, but everything I could see, hear, feel, and smell (the exhaust got a touch thicker, didn’t it?) told me there was a blockage in traffic ahead on the right side – and my experience with these roads told me obstacles like that don’t stay on one side of the road for long, they spread quickly until only a trickle of traffic can make it through the gridlock. I could either break ahead of the mess right now, from the left, or I’d be stuck here for 10 minutes waiting. I slipped through the gap just before the bus closed it, and sped out ahead. Me and the 2 or 3 others who had banked left rushed out into open road as the knot behind us tightened. […]
I wish I didn’t have to say that those dangerous, exhaust-fume-reeking days in Hanoi were some of the greatest peak experiences of my life, but here we are. I have journal entries from that year, long winding devotional prose poems to the Goddess of the Gap – an embodiment of that perfect gap in traffic that moves with divine smoothness, if you can just devote yourself to it and prove yourself worthy of staying in it. […]
I came literal inches from death over and over again in pursuit of it. […]
I drove around Hanoi without a helmet for a long time. I didn’t really understand why. It was stupid, I knew it was stupid. I felt really Alive without it though, and I couldn’t figure out how I could be so smart in so many ways, and so deadly stupid about this – and how even while knowing all this and thinking about it, I kept not wearing a helmet, because some blood-deep devotion to the Goddess of the Gap was somehow making me Alive, waking up some latent essence that had been sleeping inside me my whole life.
The arrow doesn’t fly if the bowstring is never pulled taut. Without tension – true, dangerous tension – you never even get the opportunity to hit the target.
A Woman Who Left Society to Live With Bears Weighs in on “Man or Bear”
When I’m alone in the backcountry and come across a man, I feel a very low level of vigilance. Depending on the situation, I might even be happy to see him. He’s a fellow human! Maybe we’ll be friends! I’m likely to smile genuinely and say hello.
I don’t feel afraid, but I am aware. As we chat, my intuition absorbs a thousand things at once. His body language. His tone. How he looks at me and interacts. Most of the time, this produces an increased sense of security. Most men are friendly, respect my boundaries, and don’t want to hurt me. Most of the time, I feel very safe around men.
But not all the time. Sometimes, my intuition absorbs things that increase my level of vigilance. […] It could be something he says. Maybe he makes a comment about my body or my appearance. Or he asks if I’m carrying a weapon and then presses for details about where I’m camping that night. Sometimes, it’s a shift in his tone, a leer, the way he puts his body in my space. But, usually, it’s a combination of things, a totality of behaviors that add up to a singular reality: this man is either not aware that he’s making me uncomfortable, or he doesn’t care. Either way, this is the danger zone. Even if he has no intention of harming me, the outcome of that intention is no longer possible for me to assess or predict.
In this moment, my mind snaps into a single, crystalline point of focus. My intuition rises to the surface of my skin. I become a creature of exquisite perception. The world is a matrix of emotional data: visceral, clear, direct.
I need to get away from the man. But I need to do it in a way that doesn’t anger him. This is the tricky bit. Men who lack social awareness or empathy often also lack other skills in emotional management. And usually, what men in these situations actually want is closeness. They’re trying to get closer to me, physically or emotionally, in the only way they know how. That combination of poor emotional skillsets and a desire to get closer is exactly what puts me in danger.
If I deny his attempts at closeness by leaving or setting a boundary, he could feel frustrated, rejected, or ashamed. If he doesn’t know how to recognize or manage those feelings, he’s likely to experience them as anger. And then I’m a solo woman stuck in a forest with an angry man, which is exactly what women are most afraid of.
There’s no time to think, so I operate on instinct. My task is ridiculously complex. I need to deescalate any signs of aggression, guide the man into a state of emotional balance, and exit the situation safely, all at once. This process requires all of my attention, energy, and intellect. It’s really hard.
I’ve been in this position so many times that it exhausts me just to write about it. Sometimes, it’s not that I’m afraid of men; I’m just really, really tired.
Spencer Greenberg on distributions and personality traits
Important but often overlooked: when groups differ a small amount in their means, they may differ *dramatically* in their tails.
For instance, in a personality study, we found males to have a little bit lower average compassion score than females (1.4 vs. 2) […]
Small differences like this in averages are typically not noticeable or important. Most people are somewhere near the middle.
If you knew only someone’s compassion level and had to guess their sex from it, you’d be wrong more than one-third of the time (predicting optimally)!
However, small differences in means can lead to much bigger differences in the “tails” (i.e., way on the right or way on the left of the chart). In other words, whereas the percentage of people just above the mean (or just below it) may come from the two groups in roughly equal proportions, the percentage of people who have very high levels of the traits (or very low levels of it) may come from just one of the two groups most of the time.
To see this happening for the example of compassion: despite only a small difference in mean compassion levels between males and females, among just the most compassionate people in our study, there were about 2x more females than males […].
Moreover, the least compassionate people […] were almost all males! […]
Similarly, on average, females usually test only a bit higher than males on peacefulness and forgiveness.
But, if we look at the tails of behavior, we see extreme differences. Males accounted for 96% of U.S. mass shootings and 90% of homicide convictions.
I suspect that one reason so many people believe that groups differ much more, on average, than they really do (and engage in dichotomizing and stereotyping) is that tail behavior is sometimes much more visible than typical behavior.
When you meet most people, you don’t really think about whether their compassion level is slightly above average or slightly below average (and then correlate it with sex). You just wouldn’t even notice one way or the other.
But when you see that the vast majority of serial killers are male, that stands out.
Most males are not very low in compassion. But most people who are very low in compassion are males!
For instance, ~4x more males than females have psychopathy/sociopathy.
Suomeksi (In Finnish)
Paljon puhetta tyhjästä – Tekoäly ja tunteet, vieraana Kaj Sotala.
Olin vieraana Paljon puhetta tyhjästä -podcastissa, kolmena isona pääteemana tekoälyn uhkakuvat, tekoälyn mahdollisuudet, sekä mielen rakenne ja toiminta. Ei tullutkaan kuin neljän tunnin keskustelu.
Karkea sisällysluettelo:
0:00:00 – Intro
0:04:14 – Transhumanismi, teknologinen singulariteetti, tekoälyn uhkakuvat
2:06:03 – Tekoälyn myönteiset mahdollisuudet
2:46:40 – Mielen rakenne ja toiminta
(Käytänpä mä paljon “silleen” -sanaa.)
Sosiaaliturvaleikkausten vaikutukset työnteon kannustimiin.
Vuonna 2024 voimaan tulevat sosiaaliturva- ja veromuutokset muuttavat melko paljon Kelan työttömyysetuuksia saavien kotitalouksien käytettävissä olevia tuloja ja työnteon rahallisia kannustimia. Esimerkkitalouksia koskevan tarkastelumme perusteella kannustimet työskentelyyn pienillä palkoilla tai osa-aikaisesti heikkenevät. Kun toimeentulotuki huomioidaan, kannustimet voivat heikentyä jopa yllättävän suurilla, yli mediaanitulon palkoilla.
Why I no longer identify as transhumanist
Someone asked me how come I used to have a strong identity as a singularitarian / transhumanist but don’t have it anymore. Here’s what I answered them:
——-
So I think the short version is something like: transhumanism/singularitarianism used to give me hope about things I felt strongly about. Molecular nanotechnology would bring material abundance, radical life extension would cure aging, AI would solve the rest of our problems. Over time, it started feeling like 1) not much was actually happening with regard to those things, and 2) to the extent that it was, I couldn’t contribute much to them and 3) trying to work on those directly was bad for me, and also 4) I ended up caring less about some of those issues for other reasons and 5) I had other big problems in my life.
So an identity as a transhumanist/singularitarian stopped being a useful emotional strategy for me and then I lost interest in it.
With regard to 4), a big motivator for me used to be some kind of fear of death. But then I thought about philosophy of personal identity until I shifted to the view that there’s probably no persisting identity over time anyway and in some sense I probably die and get reborn all the time in any case.
Here’s something that I wrote back in 2009 that was talking about 1):
—
The [first phase of the Excitement-Disillusionment-Reorientation cycle of online transhumanism] is when you first stumble across concepts such as transhumanism, radical life extension, and superintelligent AI. This is when you subscribe to transhumanist mailing lists, join your local WTA/H+ chapter, and start trying to spread the word to everybody you know. You’ll probably spend hundreds of hours reading different kinds of transhumanist materials. This phase typically lasts for several years.
In the disillusionment phase, you start to realize that while you still agree with the fundamental transhumanist philosophy, most of what you are doing is rather pointless. You can have all the discussions you want, but by themselves, those discussions aren’t going to bring all those amazing technologies here. You learn to ignore the “but an upload of you is just a copy” debate when it shows up the twentieth time, with the same names rehearsing the same arguments and thought experiments for the fifteenth time. Having gotten over your initial future shock, you may start to wonder why having a specific name like transhumanism is necessary in the first place – people have been taking advantage of new technologies for several thousands of years. After all, you don’t have a specific “cellphonist” label for people using cell phones, either. You’ll slowly start losing interest in activities that are specifically termed as transhumanist.
In the reorientation cycle you have two alternatives. Some people renounce transhumanism entirely, finding the label pointless and mostly a magnet for people with a tendency towards future hype and techno-optimism. Others (like me) simply realize that bringing forth the movement’s goals requires a very different kind of effort than debating other transhumanists on closed mailing lists. An effort like engaging with the large audience in a more effective manner, or getting an education in a technology-related field and becoming involved in the actual research yourself. In either case, you’re likely to unsubscribe the mailing lists or at least start paying them much less attention than before. If you still identify as a transhumanist, your interest in the online communities wanes because you’re too busy actually working for the cause. (Alternatively, you’ve realized how much work this would be and have stopped even trying.)
This shouldn’t be taken to mean that I’m saying the online h+ community is unnecessary, and that people ought to just skip to the last phase. The first step of the cycle is a very useful ingredient for giving one a strong motivation to keep working for the cause in one’s later life, even when they’re no longer following the lists.
—
So that’s describing a shift from “I’m a transhumanist” to “just saying I’m a transhumanist is pointless, I need to actually contribute to the development of these technologies”. So then I tried to do that and eventually spent several years trying to do AI strategy research, but that had the problem that I didn’t enjoy it much and didn’t feel very good at it. It was generally a poor fit for me due to a combination of various factors, some including “thinking all day about how AI might very well destroy everything that I value and care about is depressing”. So then I made the explicit decision to stop working on this stuff until I’d feel better, and that by itself made me feel better.
There were also various things in my personal psychology and history that were making me feel anxious and depressed, that actually had nothing to do with transhumanism and singularitarianism. I think that I had earlier been able to stave off some of that anxiety and depression by focusing on the thought of how AI is going to solve all our problems, including my personal ones. But then it turned out that focusing on therapy-type things as well as focusing on my concrete current circumstances were more effective for changing my anxiety than depression than pinning my hopes on AI. And pinning my hopes on AI wouldn’t work very well anymore anyway, since AI now seems more likely to me to lead to dystopian outcomes than utopian ones.
My idea of sacredness, divinity, and religion
Here’s a conception that I have about sacredness, divinity, and religion.
There’s a sense in which love and friendship didn’t have to exist.
If you look at the animal kingdom, you see all kinds of solitary species, animals that only come together for mating. Members of social species – such as humans – have companionship and cooperation, but many species do quite well without being social.
In theory, you could imagine a world with no social species at all.
In theory, you could imagine a species of intelligent humanoids akin to Tolkien’s orcs. Looking out purely for themselves, willing to kill anyone else if they got away with it and it benefited them.
And then in another sense, some versions of love and friendship do have to exist.
Social species evolved for a reason: cooperation does pay off. Individuals who band together and help each other out even when that requires making the occasional personal sacrifice actually do have greater success.
If no social species existed yet, but the conditions were right for them to evolve, then given enough time it would be inevitable for them to evolve.
Think about that for a second. It would be inevitable for them to evolve. Why? Because of how the laws of natural selection and game theory work. There’s a path from pure selfishness to cooperation, and cooperation is beneficial.
It’s not a given that any single species that evolved sociality would be successful – they could still get unlucky and be wiped out by something completely different. Sociality won’t help you if the only valley you live in gets covered by a volcanic eruption.
But sociality would still keep evolving, over and over again, until some social species would manage to maintain themselves.
There’s a sense in which the laws of natural selection and game theory are deeper than the laws of physics. Cooperation would likely still be beneficial in universes with five physical dimensions, or where life was based on elements very different from carbon, or ones where D&D-style magic was real. Assuming that something like life could evolve at all, cooperation would be beneficial in a broad range of possible universes.
And where do the laws of natural selection and game theory come from? At their heart, they are mathematical laws, inevitably following some given axioms. There are universes where those axioms wouldn’t necessarily be true – ones where no organisms could exhibit heritable variation, for instance. In those worlds, nothing could evolve.
But if the “structure of reality” refers to our laws of physics, then the laws that create companionship somehow go even deeper than the structure of reality. Across a vast range of possible realities, something akin to companionship must come into existence. Somehow, this is just an intrinsic fact about something that governs everything that could ever exist.
In Egyptian mythology, the first god Atum created himself out of the waters of chaos. Likewise, the structure-of-that-deeper-than-reality is such that across many many possible universes, cooperation and companionship will create themselves out of the waters of selfishness.
Love doesn’t always win. There are situations where loyalty, cooperation, and love win, and there are situations where disloyalty, selfishness, and hatred win. If that wasn’t the case, humans wouldn’t be so clearly capable of both.
It’s possible for people and cultures to settle into stable equilibria where trust and happiness dominate and become increasingly beneficial for everyone, but also for them to settle into stable equilibria where mistrust and misery dominate, or anything in between.
When I look around in my room, it’s filled with the fruits of cooperation. For one, the very fact that I even live in a building that someone else built, surrounded by roads that someone else built, filled with everything from books to sophisticated electronics to everything else that I did not personally create. And yes, some amount of coercion and exploitative labor practices no doubt played a role in the creation of some of that, but it’s still our nature as a social species that allows us to have any amount of specialization and the creation of such things in the first place.
To me, if anything deserves the label “divine”, it’s this force of cooperation that transcends universes. Everywhere I look, I am surrounded by artifacts that are of divine origin. Even discarded food packaging is a divine artifact.
And I have friends and I have people who I love, I have happiness. I live in a local pocket of the universe where even my interactions with complete strangers – such as the salespeople at stores – tend to be friendly and warm in tone, or at least politely neutral.
All that because of a cosmic force of love and cooperation which happens to be predominant where I live, whose energy and nature shapes much of my psyche – even as I also carry the energy and nature of unhealthy selfishness and destructiveness in the shape of much of my psyche.
If there is a multiverse, if there are universes beyond our own – then those two forces are locked in an eternal struggle across all the universes that are capable of supporting something like intelligent life. The dance of good and evil within me, in some form mirroring the dance of good and evil across the entire multiverse.
To get even more mystical about it – to me, this doesn’t feel like just an abstraction. I’ve experienced altered states of consciousness – induced by things like meditation, therapy and ritual – where I have felt something like an energy pattern of pure love, confidence and okayness arising through my body, experienced the term “uplifting” as a literal force pulling me up on my feet.
There is a feel to all that, a particular kind of – for a lack of a better word – “energy”, that feels different from the kind of “energy” that comes up when I am being insecure, or selfish in a bad way, or subtly manipulative. Now being able to notice the difference doesn’t always mean that I’d be capable of acting any differently – I can recognize that the energy of what I’m doing doesn’t feel good, and still end up acting according to it since it’s the only energy I have available at the moment.
But feeling that energy does at least let me know that I could be doing better. It does let me know that my actions are not in the service of the kind of power I would like to be serving.
For that’s what it feels like when I am best able to tap into it. Like I am in the service of a greater power, that is by its nature impersonal but takes a human form by channeling itself through me as one of its many vessels.
This has a religious and mystic flavor to it, but not the kind of religious that would require belief in anything that would be called supernatural or contradictory to science. It is the kind of religious and mystic practice that Eric Raymond describes in his essay Dancing with the Gods: one grounded in experience, not belief. As Raymond puts it, we do not need to “believe” without evidence that the sun rises in the morning; we can experience it ourselves.
In the same way, I do not need to “believe” in a god of which I have no direct experience; I can feel it in myself and my body, see what it does to my behavior and state of mind, and call the thing-which-is-all-that a god or divine power that I would like to see myself in the service of.
The 99% principle for personal problems
Often when people are dealing with an issue – emotional, mental, or physical – there’s genuine progress and the issue becomes less and seems to go away.
Until it comes back, seemingly as bad as before.
Maybe the person developed a coping mechanism that worked, but only under specific circumstances. Maybe the person managed to eliminate one of the triggers for the thing, but it turned out that there were other triggers. Maybe the progress was contingent on them feeling better in some other way, and something as seemingly trivial as sleeping worse brought it back.
I’ve been there many times. It is often very, very frustrating. I might feel like all the progress was just me somehow perpetuating an elaborate fraud on myself, and like all efforts to change the thing are hopeless and it will never go away.
And I know that a lot of other people feel this way, too.
Something that I tell my clients who are experiencing this despair is something that I got from Tucker Peck, that I call the 99% principle:
The most important step is not when you go from having the issue 1% of the time to 0% of the time, but when you go from having the issue 100% of the time to 99% of the time.
It’s when you go from losing your temper or going into a fawn reaction in every disagreement, to staying cool on some rare occasions.
It’s when you go from always procrastinating on an unpleasant task, to sometimes tackling it head-on.
It’s when you go from always feeling overwhelmed by anxiety to having some moments where you can breathe and feel a bit more at ease.
When you manage to reduce the frequency or the severity of the issue even just a little, that’s the beginning of the point where you can make it progressively less. From that point on, it’s just a matter of more time and work.
Of course, not all issues are ones that can ever be gotten down to happening 0% of the time, or even 50% of the time. Or even if they can, it’s not a given that the same approach that got you to 99%, will get you all the way to 0%.
But even if you only get it down somewhat. That somewhat is still progress. It’s still a genuine improvement to your life. The fact that the issue keeps occurring, doesn’t mean that your gains would be fake in any way.
And also, many issues can be gotten down to 0%, or close to it. Over time both the frequency and severity are likely to decrease, even if that might be hard to remember in the moments when the thing gets triggered again.
For many issues, it can be the case that the moment when it finally goes to 0% is something that you won’t even notice – because the thing had already become so rare before, that you managed to forget that you ever even had the problem.
This work by Kaj Sotala is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
In addition to making a public comment, you may also send me anonymous feedback.