Technology will destroy human nature
Scott recently made two posts [1 2] about some of the dangers of technology, and of becoming too powerful for yourself. Now, I’ll admit that I didn’t entirely understand his concern. As far as I could tell, he was worried that at some point, we might perfectly know the best possible strategy for pursuing all of our desires, and have the willpower to do so. Then, in a sense, one could say that we’d no longer experience having a free will. There would always be only one reasonable action in any situation, and we would always pick that one.
Well, I’m not too concerned about that. But the post hilighted one possible way that technology could damage something that we consider dear and essential, by removing essential constraints. That’s actually a rather major worry, and a far broader one than just one example suggests. (This essay was also influenced by a recent comment by Randal Koene.)
First, though, let’s review a bit of history.
In 1967, the biologist Sol Spiegelman took a strand of viral RNA, and placed it on a dish containing various raw materials that the RNA could use to build new copies of itself. After the RNA strands had replicated on the dish, Spiegelman extracted some of them and put them on another dish, again with raw materials that the strands could use to replicate themselves. He then kept repeating this process.
No longer burdened with the constraints of needing to work for a living, produce protein coats, or to do anything but reproduce, the RNA evolved to match its new environment. The RNA mutated, and the strands which could copy themselves the fastest won out. Everything in those strands that wasn’t needed for reproduction had just become an unnecessary liability. After just 74 generations, the original 4,500 nucleotide bases had been reduced to a mere 220. Useless parts of the genome had been discarded; the viral RNA had now become a pure replicator, dubbed “Spiegelman’s monster”. (Source.)
What happens in evolution is that organisms adapt themselves to exploit, and protect themselves from, the various regularities of the environment. Light reflects off distant objects in a predictable manner, so creatures have evolved eyes that they can use to see. If the environment ceases to possess some regularities, it will necessarily change the organisms. Put a fish with eyes in a cave with no light, and it will lose its sight over a few thousand years at most. Even humans have kept evolving as our environment has changed. Sickle-cell disease is more common in people whose ancestors are from regions with malaria. A single sickle-cell gene makes you more resistant to malaria, but two give you the disease. That’s an acceptable tradeoff in an environment with a lot of malaria, but a burden outside that environment.
You could say that the environment constrains the kind of organisms that can exist there. Now, those constraints aren’t immediate: that cave fish won’t lose its eyes right away. But over enough time, as different kinds of fish compete for survival, the ones which don’t waste their energy on growing useless eyes will win out.
Humans, as I was suggesting before, have also evolved to meet some very specific environmental constraints. As our environment has changed – either by our own doing, or due to reasons that have nothing to do with us – those constraints have changed somewhat, and we have changed with them. But many things about our nature, things that we might consider fundamental, have not changed. We still tell stories, enjoy the company of others, and are distinct individuals. Sure, the exact forms that those things take have changed over time. Today we are more likely to watch a story on TV than to hear one over a campfire – but both are still recognizable forms of story-telling. Countless of human universals are found in cultures all over the planet:
aesthetics; affection expressed and felt; age grades; body adornments; childhood fears; classification of kin; cooking; cooperation; customary greetings; daily routines; dance; distinguishing right and wrong; dreams; emotions; empathy; envy; family (or household); folklore; generosity admired; gossip; hope; hospitality; imagery; jokes; judging others; leaders; likes and dislikes; manipulating social relations; marriage; meal times; mourning; music …
Individuals may disagree about which of those things really are fundamental – whether losing some specific universal would really be a loss – but most people are likely to say that at least some of those things are important and worth keeping.
But as technology keeps evolving, it will make it easier and easier to overcome various constraints in our environment, our bodies, and in our minds. And then it will become increasing tempting to become a Spiegelman’s monster: to rid yourself of the things that the loosened constraints have made unnecessary, to become something that is no longer even remotely human. If you don’t do it, then someone else will. With enough time, they may end up ruling the world, outcompeting you like Spiegelman’s monster outcompeted the original, umutated RNA strands.
Exactly what kinds of constraints am I talking about, here? Well, there are several, in a roughly increasing order of severity:
- Not being too powerful for yourself. Scott’s concern: that at some point, we might perfectly know the best possible strategy for pursuing all of our desires, and have the willpower to do so. Then, in a sense, one could say that we no longer experienced having a free will – there would always only be one reasonable action in any situation, and we would always pick that one.
- Having distinct minds. We might not be too far away from having the ability to directly connect brains with each other. I think about something, and the thought crosses over to your brain, merging with your stream of consciousness. With time, this technology could be perfected so that large groups of people could join together into a single entity, coordinating and doing everything much better than any “traditional” human. Combined with an ability to copy memories, the concept of “personal identity” might cease to have any meaning at all – there would be no persons, just an amorphous mass of consciousnesses all sharing most of the same memories.
- Unmodifiable desires: as desire modification becomes possible, anyone could reprogram their brains to be constantly perfectly satisfied and never do anything else (except possibly the bare minimum needed for survival). Sure, the possibility feels unappealing now… but maybe you’re having a bad day, and you choose to modify your brain to feel just a little better, all the time. And then the thought of being permanently blissed out doesn’t feel so bad after all, and you modify your brain just a little more… how could you not envy the folks who are never unhappy, especially since the option to self-modify is always there?
- Inability to design superintelligent AGIs. We are constantly investing in ever-improving AI, for the obvious economic reasons: it allows for ever-more work to be automated. It may indeed prove impossible to regulate AI development in order to stop super-intelligent AGIs (artificial general intelligences) from arising. If so, then it might also prove impossible to ensure that safe and human-friendly AGIs prevail: like with Spiegelman’s monsters, the AGIs not burdened with the constraints of respecting human life and property may end up winning the AGIs that wish to protect humanity, after which they’ll recycle human settlements into their raw materials.
- An inability to become mindless outsourcers. Nick Bostrom suggests a scenario where we learn to offload all of our thought to non-conscious external programs. To quote: “Why do I need to know arithmetic when I can buy time on Arithmetic-Modules Inc. whenever I need to do my accounts? Why do I need to be good with language when I can hire a professional language module to articulate my thoughts? Why do I need to bother with making decisions about my personal life when there are certified executive-modules that can scan my goal structure and manage my assets so as best to fulfill my goals?” And so, we give in to the temptation to cut away more and more parts of our brains, letting computer programs run those tasks… until there is no conscious experience left.
- An inability to copy the best workers, choosing only the ones best fit for their tasks. If we could upload brains to computers, it could also become possible to copy minds. This could be far quicker than ordinary reproduction, making copying the primary method by which humans multiplied – and one’s ability to acquire and retain more hardware to run one’s copies on, would become the main criteria that evolution selected for. As Nick Bostrom writes: Much of human life’s meaning arguably depends on the enjoyment, for its own sake, of humor, love, game-playing, art, sex, dancing, social conversation, philosophy, literature, scientific discovery, food and drink, friendship, parenting, and sport. We have preferences and capabilities that make us engage in such activities, and these predispositions were adaptive in our species’ evolutionary past; but what ground do we have for being confident that these or similar activities will continue to be adaptive in the future? Perhaps what will maximize fitness in the future will be nothing but non-stop high-intensity drudgery, work of a drab and repetitive nature, aimed at improving the eighth decimal of some economic output measure. Even if the workers selected for in this scenario were conscious, the resulting world would still be radically impoverished in terms of the qualities that give value to life.
To rephrase what I have been saying:
“Humans” inhabit a narrow region in a multidimensional space of possibilities, and various constraints currently keep everyone stuck in that tiny space. If any of those constraints were to be relaxed – the space of possible minds stretched in any direction – then the new kinds of minds, no longer burdened with the constraints that make our fundamental values so adaptive, would be free to expand in entirely new directions. And it seems inevitable that, given a broader space of possible adaptations, evolutionary pressures would eventually lead to the dominance of minds – or at least replicators – which were very different from what most of us would value as “human”.
Get enough of one constraint, and you might still recognize the outcome as having once been human. Get rid of enough constraints, and you’ll get the equivalent of a Spiegelman’s monster, no longer even remotely human.
There have been some suggestions of how to avoid this. Nick Bostrom has suggested [1 2] that we create a “singleton”, a world-order with a single decision-making agency at the highest level, capable of controlling evolution. The singleton could be an appropriately-programmed AGI, the right group of uploads, or something else. Maybe this will work: but I doubt it. I expect all such efforts to fail, and humanity to eventually vanish. Possibly within my lifetime, if we’re unlucky.
I’ll conclude this essay with the immortal words of H.P. Lovecraft:
The most merciful thing in the world, I think, is the inability of the human mind to correlate all its contents. We live on a placid island of ignorance in the midst of black seas of infinity, and it was not meant that we should voyage far. The sciences, each straining in its own direction, have hitherto harmed us little; but some day the piecing together of dissociated knowledge will open up such terrifying vistas of reality, and of our frightful position therein, that we shall either go mad from the revelation or flee from the deadly light into the peace and safety of a new dark age.
Needless to say, Lovecraft was being too optimistic.
Introduction to Connectionist Modelling of Cognitive Processes: a chapter by chapter review
This chapter by chapter review was inspired by Vaniver’s recent chapter by chapter review of Causality on Less Wrong. Like with that review, the intention is not so much to summarize but to help readers determine whether or not they should read the book. Reading the review is in no way a substitute for reading the book.
I first read Introduction to Connectionist Modelling of Cognitive Processes (ICMCP) as part of an undergraduate course on cognitive modelling. We were assigned one half of the book to read: I ended up reading every page. Recently I felt like I should read it again, so I bought a used copy off Amazon. That was money well spent: the book was just as good as I remembered.
By their nature, artificial neural networks (referred to as connectionist networks in the book) are a very mathy topic, and it would be easy to write a textbook that was nothing but formulas and very hard to understand. And while ICMCP also spends a lot of time talking about the math behind the various kinds of neural nets, it does its best to explain things as intuitively as possible, sticking to elementary mathematics and elaborating on the reasons of why the equations are what they are. At this, it succeeds – it can be easily understood by someone knowing only high school math. I haven’t personally studied ANNs at a more advanced level, but I would imagine that anybody who intended to do so would greatly benefit from the strong conceptual and historical understanding ICMCP provided.
The book also comes with a floppy disk containing a tlearn simulator which can be used to run various exercises given in the book. I haven’t tried using this program, so I won’t comment on it, nor on the exercises.
The book has 15 chapters, and it is divided into two sections: principles and applications.
Principles
1: ”The basics of connectionist information processing” provides a general overview of how ANNs work. The chapter begins by providing a verbal summary of five assumptions of connectionist modelling: that 1) neurons integrate information, 2) neurons pass information about the level of their input, 3) brain structure is layered, 4) the influence of one neuron on another depends on the strength of the connection between them, and 5) learning is achieved by changing the strengths of connections between neurons. After this verbal introduction, the basic symbols and equations relating to ANNs are introduced simultaneously with an explanation of how the ”neurons” in an ANN model work.
2: ”The attraction of parallel distributed processing for modelling cognition” explains why we’re supposed to be interested in these kinds of models in the first place. It elaborates on some interesting characteristics of ANNs: the representation of knowledge is distributed over the whole network, they are damage resistant and fault tolerant, and they allow memory access by content. All of these properties show up in the human brain, but not in classical computer programs. After briefly explaining these properties, there is an extended example of an ANN-based distributed database storing information about various gang members. In addition to being content addressable, it also shows typicality effects – it can be asked a question like ”what are the members of the gang ‘Sharks’ like”, and it will naturally retrieve information about their typical ages, occupations, educational backgrounds, and so forth. Likewise, if asked to return the name of a pusher, it will suggest the name of the most typical pusher. In addition to explaining what the model is like, this chapter also explains the reasons for why it works the way it does.
3: ”Pattern association” describes a specific kind of an ANN, a pattern associator, and a particular kind of learning rule, the Hebb rule. Pattern associators are networks which are presented with a certain kind of pattern as input and a certain kind of pattern as output, after which they will learn to transform the input pattern to the output pattern. They are capable of generalization: if they encounter an input which is similar to ones they have encountered before, they will produce a similar output. They are also fault tolerant, in that they can produce good results even if parts of the network are destroyed. They also automatically perform prototype extraction. Suppose that there is a prototypical ”average” apple, and all other apples are noisy versions of the prototype. Pattern associators presented with several different patterns representing apples will learn to react the most strongly to an apple which is closest to the prototype, even if they have never actually seen the prototype itself.
4: ”Autoassociation” deals with autoassociator networks, and explains how the Delta learning rule works. Autoassociators are a special case of pattern associators – they are taught to reproduce the same pattern at output that was present at input. While this may seem pointless at first, autoassociators are an effective way of implementing a kind of memory: once trained, they can reproduce a complex pattern merely from seeing a small fragment of the original pattern. This has an obvious connection to the human brain, which can e.g. recall a complicated past memory from simply picking up a smell that formed a minor part of the original memory. Autoassociators are also capable of forming categories and prototypes from individual experiences, such as forming a category corresponding to the concept of a dog from seeing several dogs, without explicitly being told that they all belong to the same category. (Or, to put it in Less Wrong jargon, they learn to recognize clusters in thingspace.)
5: ”Training a multi-layer network with an error signal: hidden units and backpropagation” deals with the limitations of single-layered networks and how those limitations can be overcome by using more complex networks that require new kinds of training rules.
6: ”Competitive networks” differ from previous networks in that they can carry out unsupervised learning: while the previous nets had an explicit teacher signal, competitive networks learn to categorize input patterns into related sets on their own. They can perform both categorization, transforming related inputs into more similar outputs, and orthogonalization, transforming similar inputs into less similar outputs.
7: ”Recurrent networks” are capable of doing more than just simple transformations: they have feedback loops and more complicated internal states than non-recurrent networks can. This can be used to create sequences of actions, or to do things like predicting the next letter in a string of words and to identify word boundaries.
Applications
8: ”Reading aloud” can be difficult, especially in a highly irregular language like English, where most rules of how to transform a spelling to sounds have frequent exceptions. A child has to try to discover the regularities in an environment where there are both regularities and many exceptions. This chapter first briefly discusses traditional ”2-route” models of reading aloud, which presume that the brain has one route that uses pronounciation rules to read aloud regular words, and another route which memorizes specific knowledge about the pronounciation of exception words. These are then contrasted with connectionist models, in which there is no distinction between specific information and general rules. ”There is only one kind of knowledge – the weights of the connections which the model has acquired as a result of its experiences during training – and this is all stored in a common network.” The chapter then discusses several connectionist models which are successful in reading words aloud correctly, and which produce novel predictions and close matches to experimental psychological data.
9: ”Language acquisition” ”examines three aspects of language learning by children – learning the past tense, the sudden growth in vocabulary which occurs towards the end of the second year, and the acquisition of syntactic rules”. It describes and discusses various connectionist models which reproduce various pecularities of children’s language learning. For example, some young children initially correctly learn to produce the past tense of the word ”go” as ”went”, then later on overgeneralize and treat it as a regular verb, saying ”goed”, until they finally re-learn the correct form of ”went”. As in the previous chapter, models are discussed which are capable of reproducing this and other pecularities, as well as providing novel predictions of human language learning, at least some of which were later confirmed in psychological studies. The various reasons for such peculiarities are also discussed.
This chapter discusses many interesting issues, among others the fact that vocabulary spurts – dramatic increases in a child’s vocabulary that commonly happen around the end of age two – have been taken as evidence of the emergence of a new kind of cognitive mechanism. Experiments with connectionist models show that this isn’t necessarily the case – vocabulary spurts can also be produced without new mechanisms, as learning in an old mechanism reaches a threshold level which allows it to integrate information from different sources better than before.
10: ”Connectionism and cognitive development” elaborates on the issue of new mechanisms, discussing the fact that children’s learning appears to advance in stages. Traditionally, such qualitative changes in behavior have been presumed to be due to qualitative changes in the brain’s architecture. This chapter discusses connectionist models simulating the apperance of object permanence – the realization that objects continue to exist even when you don’t see them – and the balance beam problem, in which children are asked to judge the direction in which a balance beam loaded with various weights will tilt. It is shown that as the models are trained, they undergo stage-like change like a child, even though their basic architecture remains constant and the same training rule is used.
11: ”Connectionist neuropsychology – lesioning networks” shows that selectively damaging trained networks can closely mimic the performance of various brain damaged patients. Models of damaged performance are examined in the fields of reading, semantic memory, and attention.
12: ”Mental representation: rules, symbols and connectionist networks” discusses and counters claims of connectionist networks never being able to learn some kinds of rules which require one to use rules or symbols, due to having no explicit representation of them.
13: ”Network models of brain function” discusses two models which attempt to replicate brain functionality and experimental data about the brain: a model of the hippocampus, and a model of the visual cortex. Both are shown to be effective. The hippocampus model in good in storing and recalling patterns of data. The visual cortex model, on the other hand, succeeds in identifying faces regardless of their position in the visual field, and regardless of whether the face is seen from the front or from the side.
14: ”Evolutionary connectionism” has a brief discussion of connectionist networks that are trained using evolutionary algorithms. 15: ”A selective history of connectionism before 1986” is pretty much what it sounds like.
———–
In addition to providing a nice analysis of various models originally published in journal papers, the book also provides a bit of a historical perspective. Several of the chapters start with analyzing an early model that produced promising but imperfect results, and then move on to a later model which built on the previous work and developed more realistic results. References are provided to all the original papers, and readers who are interested in learning more about the various topics discussed in the book are frequently referred to sources which discuss them in more depth.
I would recommend this book to anyone who has an interest in psychology, can handle a bit of math, and who isn’t yet familar with all the various and fascinating things that connectionist models are capable of doing. Although the models discussed in the book are all very simple as compared to the ones in the brain, they do make the idea of the brain being successfully reverse-engineered during this century feel a lot more plausible.
Political logic 101
Liberal Logic 101 is an interesting site. As with any political site of this nature, a lot of the pictures and comparisons are just dumb: comparisons picked to make the enemy look bad with no context provided, or outright just stating “liberals boo, conservatives yay” without even trying to offer any kind of justification. But seeing somebody of a different ideology pass these kinds of pictures around as self-evident does help realize how many of the pictures you agree with and see passed around with are similarly vacuous and serving no purpose for any useful debate.
The idealist in me wishes that everyone was made to study lots of these kinds of pictures from a variety of ideologies, to see how silly they all were and to stop passing them around. The realist thinks that most people would just end up thinking “wow, people of [any group I don’t agree with] are really stupid”.
And then there are some pictures that genuinely are clever and funny, in that they point out something inconsistent – or at least something that seems inconsistent if you’re inclined to accept certain premises, such as fetuses being people. Maybe if people looked at these kinds of pictures, they’d see how much of our reasoning depends on accepting various premises (or moral intuitions), and how an argument that seems silly if you reject one premise becomes insightful once you accept it, and there’s no objective reason for accepting or rejecting any particular one. Then again, maybe not…
Also interesting are the “Furious Liberal Friday” pictures. I don’t know whether these are genuine quotes from people or just made up, but many of them do sound like ones that people would make in real life. E.g. here’s the text of one of the Furious Liberal Friday pictures on that page:
“‘I don’t tolerate intolerance. I know that’s a hard concept to grasp, but you can’t have your cake and eat it too.’
(Actual comment left on LiberalLogic101 Facebook page)
(I suppose it would be funny if it weren’t so sad that the person who wrote it still does not get it.)”
I’m not sure how well one can emphasize with that without being part of a frequently misunderstood (political?) movement, but having a lot of experience with being told that political pirates just hate artists and went to get everything for free, this certainly struck a chord. The feeling of frustration-combined-with-hopeless-amusement when somebody makes a claim about you that you feel is completely off-base is certainly very familiar. And I can certainly imagine that most conservatives won’t feel like they’re intolerant, and instead feel that they’re motivated primarily by altruism and rational thought. Just like most liberals feel like they are motivated primarily by altruism and rational thought.
On unhealthy relationships
Clarisse Thorn: How my life wasn’t always Happy Fun Boundaries Are Perfect Land.
“Here is the strange part, for me, in remembering him: I don’t think he consciously wanted me to hurt myself like that. If he had been deliberately abusive, if he had really wanted to tear me apart, if he’d been physically abusive […] Maybe then I would never have gotten involved? Maybe then I would have walked away sooner? But maybe not.”
“Can I teach other people to set boundaries in situations like that? I don’t know. The feminist ideas and gender analysis I was exposed to as a kid didn’t prevent that experience (although, again, maybe those things would have helped if the situation had been more obvious: if he’d been physically abusive, for example, or more overtly controlling).”
I recommend the above article particularly for those with little experience with relationships. There’s a lot about this text and situation that seems familiar to me: in my first relationship, I too should have been better at setting my borders and policing them. And when my partner didn’t properly respect my borders, it wasn’t out of malice either: I have no doubt that she really did love me, but rather just didn’t realize what she was doing, or just couldn’t help being needy when she did.
It was exactly that which made it so hard for me to say no when I should have: had the relationship been openly abusive, I would have realized it pretty quickly, but when I was already in the relationship and my partner needed me and clearly cared about me, how could I have said no? (At least I would have left it had it been openly abusive from the start. It didn’t seem dysfunctional at first, either.) And even if the requests seemed unreasonable, wasn’t it reasonable that I who was better off compromised on what I wanted? And if it was impossible to even raise the issues of what I experienced as unfair without her pretty much breaking down and starting to hate herself, and me being forced to patch her back together without us ever really getting to the point of talking about those issues… then sometimes, that resentment had few other places to go than to turn inward, and I might wonder whether I was the one who should have tried harder.
Some people will know or guess who I’m talking about, so let me emphasize again that I don’t blame her, nor bear her any ill will. Again, I’m sure that she really did care about me, but was just undergoing a really hard time, and was in a really bad shape. She’s better now. We’ve talked about it, she’s sorry about it, and I’ve forgiven her. (I also let her read this text beforehand and made sure that she was okay with me posting it.) And she did teach me to be far more sensitive about my borders, and I think that I’ve done a much better job of setting my limits since then. It’s rare, but sometimes that which doesn’t kill you really does make you stronger.
Would I have done things any differently if I had read this article beforehand? As the author of the article says, when asking herself whether she would have acted differently in case her lover would have been more abusive – maybe not. I was a lonely teenager, being really in love for the first time in his life, after having had experienced many unrequited crushes before. It’s possible, and perhaps even likely, that I would have regardless just tried to do everything to make the relationship work, just as I did back then. But maybe this will help someone else instead.
Of course, none of this is to say that people should dump their partners if the partner is having a difficult time, or that you shouldn’t ever compromise on your desires if you’re clearly better off than your partner is. That’s what makes these issues so hard – there are no clear lines of what to do when. But at least make sure that you really are helping because you genuinely want to help… not because you’re being guilted into it.
This work by Kaj Sotala is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
In addition to making a public comment, you may also send me anonymous feedback.