Experimentation with AI-generated images (VQGAN+CLIP) | Solarpunk airships fleeing a dragon

A few days ago I found the Twitter account @images_ai, which posts AI-generated images and links to these instructions for generating your own. I started playing around with it; some of my choice picks:

The first image I generated, “sci-fi heroes fighting fantasy heroes”

“cute catboys having a party”

Someone had figured out that if you add words like “unreal engine” to your prompt, you get more realistic graphics.

so here’s “sci-fi heroes fighting fantasy heroes trending on artstation | unreal engine”

“young sorcerer heiress of a cyberpunk corporation, with dragons on her side | unreal engine”

“the cat is over the moon|unreal engine”

“Ghosts of Saltmarsh”

wikiart16384 model, “Fully Automated Luxury Gay Space Communism:2 | unreal engine:1 | logo:-1”

At this point I read a tweet suggesting that a 16:9 image ratio generally produces better results, so I switched to using that.

“solarpunk forest village | studio ghibli | trending on artstation”

“mountain expedition encounters stargate | full moon | matte painting”

“xcom fighting sectoids | matte painting”

I decided to do some systematic experimentation: picking a starting prompt (“solarpunk airships fleeing a dragon”), fixing the random seed, and varying parts of the prompt to see how they changed.

In retrospect the initial prompt wasn’t great, since I’m not sure if any of the pictures really incorporated the “fleeing” aspect, so I could have picked a more fully meaningful prompt. In general, this AI seems to be better at understanding individual words than it is at understanding sentences. Oh well.

All of these use seed 13039289688260605078 (no special significance; it’s what got randomly rolled for the first one so I kept using it), image size 640*360.

I don’t have anything interesting conclude besides what you see from the images themselves. As I was posting these on Facebook, Marcello Herreshoff kindly joined in on the experimentation, coming up with additional prompts and generating pictures with them; I’ve credited his images where they appear. [EDIT: Marcello let me know that he hadn’t remembered to fix the seed for his pictures, but they got similar enough to my pictures anyway that I never even suspected!]

“solarpunk airships fleeing a dragon”

“solarpunk airships fleeing a dragon | Dr. Seuss” (Marcello)”

“solarpunk airships fleeing a dragon | Dr. Seuss | text:-1” (Marcello)

“solarpunk airships fleeing a dragon | paul klee” (Marcello)

“solarpunk airships fleeing a dragon | Salvador Dali” (Marcello)

“solarpunk airships fleeing a dragon | Ukiyo-e” (Marcello)

“solarpunk airships fleeing a dragon | van gogh” (Marcello)

“solarpunk airships fleeing a dragon in the style of h.r. giger”

“solarpunk airships fleeing a dragon in the style of h.r. giger | pastel colors”

“solarpunk airships fleeing a dragon in the style of my little pony friendship is magic”

“solarpunk airships fleeing a dragon | studio ghibli”

“solarpunk airships fleeing a dragon | trending on artstation”

“solarpunk airships fleeing a dragon | studio ghibli | trending on artstation”

“solarpunk airships fleeing a dragon | unreal engine | trending on artstation”

“solarpunk airships fleeing a dragon | watercolor”

“matte painting of solarpunk airships fleeing a dragon”

“matte painting of solarpunk airships fleeing a dragon | trending on artstation”

“matte painting of solarpunk airships fleeing a dragon in the style of h.r. giger | studio ghibli | unreal engine | trending on artstation”

“children’s storybook illustration of solarpunk airships fleeing a dragon”

“solarpunk airships fleeing an octopus | trending on artstation”

“airships fleeing a dragon | trending on artstation”

“airships fleeing a dragon | ps1 graphics”

“airships fleeing a dragon | SNES graphics”

“airships fleeing a nuclear blast | trending on artstation”

“airships | trending on artstation”

“solarpunk trains fleeing a dragon | trending on artstation”

“solarpunk trains fleeing a dragon | ursula vernon”

“cyberpunk airships fleeing a dragon | trending on artstation”

“cyberpunk trains fleeing a dragon | trending on artstation”

“cyberpunk trains fleeing a baby | trending on artstation”

Imaginary reenactment to heal trauma – how and when does it work?

Some therapies involve various forms of imaginary reenactment, where you heal a trauma by first recalling the memory of it and then imagining how things could have gone differently. Sometimes the imagined alternative can be quite fantastical in nature, such as your current adult self traveling back in time to when you were a child and saving your child self from the bullies tormenting you. (Here by trauma I mean to also talk about “small-t trauma”, e.g. various painful experiences that might not be what we’d ordinarily call trauma, but are still a little unpleasant to think about, or have left some other kind of a negative effect on your psyche.)

In my experience, imaginary reenactment works, at least assuming that I’ve managed to get an emotional hold of what exactly in the memory it is that made it feel so unpleasant. (Did I feel like I was alone? Or inadequate? Or that I did something wrong? Etc.) Also assuming that the memory of the old trauma isn’t so painful as to be completely overwhelming and leave no room to imagine any alternatives.

Here’s my current guess of how and when this works:

The basic process by which any emotional learning gets changed is memory reconsolidation. There’s a generalization that your mind has drawn about the meaning of some past event that feels true to you. E.g. “nobody helped me when I was in that situation, so nobody cares about my suffering”. If you can bring that felt truth to mind while also experiencing a contradictory belief – e.g. the belief that you have a friend who does care about you – as true at the same time, your brain will notice that it believes in two contradictory things at the same time, and will revise its beliefs to fix that inconsistency.

Often, this takes the form of concluding that what it considered to be a general truth isn’t the case after all – e.g. changing the previous assessment to “nobody helped me in that situation, but there are still people who care about me and who I can reach out to for help”.

Now, you can also imagine things that feel true, if they’re the kinds of things you feel could happen. For instance, maybe you have a friend who buys a lot of products from the Acme Corporation, and you then imagine your friend excitedly telling you about the Acme Super-Duper Toothbrush that they bought. Even if they have never done this, the imagined scene can still feel real because it involves the kind of a thing that your friend could do.

I suspect what’s going on in therapeutic reenactment is that you are imagining something that feels like it could have happened and thus serves as counterevidence for the emotional belief in your trauma, but the “could be true” is on an emotional or symbolic level rather than on the level of physical possibility.

So for example, suppose that I had a childhood experience where I was being picked on by bullies and nobody helped me. From this experience, my brain might form the generalization “nobody helped me, so nobody cares about my suffering”.

Now if I manage to recall this experience in such a way that I can feel empathy towards my past self, then the act of feeling that empathy now proves that someone does care. Then if I imagine a scene in which I travel back in time to when I was a child and I beat up my bullies, it doesn’t matter if the literal content is physically impossible. Because what matters is the emotional feeling of “someone cares so someone could have helped”, which is evaluated as true.

(My adult self caring about my child self doesn’t mean that my adult self could actually have helped my child self, but one person caring is enough to disprove the generalization of “nobody cares”. So then if at least one person cares, then that implies that there were also others who would have cared, and they would have helped if they’d known and had the opportunity to. More broadly, the reason why we draw generalizations from past experiences is to predict the future, so what really matters is knowing that it’s possible to get help from people in general, and that people don’t think that your suffering is intrinsically meaningless.)

That said, if I try to do this and I haven’t really gotten a good intuition of why the memory is so painful, it usually doesn’t work – the generalization that I have formed from the experience needs to be at least somewhat explicit. Otherwise I can’t experience that generalization as real (as is required for the memory reconsolidation process to work), nor can I find the right emotional flavor that I need to imagine for the new scene to count as counter-evidence for the generalization.

Open loops in fiction

A fiction-writing trick I find particularly compelling are open loops.

A cliffhanger is an example: you want to know how the hero survives, so your thoughts keep looping back to the situation, trying to figure out what happens next. But you need the author to tell you.

Really good writing uses open loops at the sentence level as well. The first sentence of the story is meaningful on its own, but also makes reference to something else; that something else is required to understand the full meaning. That sets up an open loop; I want to know more.

The following sentences provide additional context, helping close the first loop, but in so doing they make reference to other things. Even as loops are closed, new ones get opened. As the author is doing this, various characters are established. Their goals and intentions set up long-term open loops: ones that require a whole chapter or the whole story to resolve.

At any given time there may be a dozen open loops of varying timescales.

As a concrete example, let’s look at a few early sentences from the novel that prompted this thread as I started reading it and appreciating all the loops it was opening: Melanie Rawn’s Dragon Prince.

We open with: “Prince Zeheva squinted into the sunlight and smiled his satisfaction.”

This establishes that our character is a prince and satisfied with something that he sees. But to fully understand the situation, we need to know what he sees that satisfies him. Curious.

A somewhat different sentence would have been much less compelling: “It was a bright day, and Prince Zeheva smiled.”

This sentence would have made it sound like Zeheva was just smiling at the sunlight. He’s not actively looking at something – squinted into – and it feels like we can already understand the situation from this sentence alone. There’s no reference to something else left hanging.

Next sentence: “All the signs were good for the hunt today: claw marks on the cliffs, wing marks on the sand, and the close cropping of bittersweet plants along the canyon ridges.”

This closes the first loop – we now know what the prince was looking at that made him smile. It also opens new ones. What is he hunting, and what is going to happen next? The “claw marks” and “wing marks” suggest to me some griffin.

Notice how this sentence sets up both short- and long-term loops. The most immediate loop is “what is the prince hunting?”; that will get resolved within a few sentences. “How will the hunt go” will be an ongoing one.

“But the prince’s perceptions were more subtle and had no need of these obvious signs.”

This suggests that there’s something special about the prince’s senses. How are they subtle, and why are they special? Those are the third and fourth loops we’ve got open.

“He could feel the presence of his prey all along his skin, scent it in the air, sense it in every nerve. His admirers said he could tell when the time was ripe for the hunt simply by glancing at the sky.”

That resolves the ‘how are his senses subtle’ question. Down to three loops.

“His enemies said it was not surprising that he could sense such things, for he himself had been dragon-spawned.”

We are told that the prince’s senses are special because he’s part dragon. That resolves the loop of “what makes his senses special”, even as the reference to his enemies – who are they? – opens up a new one, keeping us at three.

The second paragraph opens with: “In truth, he seemed a human version of the dragon he hunted today.”

So he’s hunting a dragon. In the space of just one paragraph and one sentence, Rawn opened at least six loops while closing four of them, thus keeping the reader interested. We still don’t know how the hunt will go, so we want to keep reading…

Different kinds of language proficiency

It’s funny how forms asking for your language proficiency use “native language” to mean “best possible proficiency”.

My native languages are Finnish and Swedish, but I’m out of practice with Swedish so my English vocabulary is way better than my Swedish. Though interestingly, speaking either Finnish and Swedish with someone give me a sense of emotional connection that’s lacking if I use English.

I feel I know English words better than Swedish, but Swedish words have a sense of subtle emotional nuance that’s missing from the English ones. So there’s a dimension on which my Swedish does feel better than my English, but it doesn’t seem to translate directly to fluency in the traditional sense.

Something like… the place in my mind that holds Swedish contains less stuff, but also feels like less effort to access.

Mark Lippmann once described “the felt meaning” as “the place your mind goes to when looking for words”. My Swedish place feels closer to where I am, and easier to go to, than my English place – even if the Swedish place is smaller and has more cobwebs around things, and once there, I may need to rummage around to find where the heck I put that one word again.

That sense of closeness also translates to increased emotional closeness when talking with someone in Swedish. Both Finnish- and Swedish-speakers feel like “my people” in some sense. With English, it always feels like there’s some amount of a chasm between us. We can communicate, and we can definitely connect in quite a few ways, but it’s always shouting over a chasm – even if its presence is sometimes easy to forget.