DeepDream: Today psychedelic images, tomorrow unemployed artists

One interesting thing that I noticed about Google’s DeepDream algorithm (which you might also know as “that thing making all pictures look like psychedelic trips“) is that it seems to increase the image quality. For instance, my current Facebook profile picture was ran through DD and looks sharper than the original, which was relatively fuzzy and grainy.

Me, before and after drugs.

Me, before and after drugs.

If you know how DD works, this is not too surprising in retrospect. The algorithm, similar to the human visual system, works by first learning to recognize simple geometric shapes, such as (possibly curvy) lines. Then it learns higher-level features combining those lower-level features, like learning that you can get an eyeball by combining lines in a certain way. The DD algorithm looks for either low- or high-level features and strengthens them.

Lines in a low-quality image are noisy versions of lines in a high-quality image. The DD algorithm has learned to “know” what lines “should” look like, so if you run it on the low-level setting, it takes anything possible that could be interpreted as a high-quality (possibly curvy) line and makes it one. Of course, what makes this fun is that it’s overly aggressive and also adds curvy lines that shouldn’t actually be there, but it wouldn’t necessarily need to do that. Probably with the right tweaking, you could make it into a general purpose image quality enhancer.

A very good one, since it wouldn’t be limited to just using the information that was actually in the image. Suppose you gave an artist a grainy image of a church, and asked them to draw something using that grainy picture as a reference. They could use that to draw a very detailed and high-quality picture of a church, because they would have seen enough churches to imagine what the building in the grainy image should look like in real life. A neural net trained on a sufficiently large dataset of images would effectively be doing the same.

Suddenly, even if you were using a cheap and low-quality camera to take your photos, you could make them all look like high-quality ones. Of course, the neural net might be forced to invent some details, so your processed photos might differ somewhat from actual high-quality photos, but it would often be good enough.

But why stop there? We’ve already established that the net could use its prior knowledge of the world to fill in details that aren’t necessarily in the original picture. After all, it’s doing that with all the psychedelic pictures. The next version would be a network that could turn sketches into full-blown artwork.

Just imagine it. Maybe you’re making a game, and need lots of art for it, but can’t afford to actually pay an artist. So you take a neural net, feed to it a large dataset of the kind of art you want. Then you start making sketches that aren’t very good, but are at least recognizable as elven rangers or something. You give that to the neural net and have it fill in the details and correct your mistakes, and there you go!

If NN-generated art would always have distinctive recognizable style, it’d probably quickly become seen as cheap and low status, especially if it wasn’t good at filling in the details. But it might not acquire that signature style, depending on how large of a dataset was actually needed for training it. Currently deep learning approaches tend to require very large datasets, but as time goes on, possibly you could do with less. And then you could get an infinite amount of different art styles, simply by combining any number of artists or art styles to get a new training set, feeding that to a network, and getting a blend of their styles to use. Possibly people might get paid doing nothing but just looking for good combinations of styles, and then selling the trained networks.

Using neural nets to generate art would be limited to simple 2D images at first, but you could imagine it getting to the point of full-blown 3D models and CGI eventually.

And yes, this is obviously going to be used for porn as well. Here’s a bit of a creepy thing: nobody will need to hack the iCloud accounts of celebrities in order to get naked pictures of them anymore. Just take the picture of any clothed person, and feed it to the right network, and it’ll probably be capable of showing you what that picture would look like if the person was naked. Or associated with one of any number of kinks and fetishes.

It’s interesting that for all the talk about robots stealing our jobs, we were always assuming that the creative class would basically be safe. Not necessarily so.

How far are we from that? Hard to tell, but I would expect at least the image quality enhancement versions to pop up very soon. Neural nets can already be trained on text corpuses and generate lots of novel text that almost kind of makes sense. Magic cards, too. I would naively guess image enhancement to be an easier problem than actually generating sensible text (which is something that seems AI-complete). And we just got an algorithm that can take two images of a scene and synthesize a third image from a different point of view, to name just the latest fun image-related result from my news feed. But then I’m not an expert on predicting AI progress (few if any people are), so we’ll see.

EDITED TO ADD: On August 28th, less than two months after the publication of this article, the news broke of an algorithm that could learn to copy the style of an artist.

6 comments

  1. Hi there! I just read your article. It was a good read! For the record, I’m responsible for the Magic card project you cited, but that’s not why I’m here (complete coincidence actually!). The DD project and others have also got me thinking about the future of creative work.

    “It’s interesting that for all the talk about robots stealing our jobs, we were always assuming that the creative class would basically be safe. Not necessarily so.”

    In many ways, I think we could have seen this coming. Music and video recordings reduced demand for live performances, creating a situation where fewer and fewer creators captured more and more of the market. I see the automation of the creation of art as a logical progression. Of course, for many artists, their work has never been about the money (if it were, they wouldn’t be doing it). Still, this automation will put downward pressure on wages and commissions, making creative work less appealing. That in turn will create even more demand for automation.

    My concern is that these trends will lead to the suppression of art as a vehicle for social and political identification. There is no such thing as unbiased training data, digital data tends to reflect or encode the tendencies, views, and identities of the majority. I have doubts that artificially created art will tell the stories and experiences of minorities, or if they do, only as an after-thought. Add all that to the fact that minority workers are disproprotionately likely to be displaced by automation, and it all leaves me very concerned that societies could become even more hostile to minorities than they already are.

    • Oh, nice to meet you, and thank you for commenting! I had a lot of fun reading through your list of generated cards. :)

      I’m maybe a bit more optimistic than you – at least if we manage to make the transition into an economy with a guaranteed basic income replacing the need to work. In that case, I could see it benefiting minorities as well. Discrimination would become less of an issue, since a lack of money would no longer force anyone to work in a potentially hostile environment. Also the “they’re taking our jobs” argument that’s currently used against immigrants would disappear.

      Of course, this would basically depend on it becoming easier for people to isolate themselves into self-contained communities and being forced to engage with people different from themselves, and it’s not clear whether that would be good or bad for society overall. But I do kind of like the “Archipelago” ideal, of societal conflicts getting solved by everyone moving to islands with people who shared their values.

      • Pleasure to meet you as well. And I’m happy that you found my side project so entertaining. :D

        I agree with you on the point about basic income. I’m in the same camp as you. However, where I am from (the United States), there are enormous social and political barriers that have to be overcome for basic income to be implemented. I also happen to live in the South, which was and in some ways still is a hotbed for racially-motivated oppression and violence, so I understand that these issues can take generations to be resolved.

      • Yeah, I’m from Finland so I tend to think in terms of my own country, which already has considerable support for a basic income scheme right now. (In fact, the government that was elected this year has “a basic income experiment” in their list of things they intend to do while in power, though this is admittedly only one sentence in a 74-page document.)

        I do expect that there are places where things could get a lot uglier, like the US, but don’t have the expertise to comment on their situation.

  2. Cool! I wonder if in the future this can be used for medical images (like from MRIs) as well; we can keep the hardware simple and cheap by having very powerful post processing software. Although in these cases it will be hard to say a photo is probably “close enough”… :p

  3. Douglas Knight

    I would not say that the particular image has been sharpened. But assuming that these techniques do let you sharpen images. It seems to me that it would only use the bottom layer or two of the network. Thus it would probably not be “deep” and would have been an accessible technique decades ago. (for all I know, such sharpening algorithms are old)

Trackbacks/Pingbacks

  1. Syvä uni | Skrubu.net blog - […] DeepDream: Today psychedelic images, tomorrow unemployed artists. […]

Leave a Reply