The new generation of artificial intelligence tools for image synthesis are so good that they are already being used everywhere, from music albums to magazine covers.
But this is the first time I’ve ever seen one of them used to turn full songs into images, and the results are so perfect that I’m convinced that the real killer application for this technology is sleeping inside David Bowie’s music archive.
Don’t believe me? Here’s Bowie’s “Space Oddity,” created by YouTube user aidontknow. Their process involved typing the song lyrics into the AI prompt, verse by verse, to generate each image for the video.
See what I mean?
Sure, the likes of Midjourney, Dall-E, and Stable Fusion will one day leave the uncanny valley behind and provide a never-ending source of perfect stock photos that will look as convincing as the real thing. But there’s also a big downside to all this fun: These tools are essentially mining the existing work of designers and photographers who receive no AI licensing fees in a rapidly evolving field that will have negative downstream effects for the industry.
Right now, this video demonstrates that AI’s often otherworldly results are still limited: well-suited for poetry and fantastic literature, but not anything too literal. That seems to be the idea that moved aidontknow to turn some of Ziggy Stardust’s songs into music slideshows.
Aidontknow used the same methodology inside Midjourney to create this rendition of “Starman.” The video description says that there are minimal changes and influence from the author, which I found to be necessary when I tried to do the same using Midjourney in Discord.
Sometimes a verse will use the word it to reference an object or person named earlier in the song, for example, so I needed to swap it back to the subject’s more specific name so that the AI could identify what “it” was.
I only tried a few verses per song to get an idea of how well it worked with different tracks. But using Midjourney for the task felt exhilarating: All I did was paste the lyrics, and the images slowly materialized in front of my eyes without any other effort, like a genie pluming out of a lamp.
And indeed, I found out that there seems to be a clear correlation between beauty and surrealism. The images for Tom Waits’s “Alice” (results pictured over this paragraph) were as magical as the song itself:
It’s dreamy weather
We’re on [first image]
You waved your crooked wand
Along an icy pond
With a frozen moon [second image]
A murder of silhouette
Crows I saw [third image]
And the tears on my face
And the skates on the pond
They spell Alice [fourth image]
I got similarly fantastical results for the Beatles’ “Across the Universe” and the Rolling Stones’ “Gimme Shelter,” which also have pretty visually evocative lyrics. The magic also worked for the poppy R&B riffs in Snoop Dogg’s “Drop It Like It’s Hot.”
Sometimes, the AI made some errors in the literal interpretation of the words. Like in Waits’s “Tom Traubert’s Blues,” the line “See ya tomorrow, hey Frank can I borrow/ A couple of bucks from you?” turned money into literal male deer. (A “buck” I guess.) Which, mind you, actually adds to the lyricism of it all. Like Bob Ross would say: “There’s no mistakes. Just happy accidents.”
