How to selectively modify a Midjourney image to create an artistic statement
This is where we're going to end up, blending my creative ideas with generative capabilities provided by Midjourney and Photoshop.
At the beginning of 2023, we were wowed by AI turning text into a representative image. As the year progressed and we started to use tools like Midjourney, DALL-E 3, and Adobe Photoshop Generative Fill more, we needed to do even more.
Users clamored for the ability to selectively modify part of an image. It was nice that Midjourney (and I'm mostly focusing on Midjourney in this article) could create an image. But it would be nicer if we could get the AI to make some changes to the generated image for a more custom feel.
Also: The best AI image generators to try right now
In the AI image creation world, this capability is often called "inpainting." Midjourney added inpainting capabilities in August 2023, calling it "Vary (Region)."
In this article, I'm going to show you how you can use inpainting to create a custom piece of art.
Nighthawks
I have always loved the 1942 painting Nighthawks by Edward Hopper. In a lot of ways, it reflects my life before I got married. I'm a night person and I spent many a late night in diners and Denny's doing homework or writing. I grew to love the empty quiet and the eclectic folks I met during the wee hours of the night.
Nighthawks
Nighthawks is a painting of a nearly-deserted all-night diner, with a few customers and a server. The Art Institute of Chicago quoted Hopper as saying, "unconsciously, probably, I was painting the loneliness of a large city."
I decided I wanted to create an image that reflected the spirit of Nighthawks, but use inpainting to make some additions that would act as commentary about our modern era.
Generating the starter image
The first step in my quest to reimagine Nighthawks for the 21st century was to create a starter image. For this, I needed a text prompt. I wanted to describe the style, but I didn't want to mention the actual painting, because I wanted Midjourney to generate a new take on it. Here's the prompt I used:
Exterior view of a downtown diner late at night, with a few people inside. Mood of isolation and urban solitude. The diner's large glass windows illuminate a dark, deserted street, enhancing the sense of quiet and introspection. Done in the style of Edward Hopper.
I got back the following set of four images:
I liked the first image, and chose to upscale it. Upscaling is done by clicking one of the U buttons as shown:
This is the starter image I chose to work with:
Enter the Easter Bunny
Bear with me for a moment. We're creating art here. My vision will become apparent over time. It'll be worth it, I promise.
From the beginning of this project, I knew the work needed to include something about smartphones. Initially, I thought I'd do something like put a newsstand outside and have someone with a smartphone, but I just couldn't convince Midjourney to generate a newsstand.
Then I had the aha moment that changed everything for this piece. My wife was out celebrating Easter while I was working on this project. And I thought, "What if I added a giant Easter Bunny?"
I had just showcased Liam Neeson playing the Easter Bunny in my weekly newsletter, and so I must have had giant bunnies on my mind.
Where does inspiration come from? Everywhere.
To do some inpainting in Midjourney, start with the upscaled image you want to use, and click Vary (Region):
You'll be presented with an editing window. There are two tasks you need to do here. First, draw a rectangle or freehand selection around the area where you want to make your modification (shown at 1) and then create a prompt for what you want generated (shown at 2):
I got back four bunny images:
I chose the fourth image:
Unlike the other three, this bunny was looking into the diner. I thought it was perfect. Here was a giant bunny, staring wistfully into the diner. The Easter Bunny is often a symbol of spring and rebirth, and here was a bunny in the cold of night, just wishing it could go inside, into the warmth.
Notice also how it perfectly matches the painting style of the rest of the image, and how it's placed and sized based on my selection rectangle.
Smartphone myopia
Now, it was time for the smartphone. Once I saw how the bunny turned out, I knew what I wanted. I wanted a guy fixated on his smartphone. Like most people using smartphones these days, this person was to be oblivious of his surroundings, which would mean he likely isn't even noticing the giant Easter Bunny standing near him.
I clicked Vary (Region) on the image with the chosen bunny, and got the editing window back:
This time, I selected an area next to the bunny, added my prompt, and sent Midjourney to do its work. I got back this set of options.
The first image was the best for what I wanted, so I upscaled that as well:
Here's my almost finished art project:
There were a few things I wanted to tweak, and while I probably could do them in Midjourney, I used Photoshop because I knew it would make quick work of these bits.
Photoshop the little bits
The first thing I wanted to get rid of was the cryptic neon sign. I wanted the environment to feel dark, except for the phone screen and the inside of the diner. So I zoomed in on the image, selected the sign, and used Generative Fill:
As you can see, it replaced the sign with a painted block that matches the style of the rest of the painting:
Next, I wanted to get rid of the pocket on our smartphone user's pants. There was nothing wrong with the pocket, but it distracted from the look I wanted. Once again, I selected the area and hit Generative Fill:
The result was pants without the distracting pocket.
For the white spots, one of which could be interpreted as the moon, I just used Photoshop's healing brush.
The final reveal
So here's the final image. I love how the man quite possibly doesn't even notice that there's giant white rabbit just a few feet to his right. It's an allegory for the idea that we're so engrossed in our phones that we miss even the most obvious things around us. You also sense that the man would rather just tap on his phone than go in and get a nice piece of pie or a hot beverage, putting off creature comforts in favor of whatever fascinates him so much on that screen.
I also love how the rabbit represents change and newness, but conveys a deep sense of longing, because he can never be inside and part of the diner milieu.
It's very much evocative of Hopper's Nighthawks, but it also stands alone. This piece shows how human creativity plus AI can create a new piece of art. We started with my vision of creating an updated Nighthawks-style image and used the painterly skills of Midjourney to add the bunny, the phone man, and the overall commentary.
Frankly, I think it was a great creative partnership. The final result is something that's well beyond my painterly skillset, but it's not beyond my ability to imagine. Midjourney helped bring my vision to life.
Also: How to get a perfect face swap using Midjourney AI
I do wonder, though, who is the creator? Is it the AI, because it knew where to put the pixels? Or is it me, because the scene is my vision and wouldn't have existed without my direction? One paradigm we could use is that of a movie director. The movie director brings the movie to life, but certainly doesn't inhabit all the characters and build all the sets. It's a collaboration, led by creative visionaries. I think AI art like this piece is of a similar nature.
What do you think? Are you going to try to create something interesting with generative AI and text-to-image technology? Have you done so already? Let us know in the comments below.
You can follow my day-to-day project updates on social media. Be sure to subscribe to my weekly update newsletter, and follow me on Twitter/X at @DavidGewirtz, on Facebook at Facebook.com/DavidGewirtz, on Instagram at Instagram.com/DavidGewirtz, and on YouTube at YouTube.com/DavidGewirtzTV.