Midjourney Mixed Media Experiments
Of course Midjourney only generates 2D arrays of pixels, but that doesn’t mean that you can’t use it to make images of 3D art pieces. This article includes prompts for a diorama and a sculpture (among 20+ others). But what’s interested me most of all is reliefs.
Unfortunately in this genre, Midjourney gives patchy results, which I think is part of a pattern across the board with less famous art styles that would appear less in its training data. It’s hard to get a relief of anything much more complex than the palm tree above.
But that matters a lot less now. Thanks to the new inpainting function, it’s now possible to do the following three things:
- Add in one item at a time, positioning each how I want as I go along
- Specify parts of the relief to be made out of different materials
- Decorate the relief with colours and designs
My first attempt at such a piece was this:
I was impressed by what Midjourney could do, but less impressed by what I had done with it. I’d merely chucked things together without thinking about the “why”. I won’t bother to go through the prompts and stages to make the above image, because they’re very similar to what I describe for my next piece.
This was to be an abstract mixed media art piece, made for a wall in a dimly lit room. In terms of the general style of the piece, I wanted:
- When something in the piece is meant to be near the viewer, the object is included in a way that the viewer can touch it (e.g. bas relief, changing the surface of the artwork, including a physical object). I figured that touch would be especially important in a dimly lit room.
- Things that are meant to be far from the viewer are depicted by painting. I’d been inspired by Christie C.’s mention of the possibilties for making encaustic art in Midjourney, in this Medium post. But I decided to go with carnauba wax because it would shine brighter in the dim light.
- The piece prioritises abstraction and geometric patterns over perspective and realism.
In terms of specifics for the piece, I wanted:
- A landscape orientation (since people touching a piece can easily walk from one side to the other, but will not be able to reach anything much above head height)
- Most of the piece to be made of frosted glass, to catch and reflect what little light would be in the room
- For the rightmost third of the piece to be made of kind of sandstone called Beacon Sandstone
- Bas reliefs of vegetation carved out of the frosted glass, specifically a silver fern across the top, a krummholz on the left, and tussock grass at the bottom
- Something that would represent a thunderstorm, painted in the top left corner of the painting (so over the silver fern bas relief)
- An aurora beam starting on the silver fern and going horizontally right onto the sandstone
- A conical composite volcano painted in the centre of the piece, with a lava lake (I’d be happy for the bot to break the rules of realism in order to make the lava lake more obvious, and it ended up doing that, in an unexpected way)
- Palm trees painted to the left of the volcano
- To paint something that represents an incoming ocean swell, on the sandstone in the bottom right
- Something else that would represent the sea, directly under the volcano
Here’s what I got:
But I started from a blank canvas, prompting: dimly lit blank pane of frosted glass — s 50 — style raw — ar 3:2 and getting:
I didn’t want the black space, but figured I could crop it out after making the piece. And it came in handy when inserting the last item.
Just like with Aurora Rice Cavates and Tropical Zen Cave, I used image prompts to inpaint the sandstone (and many other things). To get my image of the sandstone, I prompted: dimly lit Beacon_sandstone horizontally scoured with glacial_striations and ripple_marks well_sorted, glossopteris_fossils — ar 1:2 — s 50 — style raw
If you’re wondering why I put underscores between some words, I explain that in this post.
At first I got the below image on the left, and then inpainted it with the same prompt to get the one on the right:
Then I selected the right third of my blank glass pane for inpainting, with the prompt: <URL of Above Right Image> dimly lit Beacon_sandstone horizontally scoured with glacial_striations and ripple_marks well_sorted, glossopteris_fossils — s 50 — style raw
I had my substrate! There was an ugly bit of sandstone sticking out, but that was right where I was planning to inpaint (as my very next step actually).
Now it was time for the silver fern. I clicked Vary (Region) and selected the top third of the frosted glass. Inside that I prompted: frosted_glass bas_relief silver_fern frond fractal_patterns tactile_art left_to_right, and chose:
I particularly liked the swirling part in the top left. It reminded me of a thunderstorm, and it was exactly where I had hoped to paint on the element symbolising a thunderstorm.
I also noticed something really weird here: I hadn’t selected anywhere near the bottom right corner, yet the bot had put in a purple glitchy thing here. Going back to Aurora Rice Cavates, I noticed the same thing had happened with glitchy things in the fog.
Next it was time to inpaint the aurora. The bot was a bit recalcitrant here. I got it to work by selecting a thin horizontal band from the base of the fern to the right edge of the sandstone, and typing in the multi-prompt:
horizontal vivid green_aurora carnauba encaustic on frosted_glass, vibrant colours saturated_colours, intricate geometric fluid fractal_patterns::9
frosted_glass bas_relief silver_fern frond fractal_patterns tactile_art
— style raw — s 50
I used a line break to highlight the two parts of the multi-prompt.
This was a balancing act: I had to keep the prompt for the fern as one element of the sub-prompt, because otherwise Midjourney would erase the relief of the fern underneath the aurora. But the aurora prompt needed a much higher weight otherwise no aurora would appear.
And at first the aurora was very weak and didn’t got far enough to the right. I had to extend it by repeatedly going into inpainting (using the same prompt as above) and selecting a rectangle consisting of the right hand side of the aurora, plus everything from that all the way to the right hand edge of the sandstone, then choosing the best of the 4 results to do it again.
I started with the left, and after 4 cycles of inpainting, finished with the right.
⚠️ It’s important to note here that I was careful not to select all of the aurora. When I did this, the bot had a tendency to erase the entire aurora.
Now it was time for the lightning. I selected a rectangle starting from just to the right of the aurora’s left end, and going up and down almost all of the way to the edges of the silver fern, and prompted: violet_lightning, glossy carnauba encaustic, intricate geometric fractal_patterns, vivid, vibrant, saturated_colours — style raw — s 50
In terms of shape it didn’t look so much like lightning, but I felt like the vivid violet was enough, since it was in conjuction with the swirl in the fern. And I really liked how Midjourney had put the violet paint over the bas relief.
⚠️ Just like with the aurora, I was careful not to allow the inpainting selection to reach the endpoint of the aurora or to go over the edges of the fern. If you exclude such an endpoint or an edge (or at least some of the edge), then Midjourney is much more likely to reconstruct your original object in the inpainting zone, and then inpaint the new stuff over it. Otherwise, Midjourney tends to delete your original object. This principle was applicable throughout the construction of this piece.
The bas reliefs of the rest of the vegetation were easy. I selected the left half of the frosted glass pane (below the fern) and prompted: bas_relief on frosted_glass of gnarled krummholz in fractal_patterns — style raw — s 50
Then I selected the bottom half of the remaining glass, and prompted: bas_relief on frosted_glass of tussock in fractal_patterns — style raw — s 50 — ar 3:2
The first and second stages above are shown below on the left and right, respectively:
The last remaining vegetation to do was painting in the palm trees. This was tougher. I only got what I wanted by doing the following two things:
- Using an image prompt
- Selecting a much larger area than where I actually wanted palm trees (a rectangle covering the entire width of the frosted glass, going down from just below the bottom of the fern to just above the top of the tussock grass)
The second thing was less of a problem than it might sound, because I was going to paint in the volcano over the part of the selection where I didn’t want palm trees.
Anyway, to generate the image for the inpainting prompt, I used the text: palm tree treetops in the distance, fog, glossy carnauba encaustic on frosted_glass, intricate geometric fractal_patterns, vivid, vibrant, saturated_colours — style raw — s 50
I tried using the below two resulting images for inpainting in my main piece:
The text prompt I used was: <URL of the Image of Palm Trees>::9 palm tree treetops in the distance, fog, glossy carnauba encaustic on frosted_glass, intricate geometric fractal_patterns, vivid, vibrant, saturated_colours — style raw — s 50
I had to crank the image weight up to 9 to get any palm trees at all, and even then I only had success with the right hand image. I wonder if it was because the pink shading in the left image was too different from my main piece. Anyway, I got:
Inpainting in the volcano was easier, although I still needed an image prompt. I generated this with the text prompt: smoking stratovolcano in the distance, towering out of a cloud_inversion silhouetted against thick fog, translucent glossy carnauba encaustic on frosted_glass, intricate geometric fractal_patterns, vivid, vibrant, saturated_colours — ar 3:2 — s 50 — style raw
Then I went to the main image, clicked Vary (Region) and selected a rectangle with everything to the left of the sandstone, below the fern and above the tussock grass. I prompted: <URL of Above Image> smoking stratovolcano in the distance, towering out of a cloud_inversion silhouetted against thick fog, translucent glossy carnauba encaustic on frosted_glass, intricate geometric fractal_patterns, vivid, vibrant, saturated_colours — s 50 — style raw and got:
For the lava lake there was good news and bad news: The good news was that, as long as I used an image prompt, Midjourney was very willing to paint in a lava lake. The bad news was that it usually did it in an ugly way. The best I could do was a lava lake that half looked like an abstract halo. This wasn’t what I’d expected, but it was OK because I was hoping for the bot to break realism anyway (otherwise the lava lake would have been too small). I got best results by first generating an image with the prompt: volcanic_crater with a glowing lava_lake like Mount Nyiragongo, thick_fog, glossy carnauba encaustic on frosted_glass, intricate geometric fractal_patterns, vivid, vibrant, saturated_colours — style raw — s 50
Then in the main image, I clicked Vary (Region) and selected a rectangle including everything above and including the “shoulder” of the volcano. I prompted: <URL of Above Image of Lava> translucent glossy carnauba encaustic on frosted_glass, of thick_fog around a glowing lava_lake with ropey corded pahoehoe in a volcanic_crater like Mount Nyiragongo, thick_fog, intricate geometric fractal_patterns, vivid, vibrant, saturated_colours — style raw — s 50
At first I got:
Midjourney had flattened the mountain from a stratovolcano to a shield volcano! I reshaped it by selecting a large area of the volcano and of the area above it (being careful to leave some of the lava lake halo) for inpainting, prompting: <URL of Image Used In Prompt For Volcano (shown again below left)> glowing volcanic_crater on a stratovolcano in the distance, towering out of a cloud_inversion silhouetted against thick fog, translucent glossy carnauba encaustic on frosted_glass, intricate geometric fractal_patterns, vivid, vibrant, saturated_colours — ar 3:2 and getting the image below right:
Now that I’d made the volcano into a steeper cone, I wanted to flatten the part right at the top to suggest a crater. I also wanted some smoke spewing out and flowing to the left. I selected a large rectangle on and to the left of the summit, prompted: smoke spewing out of a volcanic_crater with a glowing lava_lake, on a stratovolcano, translucent glossy carnauba encaustic on frosted_glass, intricate geometric fractal_patterns, vivid, vibrant, saturated_colours — ar 3:2 and got 3 nice images:
It was a tough decision, but eventually I chose the one on the far right.
Next, I wanted to paint the bottom part of the sandstone with something representing ocean waves coming in. I scaled my ambitions down to just changing the sandstone bands from orange to blue, but even that was a challenge. Again, I did the inpaint with an image prompt, which I generated with the multi-prompt: wavy horizontal_lines, vibrant aquamarine, vivid turqoise, saturated teal, intricate geometric fractal_patterns translucent glossy carnauba encaustic::
dimly lit Beacon_sandstone horizontally scoured with glacial_striations and ripple_marks well_sorted, glossopteris_fossils
— style raw — s 50 — ar 3:2
I then selected the bottom third of the sandstone in my main piece for inpainting, using the prompt: <URL of Above Image>:: wavy horizontal_lines, vibrant aquamarine, vivid turqoise, saturated teal, intricate geometric fractal_patterns translucent glossy carnauba encaustic — style raw — s 50
That yielded:
Almost there, but the last part was tricky. I tried all kinds of things to represent the ocean (in the zone under the volcano), but Midjourney either gave me nothing, or got rid of the tussock grass. I only had luck when I decided to use a cairn made out of chalk boulders, inpainting with the prompt: cairn of rounded chalk_boulders glauconite — style raw — s 50 and selecting everything down from the volcano, right to the bottom of the screen. It wasn’t enough just to select to the bottom of the glass.
Now only gave me a single rock rather than a cairn, but it matched the shape of the volcano in the way I was hoping to achieve with a cairn. And it didn’t give me glauconite, but after some experiments with Midjourney, I concluded that the AI doesn’t know what this mineral is. So I cropped this down to exclude the black areas for the final piece. If you consider this as a 2D artwork, there’s a bit of a trompe-l’œil effect happening with the stone.
After this, I tried an interesting experiment. I fed my art into Midjourney’s /describe function a bunch of times, and then cherry picked some phrases from that to prompt the bot (cranking — style up to 750). I tried getting both square images and landscape images (8:5), and they were all beautiful, though absolutely nothing like my art.
I also tried rewriting the above prompt in natural language:
Whew! This was a big project, and I learned a lot from it:
- It’s definitely possible to use Midjourney to set up a “blank canvas” of a single material (or two) and then paint over them or texture them into bas reliefs.
- Carnauba encaustic on frosted glass is a lovely medium.
- You can get trompe-l’œil effects if Midjourney leaves blank space around your “blank canvas”, because then you can select some of that space for inpainting, and then crop it out at the end.
- If you use Vary (Region) to put paint on a 3D object such as a bas relief, avoid selecting some of the object (e.g. the ends or edges) if at all possible. That way, Midjourney will be much more likely to keep your original 3D object.
- If you’re doing the above, it’s hard to get Midjourney to put detailed designs on the 3D object (including textured surfaces). Splashes of colour is usually the best it can do.
- When using Vary (Region), the bot can do glitchy things to areas on the other side of the image from where you’re inpainting.
- It’s possible to get interesting prompts for (realistic) Midjourney pieces by feeding in abstract art to the /describe function.
Thanks for reading all the way, and I hope you got something out of it too.
