Creating Artwork with AI: Fine-tuning Drawings with Midjourney and Lumenor.ai

Image generated via Midjourney

I am Mikhail Suvorov, and I am a lead communications designer at X5 Tech. In the article we will discuss several effective ways to ‘persuade’ AI to draw exactly what you need. I am using AI as the image generator, primarily to create illustrations and artwork.

If you are already familiar with txt2img AI (i.e., tools to turn your text requests into images) and use them quite often, you might have noticed that you don’t always get what you want — you don’t always achieve the result that you have ‘in your head’ from the first try. And you cannot force the AI to fulfill your requests more precisely.

In the material, we will consider the two AI tools that are available online and, as of today, best cope with the task of visualization ‘according to your brief’ — Midjourney and Lumenor.ai.

Midjourney and Its Vary (Region) feature

In the first case, all the artwork generation and additional refinements were carried out in Midjourney. What is important for this case is that we initially had in our mind what kind of result we should get, who would be the hero in the illustration and its general mood.

I prefer the most detailed queries that include all the details of the future image:

blue sky and interface buttons, man in ray ban navigator sunglasses in panama hat, background in desert, interface green buttons, interface UI elements floating in air, blue buttons, dribble best cases, neon lights, nevada las vegas desert, fear and loathing in Las Vegas, octane render, screens, view, collage, double exposure, wireframe, literature, vector art, magazine — ar 16:9 — s 200

I deliberately skip those preliminary stages which finally led me to this particular type of request, otherwise the article will be overloaded with illustrations.

Here’s what we’ve got:

We liked the first of the four variants; however, it ignored our expectation that the hero should have a sun (bucket) hat. (as it later turned out, Midjourney has difficulties with drawing hats); besides that, our hero is dressed completely differently from what we need.

Fortunately, the latest release of Midjourney has the Vary (Region) functionality, and we can gradually refine the image to what we need according to the concept.

It is best to change the elements in the generated image one by one, so that the anchor points in other parts of the image are preserved (in our case, these are the head and neck, which indicate that we want to show our hero from a lower perspective).

Here we have highlighted the area for replacement and entered a new query — a Hawaiian shirt, specifying its color

The result turned out great, you just have to choose the one you like the most

Now we repeat the same procedure, but this time — with the head of our hero:

And here are the resulting images:

Unfortunately, it is still not a sun hat (or bucket hat), but one of the variations is made from such an angle that the hat is actually perceived as a sun hat

Let’s stop there, and after a little retouching in Photoshop we finally get the result we need:

Lumenor.ai

In the second case, we resorted to the Lumenor.ai tool to loosely tweak the image based on a sketch (it was a low-res image from the Midjourney generations, but it could easily have been a collage, a reference from Pinterest, or even a hand-drawn sketch instead).

The goal was to make a set of robots with different emotions for a sticker pack.

There have been many different robot heads, but this one, with its ‘facial’ expression has really etched our heart

I decided to fine-tune this robot using the Lumenor.ai online tool — one among many that works using the same method as in Stable Diffusion, and which, in addition to many brilliant features, can work with reference images.

So we uploaded our robot, cut out from the rest, and played with different style settings, such as light / photography settings (they determine how close the resulting image will be to your reference one, as well as the overall style of the final artwork).

It is important to set the CFG Scale value to at least 5 — this will determine how much the resulting artwork will resemble the original image

I used the same text request as for Midjourney.

After several iterations, I chose the two most appropriate in terms of details and blended them together in Photoshop

The final sticker

Key takeaways:

Now, towards the end of 2023, the possibilities of using AI for generative art are much wider than they were less than a year ago. The cases above are just some of the few possible examples of using relatively new functionality of AI.

How do you use these new tools? Have you found any interesting applications of AI in your work? Share in the comments.