Can AI Make My Newbie Sketches Look Awesome?

How to transform basic sketches into professional-quality drawings with Stable Diffusion

Sketch of a Wooden House mixed with the image generated on Stable Diffusion | Image by Author

I’ve been playing with Stable Diffusion for the past months to generate images using prompts. Recently, I discovered a script to perform image modification called img2img.

This article demonstrates how I’ve used this script to make my newbie sketches look professional.

How is this gonna work?

1. I’m going to draw something in Paint using my artistic skills.
2. Then, I’ll add my sketch into Stable Diffusion using img2img.
3. I’ll write a prompt to guide the model on how to use my drawing.
4. Press “Generate”.
5. Save the images and take credit for the masterpiece!

Disclaimer

All generated images used the following parameters unless I explicitly describe them differently near the prompt:

1. Model: Stable Diffusion v1.4
2. Sampling Steps: 28
3. Sampling Methods: DPM++ 2S a Karras
4. CFG Scale: 11
5. Denoising Strength: 0.7

I do know a little about these hyperparameters. Still, I do not have a technical explanation for why I’m choosing these values besides my intuition.

If you have any suggestions, tell me in the comments.

1. A Sketch of a Pineapple

At first, I wasn’t sure how Stable Diffusion would react to my drawings, so I decided to go for more straightforward sketches, like a Pineapple.

My Drawing

Sketch of a Pineapple | Image by Author

Like I said in the title, I’m a newbie artist, and the best I could do was draw a carrot-like pineapple! Let’s see what AI can do with this.

AI-Generated

Prompt: A Pineapple, detailed | Created by Author using Stable Diffusion

It generated a more detailed version of my sketch, but I was not particularly impressed with the result. I was expecting some hyper-realistic Pinneaple with juicy flavors, but it did what I asked for, detailing my sketch.

AI-Generated — Improved Version

To be fair, the prompt was pretty dull. What happens when I use a more detailed prompt? Can I get better results?

Prompt: A Pinneaple, Hyper-Realistic, Detailed Fruit, Detailed leaves | Created by Author using Stable Diffusion

That’s way better! It did not transform my sketch into a super detailed Pinneaple again. Still, it’s a way better image than what I’ve inputted.

2. A Sketch of an Elephant in the Moonlight

After my pineapple sketch, I decided to create a sketch with more details and colors and aim for a photo-realistic elephant.

My Drawing

Sketch of an Elephant in the Moonlight | Image by Author

If you’re like me and think it is impossible to make a better drawing than I did. Let’s check what Stable Diffusion can do!

AI-Generated

Prompt: Photo of elephant, night, lots of stars, shiny moon, moonlight, dark light, beautiful sky, photorealistic | Created by Author using Stable Diffusion

I’m impressed with this first try. Adding more words from the start did improve the performance of my output.

AI-Generated — Improved Version

For this one, I will cheat a little bit. Let’s increase the value of my Denoising Strenght and see if I can make it look more realistic. Values closer to one might generate images that are not semantically consistent with my sketch.

Prompt: Photo of elephant, night, lots of stars, shiny moon, moonlight, dark light, beautiful sky, photorealistic Denoising Strenght: 0.85 | Created by Author using Stable Diffusion

At this point, I’m convinced that this is awesome. I didn’t want to use higher values to keep the integrity of my sketches, but it seems the model kept the overall concept of my drawing.

3. A Sketch of a Detailed Lake House

It’s time for the final boss. I’m gonna test a complex sketch with lots of details and components and see what happens.

My Drawing

Sketch of a Wooden Lake House on the mountains | Image by Author

This one has a lot of information, and I doubt you could guess all the components of this image. It’s a Lake House made of wood, near a green lake, with some trees behind it and a cloudy mountain in the background.

AI-Generated

Prompt: A wooden house, white sand beach, lake house, green to blue lake, rocky mountain in the middle of clouds, Hyper-Realistic, Natural Light | Created by Author using Stable Diffusion

The result is fantastic! Still, the AI didn’t comprehend that the grey area under the trees supposes to be a mountain. I could have drawn their shapes to a more mountain-relatable shape and made their colors more distinguishable than a cloud.

If you are wondering why the sizes of the images are different, it’s because my GPU wasn’t able to allocate enough memory to generate the exact resolution, and I didn’t want to redraw this massive sketch. So, I resized the output.

AI-Generated — Improved Version

The water on the generated image could be better. Let’s add a reflection of the house to our sketch to see what happens!

Sketch of a Wooden Lake House on the mountains with water reflection | Image by Author

I decided not to change the prompt but keep the seed the same as the one generated above. The seed is the starting point for generating random numbers, causing your model to be deterministic if the input is the same.

Prompt: A wooden house, white sand beach, lake house, green to blue lake, rocky mountain in the middle of clouds, Hyper-Realistic, Natural Light | Created by Author using Stable Diffusion

To be completely honest, I thought the model would produce a nonsense image, but adding the reflection in my sketch significantly improved the output in general.

To conclude,

Can AI make sketches more professional looking?

The answer is a resounding yes. Even a beginner “paint artist” like myself could draw beautiful images that resembled my original sketch.

Are artists about to lose their jobs?

I don’t think so. What about you?

If you’re interested in reading other articles written by me. Check out my repo with all articles I’ve written so far, separated by categories.

Thanks for reading