Stable Diffusion Ultimate Guide pt. 5: ControlNet

Umberto Grando
6 min readMay 6, 2023

Hello World!

In the fifth part of our Stable Diffusion guide, we are going to take a look at an incredibly powerful tool for image generation: ControlNet.

ControlNet enables us to guide the generation of our pictures in a non-destructive way. For example, take a look at the following example:

Courtesy of the ControlNet Github page

In the above example, we have an input image which gets elaborated using a Normal map (you can see the map as the first result on the right). This normal map is then used by SD to generate an output incredibly similar to the original input!

Right now, there are 14 different ControlNet models (11 production-ready and 3 experimental). In this guide, we are going to see how to install everything we need and then use the ControlNet extension to generate awesome pictures.

If you are a total beginner, I recommend to start from the first chapter of this guide:

--

--

Umberto Grando

I’m Umberto Grando, an IoT Specialist with a passion for programming, gaming and technology in general.