Taking a closer look at the “HiRes fix” latent upscaler for Stable Diffusion to optimize image quality
In the search for the perfect AI-generated image, I investigate the effects of the parameters denoising strength and hires steps.
AUTOMATIC1111’s Stable Diffusion WebUI has proven to be a very good tool to generate AI-generated images using StabilityAI’s Stable Diffusion. To create the images I used a forked client from vladmandic, which extends the WebUI with some features and reduces generation time. In my search of the optimal settings for the perfect AI generated image, I created a total of 77 images with different settings in the course of this article in order to examine the effects and correlations of the parameters denoising strength and hires steps.
I used the following image as a starting point:
Parameters used: miaobishenghua, a medieval town in Germany with a river, raytracing, ornate, intricate, beautiful, atmosphere, vibe, flowers, concept art illustration, volumetric lighting, sunbeams, particles, oversaturate, volumetric lighting, beautiful, rich deep colors masterpiece, sharp focus, perfect composition, cinematic perfect light, masterpiece, <lora:miaobishenghua_v10:0.6>
Negative prompt: out of frame, duplicate, watermark, signature, text, missing, disfigured, kitsch, ugly, oversaturated, grain, low-res, Deformed, blurry, mutation, mutated, blurry, blur, out of focus, poorly drawn, mutilated, mangled, old, pink, people, humans,
Size:512x512, Seed: 4000545968, Model: Deliberate, Steps: 50, Sampler: DPM++ 2M Karras, CFG scale: 7
Now let’s start with the image matrix. I used the hires upscaler “latent” and upscale by 2 for all images:
- On the X-axis, the denoising strength changes in a value range of [0, 1] with a step size of 0.1 (11 steps in total).
- On the Y-axis, the hires steps change in a value range of [0, 150] with a step size of 25 (7 steps in total).
A closer look leads to the following findings:
- Regardless of the highres steps, generated images with a denoising strength of < 0.5 are very blurry and noisy.
- With increasing denoising strength, the content of the image changes enormously. The only setting with which the upscale bears a close resemblance to the previous image is a denoising strength of 0.5.
- The changes caused by the denoising strength parameter are not random and independent of the highres steps.
- The highres steps parameter causes only small changes to the image. It is also not obvious in my opinion that a higher value automatically produces “better” images.
- The reflections in the water become increasingly inaccurate as denoising strength increases
In addition to image quality, generation time is another important factor. When generating the images, I saved the elapsed time together with the parameters. Let’s take a closer look at this:
A look at the correlation table of the values clearly shows that there is only a moderate positive correlation between denoising strength and elapsed time, but a strong positive correlation between highres steps and elapsed time. In other words, denoising strength has small effects and highres steps has large effects on the time it takes to generate an image. This relationship can also be observed when looking at a 3D graph of the values.
Conclusion
Okay, let’s derive recommendations from the previous findings:
- The denoising strength parameter has a big impact on the image quality, but it also changes the content of the image a lot. It should be between 0.5–1, depending on how much you want to keep the content of the original image. At the same time, the impact on the generation time is very small.
- The highres steps parameter has only a small effect on the quality and content of the image, but has a big effect on the generation time. I recommend a value <= 25 so that images are generated faster. Once you have generated a nice image, you can experiment with higher values.
I just published an article where i compare the image quality of the other “HiRes fix” upscalers. Check it out!
Finally, let’s take a look at some of the pictures that were created during the process:
