A new isometric diffusion model hit Disco Diffusion a few days ago. With a little coaxing it produces really fun images using isometric projection. You can read more about this method of projecting 3-dimensional objects into 2-dimensional space online, but the brief summary is that in contrast to one-point or two-point projection where parallel lines meet at the horizon, in isometric project, parallel lines remain parallel. Also, the angle between any two axes is 120 degrees.
This type of projection is usually used for technical drawings but can also introduce a game-like quality to images. This is used to great effect by games like Habbo.
I thought I would give it a try. Here are some initial results:
This is an over-the-top collection of factories on fire blowing out red and pink smoke.
If you’re using Disco Art and you want to follow along, here is my config, or you can pull it directly using docarray
A series of futuristic eco-friendly buildings:
Config here or and docarray id:
discoart-99f580dbb5ca8869eea3baaefd951d0b
The model tends to favour pastel colours for some reason. Here is a shopping mall that I created:
What I’ve learnt so far
The model needs a little push at the beginning otherwise your images tend to be sparse with lots of white space. You can set perlin_init to True to use perlin noise instead of Gaussian noise. You can also supply an init image to get it started. I found perlin noise to exacerbate the overly pastel look of my images.
Instead, I’ve been using secondary_model scheduling, a little-known feature in Discoart which let’s you decide when to use the smaller and less accurate secondary model and when to use the main model. I have found pretty good results when running the secondary model for the first 20% of the run, and then 80% thereafter. This is what the setting looks like:
Bringing in the big guns
Sometimes the isometric diffusion model doesn’t get it quite right. Fine details are sometimes lacking. I have started to experiment with layering diffusion models to address this. For example, I create the initial image using the isometric model and then use it as an init image for the default model with a high number of skip steps. This has the effect of adding a little more texture to your final image.
Have a look at how the default model improved the fire in these lego house on fire images. The image on the far left was produced by using isometric model only. The next three images were created by using the first image as input into a second run using the default model. The second image ran for 30% and was then stopped, the third for 60% and last ran to completion.
The most prominent improvement is the fire. The isometric fire is very blocky whereas the touched up images have a more organic shape. It’s not perfect and I think with a little more work it could be have been better, but the improvement is clear.
There are also additional fine details, like the delineation of some of the panels and more detailed windows. I personally feel that 100% is too crunchy, the 60% is a good balance between too little and too much detail.
Final thoughts
I think the isometric model is a great addition to the growing list of custom diffusion models out there. It has its own quirks but often produces amusing images. Kudos to @revart for producing and sharing such a great model.
I’ll be honest, my best images were created with 1024x768 dimensions when I won the GPU lottery and was gifted an A100 for a few days. I find 512x512 to be a little more challenging, which is the largest image I can create with a GPU that only has 16GB VRAM.
I hope this brief write-up encourages others to both try out the new isometric model as well as experiment with various scheduling setttings that are available in Discoart.