Creating icons with AI: a simple pipeline for boosting your workflow

Over the past year text2image models (creating images from text) have been developed and implemented in many professional areas including game development and design. It’s not a surprise that specialists begin to ask themselves ‘How is AI going to influence my work? How can I use AI for boosting my productivity?’. Today we will talk about how to create icons of good quality with Stable Diffusion.

The whole process of creating icons can be divided into 3 big steps:

• generation,
• processing and editing,
• finalizing.

Step 1. Generation

In order to create icons with any text2image model you need to know how to write a prompt — a small text guiding a neural network. You will notice shortly that if you write down only 2–3 words, the final images will be photorealistic. And although the image fits the description, it’s of no use for icon generation.

We have previously published a useful material about prompt engineering — the art of writing prompts, and in our product Phygital+ we have tools for quicker promptwriting: Prompt Extend, Image-to-text prompt and ChatGPT. We also have collected one of the best prompts in small collections for characters, locations and objects.

What do we have to know to write a good prompt? In general we recommend to follow a simple rule — the more lengthy your prompt is, the better results you will get. Imagine that you have to explain to a small child what you want to see, and try to do it by reciting words and small phrases.

So, we have written a prompt, but the result is still not good enough.

To solve this problem, we need to follow one simple advice — while generating with Stable Diffusion use custom checkpoints. These checkpoints are Stable Diffusion models that were pretrained on a particular style. With them even the simple prompt can give awesome results.

Here’s an example of how we could improve the results of generations simply by choosing a custom model. The settings were the same as on the previous image.

And here how they look with DreamShaper model.

We also have got good detailed icons while using NeverEndingDream (NED) model.

If you’re looking for custom checkpoints, we recommend to visit Civitai. But while using this kind of websites you can get easily overwhelmed by the amount of available checkpoints and models. If you’re using Phygital+, you can be sure that none of the most popular checkpoints are missed and we update the list of available models regularly. Now we have more than 45 models.

Step 2. Processing and editing

The next step is to process the results we have got. In the process of creating icons you often need to produce the concept in several variations. ControlNet neural network is perfect for that: it takes the initial image and based on text prompt it modifies it into a completely new one.

In this neural network you have several ways of transforming image. For our pipeline the most suitable one is HED or Edge (Canny) — in these modes ControlNet keeps the shape and form, and based on the text it fills up the space inside the object.

ControlNet works based on Stable Diffusion, that’s why all prompt writing rules are useful here. For example, if you need to create a sword or a bottle in different colors, you just need to copy your prompt from the base generation, change the color written in your text and quickly get new concepts, ready for the further use and editing.

Step 3. Finalization.

The last step is quickly removing background and upscaling the image (2x or 4x of size) using Remove Background and Upscale Image nodes.

The full video with the described pipeline you can watch here.

If you need to create icons in your own style, we recommend reading our previous article, in which we guide how to train AI on your style step by step and how to use it for asset generation.

Wrapping up, we can say that neural networks today are our assistants that help creative professionals quicker and better use their time to create content and visuals. It takes the daunting and boring work and leaves more space for ongoing creativity. They are like paints for new generation of artists, with which you can easily work now and make stunning content.

Follow us on socials: Twitter, Instagram, Discord, Telegram.