8 beginner tips for mastering Stable Diffusion prompts
Unlike its competitors Dall-E and Midjourney, Stable Diffusion isn’t so great at using traditionally written prompts to generate desired images. That’s a trade-off you have to make to avoid ridiculous censorship and other restrictions which come with its alternatives.
But this is not to say that Stable Diffusion is bad at converting imagination into reality. On the contrary, it can do a much better job than it’s more restrictive alternatives…if you know what you’re doing!
But before you dive into the plethora of options Stable Diffusion offers when it comes to fine-tuning generations — such as image-to-image, ControlNet, AnimateDiff and a lot more — the first thing you have to master is writing prompts.
With that in mind, we have written this article to help SD newcomers understand how to craft prompts that generate the image they want to see. So read along to unleash your powers of creative expression!
- Break it down and use keywords
When it comes to writing prompts for Stable Diffusion, descriptions of the kind you typically find in books don’t work so well. Steer clear of narrative-style prompts and focus on providing specific visual cues. This helps the model generate more vivid and detailed images.
Using comma-separated keywords which describe the scene and it’s individual components is very effective. Describe the character, the camera focus, lighting, colors, style, and components as individual elements for the best results.
For example: “a Mysterious figure, standing, in a moonlit forest, fog, blue hues, impressionist style, focus on silhouette, dramatic lighting, sharp details, photorealistic, ultra hd, 8k”
2. Experiment with Styles and Perspectives
Encourage the model’s creativity by requesting an aerial picture of a cyberpunk cityscape. Give it a bold, modern look, and make it exclusively yours! Add some flair and insight to your prompts, even if you’re just starting out. It’s similar to giving the model a gentle prod to encourage it to explore the universe of creativity.
A layer of originality and creativity is added by styles and perspectives to create information, whether it be text or images. You can incorporate words like:
Realistic — This style aims to replicate the real world accurately. You can request a scene to be generated with a realistic touch, focusing on detailed and authentic elements.
Abstract — Encourage the model to think outside the box. Ask for an abstract representation of a concept, allowing the model to play with shapes, colours, and forms.
Impressionistic — Inspired by the art movement, this style emphasizes capturing the essence of a scene rather than details. Request an impressionistic landscape with broad strokes and vivid colors.
Cartoon/Comic/Manga/Anime — Specify a cartoon or comic style to evoke a more light-hearted or animated feel. Use the words “anime” or “manga” to capture the aesthetic made famous by Japanese artists. This is especially useful for creating visually engaging and playful content.
Minimalistic — Challenge the model to create with simplicity. Ask for a minimalistic representation of a subject, focusing on essential element
Bird’s Eye View — Request a scene from a bird’s eye perspective to capture an overall view. This is excellent for cityscapes, landscapes, or any scenario where an elevated viewpoint is desired.
First-Person Perspective — Immerse viewers in the experience by asking the model to generate content from a first-person viewpoint. This brings a personal touch to the narrative.
Macro Perspective — Zoom in! Ask for a macro perspective to focus on intricate details. This is perfect for exploring the beauty of small elements, like the texture of a leaf or the patterns on a butterfly’s wing.
Wide-Angle Perspective — Opt for a wide-angle view to capture a broader scene. This is useful for landscapes or architecture where you want to emphasise the expansive nature of the setting.
Over-the-Shoulder Perspective — Create a sense of intimacy by requesting an over-the-shoulder perspective. This is great for scenes involving characters or activities where you want viewers to feel a connection to the subject.
3. Incorporate Story elements
If you want a narrative touch, include elements that imply a story without explicitly describing it. This can lead to more dynamic and engaging visual outputs.
But…
4. Ditch the Narration, Dial-up the Imagery
Give up on the conventional storytelling method and explore the fantastical. Do not tell the story; instead, ask the model to imagine. Sayings like “Dragon, mountain!” will simplify your suggestions and let imagination take the lead. Just remember to set your AI guidance/CFG scale low to give the SD model freedom to generate.
5. Infuse Emotion into the Palette
Speak the language of colors and moods. Challenge the model to paint a canvas that feels “serenely mysterious” or “joyously enigmatic.” You might not expect it, but this works wonders, especially with SDXL models!
6. A ‘NO’ goes a long way
Along with the prompt, a model also receives important instructions from the negative prompts that tell it what not to include. They direct the model by taking out certain components or traits, which helps to more accurately mould the intended outcome.
To achieve a human-free environment, for instance, you add the following words in the negative prompt: “no humans.” This kind of guidance gives the model more control over the image it generates by helping it concentrate on some elements while ignoring others.
7. Emphasize with Brackets for Artistic Impact
Add emphasis to what matters using round brackets. Specify details like a “(mystical) scene with a (shining) unicorn in a (pastel) meadow at (twilight).” If you want to emphasize a word or phrase even more, you can add multiple brackets: ((mystical)) or increase its weight by adding a number after a colon: (mystical:1.4).
8. Negative Embeddings
When linguistic signals or embeddings in the input text are used to communicate a negative context or constraint to the model, this technique is known as negative embedding.
Embeddings are numerical representations of words or phrases used in language models. In order to steer the model away from particular traits or concepts, without having to write wordy negative prompts, you can simply use an embedding instead.
Generating normal hands is something generative AI models struggle with immensely. One way to fix this issue in your image would be to write a negative prompt like: “mutilated, deformed, extra fingers, extra hands, weird fingers, mutated fingers…” OR you could simply use a embedding like “BadHands” in the negative prompt to specifically solve this problem.
Consider creating prompts as a creative symphony; practice, experiment, and fine-tune like a novice composer. Make a word or color change, and watch your visual creation become something wholly your own. Keep in mind that while dealing with creative models such as Stable Diffusion, experimenting is essential. You may get the desired creative result by fine-tuning your instructions in response to the model’s reactions.
Prompts for Stable Diffusion are both a science and an art to learn. It calls for both inventiveness and methodical experimenting. If you want to practice your Stable Diffusion prompting skills, go to rendernet.ai and give it a shot. You can even run the same prompt on multiple models at the same time to see how they differ and then pick your favorite and go to work fine-tuning it!
