Diffusion Models

After procrastinating for well over a year, I have finally decided to write about the things that interest me the most. I have been closely following the field of Generative AI, long before the revolutionary DALL-Es and ChatGPTs took over the world by storm. Honestly, it makes me feel old…

If you have been following social media lately, you might have heard about diffusion models like Stable Diffusion and DALLE-2. These models are able to create super realistic images and sometimes even audio samples, but they are rarely applied in the text generation domain, ever wonder why’s that? …

With the development of artificial intelligence an engineer will have a greater number of toolings to facilitate her in design. The pace of computer technological advances dictates a modern designer to equip herself with at least basic understanding of current artificial intelligence algorithms and methods. Gigala, [1], is an attempt…

I have been meaning to write a thesis on exciting areas that transformers/LargeLanguageModels could disrupt; but this space is evolving so fast, that by the time I finish a draft, it is already outdated. So I have stuck to posting my thoughts on twitter instead. But after several conversations (and…

Stable Diffusion for IPUs on Paperspace offers text-to-image, image-to-image, and text-guided inpainting. — Written by Goran Katalinic Generative AI has well and truly hit its stride. GPT3 and its derivatives, including ChatGPT have popularised text generation, while the likes of Dall-E and Midjourney are being used to create stunning original artwork from written prompts. Not only is the technology remarkable in itself, it…

Have you heard of Stable Diffusion? If not, you must have seen this image circulating the internet captioned: Jeflon Zuckergates. The face looks similar to Jeff Bezos, Elon Musk, Mark Zuckerberg and Bill Gates combined. This creation from text to image is powered by an AI model, Stable Diffusion…

Textual Inversion: Learn the word for a given subject by optimizing the language encoder which enables us to generate new images of the given subject in new settings. Paper: https://arxiv.org/abs/2208.01618 If someone asks you what the following image is, you say “teapot”. If I ask you for more details, maybe…

In this series, we are trying to reach a (mostly) qualitative understanding of how diffusion models work by skipping a lot of the math and looking at visual interpretations. In Part 1, I explained the forward diffusion process. In Part 2, we now focus on backward diffusion, where learning happens. The Story so Far… …

DreamBooth: Assign a rare sequence of tokens as the subject’s identifier and fine-tune the diffusion model on the small set of images with the “subject”. Paper: https://arxiv.org/abs/2208.12242 The authors use the Imagen model in this paper which uses T5-XXL language model to encode the text guidance to generate a small…