How to run Stable Diffusion on Intel GPUs with OpenVINO

Note: It works on CPUs, and is accelerated!

OpenVINO Notebooks comes with a handful of AI examples. But do you know that we can also run Stable Diffusion and convert the model to OpenVINO Intermediate Representation (IR) Format, and so it can run on CPUs and GPUs efficiently? Also, by compressing the FP32 model to FP16, we reduced the model size by half (close to), and also now it requires much less RAM/VRAM to run. Most importantly, this also provides a significant speed-up in GPU processing because Intel Xe Matrix Extensions (XMX-systolic array) kicked in.

Here are some results I got running the notebook and it’s pretty fun. With my Intel Arc A770m, I can get approximately 6.0 iterations per second (without debug mode). What it means is usually it takes less about 10 seconds to generate a high-quality image below.

Stable Diffusion text-to-image results with the OpenVINO Notebooks and Intel Arc A770m.

Image-to-Image example, turning a photo into a watercolor painting.

First, here is the OpenVINO Notebooks repository. It has everything you would need to complete the demo here today.

openvino_notebooks/notebooks at main · openvinotoolkit/openvino_notebooks

Stable Diffusion is under the 225-stable-diffusion-text-to-image folder

And in the demo notebook, we introduced not only the famous Text-to-mage pipeline but also included the Image-to-Image generation pipeline. But what does it really mean and how do we run it?

Pipelines

How to Install

Quick Video Instruction.

To install OpenVINO Notebooks, you can follow the instruction here if you are using Windows: https://github.com/openvinotoolkit/openvino_notebooks/wiki/Windows

If you are a Linux user, you can follow this link: https://github.com/openvinotoolkit/openvino_notebooks/wiki/Ubuntu

On a high level, it’s a few steps below.

Install Python 3.10.x. (or below) and create a virtual environment

python3 -m venv openvino_env
source openvino_env/bin/activate #for linux

Git Clone the directory

git clone --depth=1 https://github.com/openvinotoolkit/openvino_notebooks.git
cd openvino_notebooks

Install all libraries and dependencies

pip install -r requirements.txt

Run the Jupyter Notebooks

jupyter lab notebooks

Run All Cells and wait =)

Now, if you look into the code. What we have done is really optimize the PyTorch pipeline and execute the code with OpenVINO.

The downloading and converting may take a little while for the first time. Once it’s completed you will get a set of IR files. For your convenience, I’ve already updated these pre-trained, optimized models here to huggingface.

bes-dev/stable-diffusion-v1-4-openvino at main

Now, if you are blessed with the Intel Arc GPUs, you can change the code to “GPU”. By default, it’s using AUTO, and thus it will switch to GPU automatically if it’s detected.

Make it run on GPUs

Auto-plugin. It started by first using the CPU, then switch to GPU automatically.

And here in this step, I have set the steps to 30. Ideally, I would use 50 as it will provide the best-looking results. You can generate different scenes here by modifying the input text. If you want to get some really cool looking images, you can try some of the top prompts the community put together. https://mpost.io/best-100-stable-diffusion-prompts-the-most-beautiful-ai-text-to-image-prompts/

In the end, we also generated the GIF file for you to visualize what happened between each step.

Image-to-image pipeline

Now, if you continue on the notebooks, you will see we can also use a prompt to ‘influence’ the look of the final image. Here we provided an example of converting our photo into a watercolor painting.

Generating an image based on an initial image and a prompt. This way the result will be guided.

Image-to-image result.

UPDATE (July 19th, 2023)

Stable Diffusion v2 is now available on OpenVINO Notebooks too. We have also included Intel Optimum for HuggingFace. Now you can run almost everything in a few lines of code! =)

openvino_notebooks/notebooks/236-stable-diffusion-v2 at main · openvinotoolkit/openvino_notebooks

UPDATE # 2 (Nov 20th, 2023)

We have added a new model called Latent Consistency Model (LCM), and it works amazingly well with CPU only! Check it out too!

openvino_notebooks/notebooks/263-latent-consistency-models-image-generation at main ·…

Conclusion

Today, if you want to learn how Stable Diffusion works and also see how hardware acceleration works with Intel hardware, the OpenVINO Notebooks is definitely my go-to. If you have any questions or want to show some of your best results, please make a comment here or on our GitHub discussion board! Happy coding.

Discussions · openvinotoolkit/openvino_notebooks

P.S. If you find this useful. Hit the clap button below! Thanks =)

#iamintel

Notices & Disclaimers

Intel technologies may require enabled hardware, software, or service activation.
No product or component can be absolutely secure.
Your costs and results may vary.
Intel does not control or audit third-party data. You should consult other sources to evaluate accuracy.
Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade.
No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document.
© Intel Corporation. Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries. Other names and brands may be claimed as the property of others.