Effortless Image Generation with Optimum-Intel OpenVINO™: Accelerating Stable Diffusion in a Few Lines of Code
Authors: Edge AI Evangelist Team @ Intel
As AIGC (Artificial Intelligence and Generative Creativity) models continue to advance in power and achieve remarkable results, using models like Stable Diffusion still poses challenges for developers. Firstly, the intricate setup of GPUs necessitates navigating complex configurations and compatibility issues, which can be time-consuming and frustrating. Additionally, employing Stable Diffusion models with their complex environment setup procedures introduces further difficulties, as developers are often drawn to clean and intuitive APIs that enable effortless interaction with models, streamlining the workflow. Lastly, the pursuit of hardware acceleration without requiring complicated coding and compilation remains a top priority, as we developers constantly seek efficient and straightforward solutions to fully leverage the potential of AIGC algorithms.
In this blog post, we will explore the quickest way (using the fewest line of code and dependencies installations) to enable AI hardware acceleration on Stable Diffusion V2.1 using Optimum-Intel (OpenVINO). We will also guide you through the process of installation and utilization for running Stable Diffusion on your local machine, optimized and accelerated with OpenVINO 2023.0 release. With just a few lines of Python code, you can generate images with text within seconds. Also, OpenVINO makes it simple to accelerate the work across different hardware platforms ranging from Intel Desktop CPU, iGPU, dGPU, and Xeon CPU, making your work much more flexible on deployment. For more details of the approach, please check out our OpenVINO Notebooks repository on GitHub here.
1. Installation:
Use the following 1 line of code to install Optimum-Intel[OpenVINO]:
pip install -q "optimum-intel[openvino,diffusers]"@git+https://github.com/huggingface/optimum-intel.git
2. Download Pre-Converted Stable Diffusion 2.1 IR
Stable Diffusion pipeline brings 6 elements together, a text encoder model with a tokenizer, a UNet model with a scheduler, and an Autoencoder with Decoder and Encoder models.
The base model used here is the “stabilityai/stable-diffusion-2–1-base”. This model was converted to OpenVINO format, for accelerated inference on CPU or Intel GPU with Optimum-Intel[OpenVINO]. The model weights are stored with FP16 precision, which reduces the size of the model by half. You can find the model used in this notebook is “helenai/stabilityai-stable-diffusion-2–1-base-ov”. Let’s download the pre-converted model Srable Diffusion 2.1 Intermediate Representation Format (IR) with the following code.
from optimum.intel.openvino import OVStableDiffusionPipeline
# download the pre-converted SD v2.1 model from Helena's HF repo
name = "helenai/stabilityai-stable-diffusion-2-1-base-ov"
pipe = OVStableDiffusionPipeline.from_pretrained(name, compile=False)
pipe.reshape(batch_size=1, height=512, width=512, num_images_per_prompt=1)
3. Save the pre-trained models, select the inference device and compile it
We have downloaded the pre-trained and converted IR format Stable Diffusion model. Let’s save the model and compile it for AI inferencing with the following code:
pipe.save_pretrained("./openvino_ir")
pipe.to("GPU.1")
pipe.compile()
4. Be creative, add the prompt and enjoy the result
Now it’s time to be creative! Enter your prompt with creative idea, and enjoy the generated image with the following:
# Generate an image.
prompt = "red car in snowy forest, epic vista, beautiful landscape, 4k, 8k"
output = pipe(prompt, num_inference_steps=17, output_type="pil").images[0]
output.save("image.png")
output
Here’s the image generation process over the number of inferencing steps:
Take Away / What’s Next?:
AI generated content (AIGC) has attracted tremendous attention. In this rapidly evolving AI landscape, efficient and optimized inferencing is crucial for unlocking the true potential of AIGC models for your business. In this blog we showed Optimum-Intel provides a simple interface to optimize the HuggingFace Transformers and Diffusers models, convert them to the OpenVINO™ Intermediate Representation (IR) format and run inference using OpenVINO Runtime. While OpenVINO is a comprehensive open-source software toolkit designed to optimize and accelerate AI inferencing workloads. It enables developers to deploy deep learning models efficiently across diverse hardware platforms, including CPUs, GPUs, FPGAs, etc.
Additional Resources:
Test out the code on DevCloud for the Edge
Provide Feedback & Report Issues
Acknowledgement
Special thanks to Helena Kloosterman for their invaluable guidance and support throughout the development.
Notices & Disclaimers
Intel technologies may require enabled hardware, software or service activation.
No product or component can be absolutely secure.
Your costs and results may vary.
© Intel Corporation. Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries. Other names and brands may be claimed as the property of others.