Stable Diffusion with CoreML using the HuggingFace model for iOS, iPadOS, and macOS

Let’s discuss how to convert a custom Stable Diffusion model to the CoreML model, add the necessary resources to our Xcode project, and run it on macOS, iPadOS, and iOS with some parameters.

We will begin with the Xcode project and delve into the details of machine learning.

Preparing Xcode project

Start by creating a macOS app project, then add an iOS and iPad target. Using Cocoa and UIKit will save time. Finally, add ml-stable-diffusion as an SPM package to your Package Dependencies.

Remember these names of files, later we will add them after we finish with converting custom models to CoreML. That’s actually all the ml-stable-diffusion framework requires to start generating images.

We can see it inside the framework.