A Complete Guide to Running Swin Transformer Models on iOS Using CoreML

Real-World Use Cases: Implementing Swin Transformer for Vision Tasks on iOS.

Photo by Kevin Bhagat on Unsplash

In my last article I demonstrated how to train a Swin Transformer model. Now that we have a Swin Transformer model, we may want to use it in an actual production setting. One way we can use the model is to locally process photos on an iPhone, but to do that we must first convert the model to the CoreML format.

System

• MacOS
• Python 3.10
• PyTorch 2.5.0
• CoreMLTools 8

PyTorch To CoreML

The method of conversion occasionally changes with the version. For this example I am using CoreMLTools version 8. The current supported version of Torch for CoreMLTools is 2.5.0, so that is the version we use.

While you can convert the model without a Mac, performing inference on the CoreML model will require an Apple device.

To convert we make a simple conversion script that exports the PyTorch model to a CoreML Model.

import torch
import coremltools as ct
import torch.nn as nn
import timm

# Define the model using Swin Transformer Large, this is just the class from the training script
class…