Using Sky Segmentation to create stunning background animations in iOS

Eric Hsiao
Jun 24 · 4 min read

Starting out

Download and open the repository for this tutorial.

git clone
pod repo update
pod install


We’ll be using the Fritz iOS Image Segmentation feature to generate masks for pets in photos. The Fritz SDK comes with a variety of pre-built features that run directly on your phone.

  • Options: Configuration options that are passed to the model letting you tweak how the model runs.
  • Model: The actual model that runs predictions on the input images.
  • Results: The output of the model. Each predictor has a different type of results. In this tutorial, the results are a list of FritzVisionSegmentationResult objects.

Setup Fritz Account

Setting up a Fritz Account is easy. Follow the Getting Started directions to setup your free account and connect the demo to your account. Here are the steps you’ll run through:

  1. Create an iOS App. Make sure that the Bundle ID of your project matches the one you created.
  2. Drag the Fritz-Info.plist file to your project.

Using Sky Segmentation

Let’s use sky segmentation on an image and then replace the sky pixels with a new, sliding background.

Left: Original Image | Top Middle: Sky photo to replace | Top Bottom: Original image without the sky | Right: Combine background sky with the bitmap.
  • A background image that we’ll use to animate across the sky.
Left: foreground image (mountains.jpg) | Right: background image (clouds..png)
let foreground = UIImage(named: "mountains.jpg")
let background = UIImage(named: "clouds.png")
In step 3, we’ll use these parameters to adjust the model ouput when the buildSingleClassMask method is called.
private lazy var visionModel = FritzVisionSkySegmentationModel()

  • Sky replacement — Compose and edit the background of videos and photos. Want to create a movie poster for Godzilla? Find a photo of a city skyline and swap out the background.
  • Photo effects targeting the sky — Add filters specifically for the sky without the need to apply it to the entire background. Add a reddish glow to the sky as the sun comes up in the morning.

Flawless iOS

🍏 Community around iOS development, mobile design, and marketing

Thanks to Lisa Dziuba.

Eric Hsiao

Written by

Flawless iOS

🍏 Community around iOS development, mobile design, and marketing