Using Sky Segmentation to create stunning background animations in iOS

Eric Hsiao
Jun 24 · 4 min read

Starting out

Download and open the repository for this tutorial.

git clone git@github.com:fritzlabs/fritz-ios-tutorials.git
pod repo update
pod install

Overview

We’ll be using the Fritz iOS Image Segmentation feature to generate masks for pets in photos. The Fritz SDK comes with a variety of pre-built features that run directly on your phone.

  • Options: Configuration options that are passed to the model letting you tweak how the model runs.
  • Model: The actual model that runs predictions on the input images.
  • Results: The output of the model. Each predictor has a different type of results. In this tutorial, the results are a list of FritzVisionSegmentationResult objects.

Setup Fritz Account

Setting up a Fritz Account is easy. Follow the Getting Started directions to setup your free account and connect the demo to your account. Here are the steps you’ll run through:

  1. Create an iOS App. Make sure that the Bundle ID of your project matches the one you created.
  2. Drag the Fritz-Info.plist file to your project.

Using Sky Segmentation

Let’s use sky segmentation on an image and then replace the sky pixels with a new, sliding background.

Left: Original Image | Top Middle: Sky photo to replace | Top Bottom: Original image without the sky | Right: Combine background sky with the bitmap.
  • A background image that we’ll use to animate across the sky.
Left: foreground image (mountains.jpg) | Right: background image (clouds..png)
let foreground = UIImage(named: "mountains.jpg")
let background = UIImage(named: "clouds.png")
In step 3, we’ll use these parameters to adjust the model ouput when the buildSingleClassMask method is called.
private lazy var visionModel = FritzVisionSkySegmentationModel()

  • Sky replacement — Compose and edit the background of videos and photos. Want to create a movie poster for Godzilla? Find a photo of a city skyline and swap out the background.
  • Photo effects targeting the sky — Add filters specifically for the sky without the need to apply it to the entire background. Add a reddish glow to the sky as the sun comes up in the morning.

Flawless iOS

🍏 Community around iOS development, mobile design, and marketing

Thanks to Lisa Dziuba.

Eric Hsiao

Written by

Flawless iOS

🍏 Community around iOS development, mobile design, and marketing