Open in app

Sign in

Write

Sign in

Building a PhotoRoom-like background remover app with React Native and Skia 🖌️

Ludwig Henne
13 min readJan 26, 2024

--

Automated background removal has become really popular in mobile apps, from photo editing use-cases like PhotoRoom to more fun social content like the stickers used in Amo ID. Photo cutouts are everywhere and users expect it in more and more apps nowadays.

APIs offer great ways to automatically remove the background of an image, but sometimes you want to manually adjust image cutouts with a brush. Take for example this image: Most automated background removers will cut out the hand and the sneaker but you may want to only keep the sneaker.

Sometimes you want to manually adjust an automatically created cutout

Let’s build this in React Native by using Skia and Reanimated for a smooth 60fps drawing experience.

Here’s what we’re building:

  1. An automatic background removal using a 3rd party API
  2. A cutout editor to draw a mask over the image
  3. A way to undo and redo drawing actions
  4. A small magnifier preview of the area we’re drawing
  5. Saving the new image using Skia’s makeImageSnapshot() method

You can find the code repo here.

We’re building automated background removal with manual mask adjustments

Automatic background removal 🖼️

Getting started

We start from a blank Expo project:

npx create-expo-app -t expo-template-blank-typescript

We want to pick a photo using Expo image picker:

npx expo install expo-image-picker

Lastly, we want to navigate between the cutout editor and the removal result screens with React Navigation:

npm install @react-navigation/native
npx expo install react-native-screens react-native-safe-area-context
npm install @react-navigation/native-stack

Now we can set up two basic screens: Home to pick an image and process the automatic cutout and an Editor to adjust the cutout mask in the next step.

The App.tsx looks like this:

import { NavigationContainer } from '@react-navigation/native';
import { createNativeStackNavigator } from '@react-navigation/native-stack';
import { Home, Editor } from './features';
import { RootStackParamList } from './types';

const Stack = createNativeStackNavigator<RootStackParamList>();

export default function App() {
  return (
    <NavigationContainer>
      <Stack.Navigator>
        <Stack.Screen name="Home" component={Home} />
        <Stack.Screen name="Editor" component={Editor} />
      </Stack.Navigator>
  </NavigationContainer>
  );
}

I’ve added a types.ts file for all the TypeScript declarations and a features folder with one file for each of the two screen.

Picking an image

Next we set up a button that opens the image picker and display the picked image on the Home screen.

import React, { useState } from 'react';
import { View, Image, Button } from 'react-native';
import { HomeScreenPropType } from '../../types';
import { launchImageLibraryAsync, MediaTypeOptions } from 'expo-image-picker';

function Home(props: HomeScreenPropType) {

  const [image, setImage] = useState('');

  const pickImage = async () => {
    const result = await launchImageLibraryAsync({
      mediaTypes: MediaTypeOptions.Images,
      quality: 1,
    });

    if (!result.canceled) {
      setImage(result.assets[0].uri);
    }
  };

  return (
    <View style={{ flex: 1, alignItems: 'center', justifyContent: 'center' }}>
      {image && <Image source={{ uri: image }} style={{ width: 300, height: 300 }} resizeMode="contain" />}
      <Button title="Select image" onPress={pickImage} />
    </View>
  );
}

export default Home;

Make sure you’ve added the media permissions to app.json.

Removing the background with an API

For the automatic background removal, I’m using a commercial API. If you are looking for a DIY solution, check out rembg for Python, while it’s very cheap to run on your own servers, the background removal accuracy will be worse.

There are numerous background removal APIs available, but I’d like to highlight two specifically:

  1. PhotoRoom (highest quality background removal from my experience)
  2. Pixian.ai (great value for money offering. Not quite the cutout quality of PhotoRoom but for a fraction of the price)

I’ll be using PhotoRoom for this example.

Expo filesystem helps us to access local files and store images but also uploading images to an HTTP endpoint.

npx expo install expo-file-system
Automated background removal via API

This call sends the image to the PhotoRoom API and returns the base64 image:

import * as FileSystem from 'expo-file-system';

const URL = 'https://sdk.photoroom.com/v1/segment';

// Please replace with your apiKey
const API_KEY = 'REPLACE_YOUR_API_KEY';

export const removeBackground = async (imageUri: string) => {
  try {
    const response = await FileSystem.uploadAsync(URL, imageUri, {
      fieldName: 'image_file',
      httpMethod: 'POST',
      uploadType: FileSystem.FileSystemUploadType.MULTIPART,
      headers: {
        'x-api-key': API_KEY,
        Accept: 'application/json',
      },
      parameters: {
        size: 'preview',
      },
    });

    const responseData = JSON.parse(response.body);
    return responseData.result_b64;
  } catch (e) {
    console.log(e);
  }
};

⚠️ Of course in a production environment you want to avoid calling the API directly from the client.

Now we’re changing the Home screen to display a loader during the call and the final cutout as a data URL once it’s done:

import React, { useState } from 'react';
import { View, Image, Button, ActivityIndicator } from 'react-native';
import { launchImageLibraryAsync, MediaTypeOptions } from 'expo-image-picker';
import { HomeScreenPropType } from '../../types';
import { removeBackground } from './helpers';

function Home(props: HomeScreenPropType) {
  const [image, setImage] = useState('');
  const [cutout, setCutout] = useState('');
  const [isLoading, setIsLoading] = useState(false);

  const removeBackgroundOnConfirm = async (uri: string) => {
    setIsLoading(true);
    const base64Cutout = await removeBackground(uri);
    setCutout(base64Cutout);
    setIsLoading(false);
  };

  const pickImage = async () => {
    const result = await launchImageLibraryAsync({
      mediaTypes: MediaTypeOptions.Images,
      quality: 0.2,
    });

    if (!result.canceled) {
      const imageUri = result.assets[0].uri;
      setImage(imageUri);
      setCutout('');
      await removeBackgroundOnConfirm(imageUri);
    }
  };

  return (
    <View style={{ flex: 1, alignItems: 'center', justifyContent: 'center' }}>
      {image && !cutout && (
        <Image
          source={{ uri: image }}
          style={{ width: 300, height: 300 }}
          resizeMode="contain"
        />
      )}
      {cutout && (
        <Image
          source={{ uri: `data:image/png;base64,${cutout}` }}
          style={{ width: 300, height: 300 }}
          resizeMode="contain"
        />
      )}
      {isLoading && <ActivityIndicator style={{ marginTop: 20 }} />}
      <Button title="Select image" onPress={pickImage} />
    </View>
  );
}

export default Home;

Cutout mask editor 🖌️

For the mask editor, we need a couple things to work:

  1. Display the original image with the background
  2. Use the cutout image as a mask on top of the original image
  3. Draw paths to the mask to add or remove alpha

Setting up Skia

We’ll be using React Native Skia and Reanimated 3 for the drawing.

npm install @shopify/react-native-skia@latest react-native-reanimated react-native-gesture-handler

Note: We need the latest version of RN Skia, to access notifyChanges() for a real time update of the path values later on. Thus, we’ll need to switch to a development build because Expo Go ships with an older version of Skia at the point of writing (January 2024).

First we need to wrap the navigation into the GestureHandlerRootView:

import { GestureHandlerRootView } from 'react-native-gesture-handler';

//.....

<GestureHandlerRootView style={{ flex: 1 }}>
  <NavigationContainer>
    <Stack.Navigator>
      <Stack.Screen name="Home" component={Home} />
      <Stack.Screen name="Editor" component={Editor} />
    </Stack.Navigator>
  </NavigationContainer>
</GestureHandlerRootView>

Next, we want to navigate to the Editor screen. The Editor screen should receive 2 file URIs: The original image and the cutout. So we need to save the cutout first in a temporary storage.

In helpers.ts we create 2 functions:

export const TEMP_DIR = `${FileSystem.cacheDirectory}images/`;

const imgFileUri = (fileName: string) => `${TEMP_DIR}${fileName}`;

// Checks if temp directory exists. If not, creates it
const ensureDirExists = async () => {
  const dirInfo = await FileSystem.getInfoAsync(TEMP_DIR);
  if (!dirInfo.exists) {
    await FileSystem.makeDirectoryAsync(TEMP_DIR, { intermediates: true });
  }
};

export const saveImageLocally = async (image: ImageBody) => {
  await ensureDirExists();
  const fileUri = imgFileUri(image.fileName);

  await FileSystem.writeAsStringAsync(fileUri, image.base64, {
    encoding: FileSystem.EncodingType.Base64,
  });

  return fileUri;
};

We define a temporary directory to store the image, then we check if the directory exists and create it if it doesn’t exist. We then save the base64 file in the path and return the URI.

In the Home screen, we update the removeBackgroundOnConfirm function:

  const removeBackgroundOnConfirm = async (uri: string) => {
    // ....
    const cutoutUri = await saveImageLocally({
      fileName: 'cutout.png',
      base64: base64Cutout,
    });
    navigation.navigate('Editor', { imageUri: uri, cutoutUri });
  };

Now we navigate to the Editor screen with 2 File URIs ready.
Here’s what our basic Editor.tsx looks like:

import React from 'react';
import { View, useWindowDimensions } from 'react-native';
import { Canvas, useImage, Mask, Image } from '@shopify/react-native-skia';
import { EditorScreenPropType } from '../../types';

function Editor(props: EditorScreenPropType) {
  const { route } = props;
  const cutout = useImage(route.params.cutoutUri);
  const original = useImage(route.params.imageUri);
  const { width } = useWindowDimensions();
  const canvasWidth = width;
  const canvasHeight = width;

  return (
    <View style={{ flex: 1 }}>
      <Canvas
        style={{
          width: canvasWidth,
          height: canvasHeight,
          backgroundColor: 'white',
        }}
      >
        {original && (
          <Mask
            mask={
              <Image
                image={cutout}
                fit="contain"
                width={canvasWidth}
                height={canvasHeight}
              />
            }
          >
            <Image
              image={original}
              fit="contain"
              x={0}
              y={0}
              width={canvasWidth}
              height={canvasHeight}
            />
          </Mask>
        )}
      </Canvas>
    </View>
  );
}

export default Editor;

This displays the cutout as a SkImage inside the Skia Canvas 🥳.
And there is already some magic going on: the image you see is the original one with the background, masked by the cutout. We can now simply modify the mask to reveal or hide parts of the image ✏️

Drawing the mask

To allow drawing on the mask, we’re using Skia paths. We have to wrap the canvas with a GestureDetector and then modify the paths coordinates accordingly. A basic setup looks like this:

import {
  notifyChange,
  Path,
} from '@shopify/react-native-skia';

//...

const currentPath = useSharedValue(Skia.Path.Make().moveTo(0, 0));

const tapDraw = Gesture.Tap().onEnd(e => {
  currentPath.value.moveTo(e.x, e.y).lineTo(e.x, e.y);
  notifyChange(currentPath);
});

const panDraw = Gesture.Pan()
  .averageTouches(true)
  .maxPointers(1)
  .onBegin(e => {
    currentPath.value.moveTo(e.x, e.y);
    notifyChange(currentPath);
  })
  .onChange(e => {
    currentPath.value.lineTo(e.x, e.y);
    notifyChange(currentPath);
  });

const composed = Gesture.Simultaneous(tapDraw, panDraw);

return (
  <View style={{ flex: 1 }}>
    <GestureDetector gesture={composed}>
      <Canvas
        style={{
          width: canvasWidth,
          height: canvasHeight,
          backgroundColor: 'white',
        }}
      >
        {original && cutout && (
          <Mask
            mask={
              <>
                <Image
                  image={cutout}
                  fit="contain"
                  width={canvasWidth}
                  height={canvasHeight}
                />
                <Path
                  path={currentPath}
                  style="stroke"
                  strokeWidth={30}
                  strokeCap="round"
                  blendMode="clear"
                  strokeJoin="round"
                />
              </>
            }
          >
            <Image
              image={original}
              fit="contain"
              x={0}
              y={0}
              width={canvasWidth}
              height={canvasHeight}
            />
          </Mask>
        )}
      </Canvas>
    </GestureDetector>
  </View>
);

What is going on here: We added a GestureDetector for pan and tap gestures, that modify a shared value with a SkPath object. We also render a <Path /> inside the mask on top of the cutout image.

Now we can draw on the canvas and erase parts of the cutout:

Let’s add a switch to change between erasing and drawing. To do so, we have to change the blendMode in the Skia Path:

const [isDrawing, setIsDrawing] = useState(false);

//....

<Path
  path={currentPath}
  style="stroke"
  strokeWidth={30}
  strokeCap="round"
  blendMode={isDrawing ? 'color' : 'clear'}
  strokeJoin="round"
/>

//....

<Switch value={isDrawing} onValueChange={setIsDrawing} />

When you test it now, the current path will either add or remove to the mask based on the switch value. The problem however, is that all of the drawn paths will change their blendMode. What we really want is to only apply to the paths drawn after changing the toggle. To do this, we have to store all past drawn paths in a state.

const [paths, setPaths] = useState<PathWithWidth[]>([]);
const hasUpdatedPathState = useSharedValue(false);

const updatePaths = (currentPathValue: SkPath) => {
  const newPath = {
    path: currentPathValue,
    blendMode: isDrawing ? 'color' : 'clear',
    strokeWidth: 30,
    id: `${Date.now()}`,
  };
  setPaths([...paths, newPath]);
  currentPath.value = Skia.Path.Make().moveTo(0, 0);
  hasUpdatedPathState.value = true;
};

const tapDraw = Gesture.Tap().onEnd(e => {
  currentPath.value.moveTo(e.x, e.y).lineTo(e.x, e.y);
  notifyChange(currentPath);
  runOnJS(updatePaths)(currentPath.value);
});

const panDraw = Gesture.Pan()
  .averageTouches(true)
  .maxPointers(1)
  .onBegin(e => {
    if (hasUpdatedPathState.value) {
      hasUpdatedPathState.value = false;
      currentPath.value = Skia.Path.Make().moveTo(e.x, e.y);
    } else {
      currentPath.value.moveTo(e.x, e.y);
    }
    notifyChange(currentPath);
  })
  .onChange(e => {
    if (hasUpdatedPathState.value) {
      hasUpdatedPathState.value = false;
      currentPath.value = Skia.Path.Make().moveTo(e.x, e.y);
    } else {
      currentPath.value.lineTo(e.x, e.y);
    }
    notifyChange(currentPath);
  })
  .onEnd(() => {
    runOnJS(updatePaths)(currentPath.value);
  });

//....

 <Mask
  mask={
    <>
      <Image
        image={cutout}
        fit="contain"
        width={canvasWidth}
        height={canvasHeight}
      />
      {paths.map(path => (
        <Path
          key={path.id}
          path={path.path}
          style="stroke"
          strokeWidth={path.strokeWidth}
          strokeCap="round"
          blendMode={path.blendMode as any}
          strokeJoin="round"
        />
      ))}
      <Path
        path={currentPath}
        style="stroke"
        strokeWidth={30}
        strokeCap="round"
        blendMode={isDrawing ? 'color' : 'clear'}
        strokeJoin="round"
      />
    </>
  }
>

Tada 🎉! Now we can draw and erase parts of the mask properly.

Undo and redo actions 🔁

You may accidentally draw somewhere you didn’t plan to, so having an undo and redo support is crucial for our editor. We create a simple useUndoRedo hook to achieve this:

export function useUndoRedo<T>(initialState: T[]) {
  const [past, setPast] = useState<T[][]>([]);
  const [present, setPresent] = useState<T[]>(initialState);
  const [future, setFuture] = useState<T[][]>([]);

  const canUndo = past.length > 0;
  const canRedo = future.length > 0;

  const undo = () => {
    if (!canUndo) return;

    const newPast = [...past];
    const newPresent = newPast.pop();

    setPast(newPast);
    setFuture([present, ...future]);
    setPresent(newPresent as T[]);
  };

  const redo = () => {
    if (!canRedo) return;

    const newFuture = [...future];
    const newPresent = newFuture.shift();

    setPast([...past, present]);
    setFuture(newFuture);
    setPresent(newPresent as T[]);
  };

  const updatePresent = (newState: any) => {
    setPast([...past, present]);
    setPresent(newState);
    setFuture([]);
  };

  return [present, undo, redo, updatePresent, canUndo, canRedo] as [
    T[],
    () => void,
    () => void,
    (newState: T[]) => void,
    boolean,
    boolean,
  ];
}

And in the Editor.tsx:

// Replace useState with useUndoRedo

const [paths, undo, redo, setPaths, canUndo, canRedo] = useUndoRedo<PathWithWidth>([]);

We just have to call undo() and redo() on the press of some buttons to change the path state object.

Magnifier view (picture-in-picture) 🔎

A last UX problem we have to solve is that currently it’s not really easy to accurately paint, because your fingers block the view. Most apps have solved this by adding a separate view that shows the area of your draw path.

Adding a magnifier view for more precise drawing

To do this, we basically render a second canvas and transform it oppositely to our pan coordinates.

const overlayX = useSharedValue(0);
const overlayY = useSharedValue(0);
const OVERLAY_WIDTH = 128;
const offset = OVERLAY_WIDTH / 2;

// In the onChange of panDraw we add:
 overlayX.value = -e.x + offset;
 overlayY.value = -e.y + offset - canvasHeight;

// We add animated Styles
const magnifyViewStyle = useAnimatedStyle(() => ({
  transform: [{ translateX: overlayX.value }, { translateY: overlayY.value }],
}));

const overlayViewStyle = useAnimatedStyle(() => ({
  position: 'absolute',
  pointerEvents: 'none',
  top: 0,
  left: 0,
  width: OVERLAY_WIDTH,
  height: OVERLAY_WIDTH,
  overflow: 'hidden',
  backgroundColor: 'white',
  borderWidth: 1,
}));

// And a second set of the Canvas wrapped in the 2 animated views
<Animated.View style={overlayViewStyle}>
  <View style={{ width: canvasWidth, height: canvasHeight }} />
  <Animated.View style={magnifyViewStyle}>
    <Canvas
      style={{
        width: canvasWidth,
        height: canvasHeight,
        backgroundColor: 'white',
      }}
    >
      {original && cutout && (
        <Mask
          mask={
            <>
              <Image
                image={cutout}
                fit="contain"
                width={canvasWidth}
                height={canvasHeight}
              />
              {paths.map(path => (
                <Path
                  key={path.id}
                  path={path.path}
                  style="stroke"
                  strokeWidth={path.strokeWidth}
                  strokeCap="round"
                  blendMode={path.blendMode as any}
                  strokeJoin="round"
                />
              ))}
              <Path
                path={currentPath}
                style="stroke"
                strokeWidth={30}
                strokeCap="round"
                blendMode={isDrawing ? 'color' : 'clear'}
                strokeJoin="round"
              />
            </>
          }
        >
          <Image
            image={original}
            fit="contain"
            x={0}
            y={0}
            width={canvasWidth}
            height={canvasHeight}
          />
        </Mask>
      )}
    </Canvas>
  </Animated.View>
</Animated.View>

We also add a dashed circle in the center of the preview to know the drawing area and we want the magnifier view to only appear while we are drawing. With that in mind, the Editor.tsx looks like this now:

import React, { useState } from 'react';
import { Text, Switch, View, useWindowDimensions, Button } from 'react-native';
import {
  Canvas,
  useImage,
  Mask,
  Image,
  Skia,
  notifyChange,
  Path,
  SkPath,
} from '@shopify/react-native-skia';
import Animated, {
  runOnJS,
  useSharedValue,
  useAnimatedStyle,
} from 'react-native-reanimated';
import { Gesture, GestureDetector } from 'react-native-gesture-handler';
import { EditorScreenPropType, PathWithWidth } from '../../types';
import { useUndoRedo } from '../Home/helpers';

function Editor(props: EditorScreenPropType) {
  const { route } = props;
  const cutout = useImage(route.params.cutoutUri);
  const original = useImage(route.params.imageUri);
  const { width } = useWindowDimensions();
  const [isDrawing, setIsDrawing] = useState(false);
  const [paths, undo, redo, setPaths, canUndo, canRedo] =
    useUndoRedo<PathWithWidth>([]);
  const currentPath = useSharedValue(Skia.Path.Make().moveTo(0, 0));
  const hasUpdatedPathState = useSharedValue(false);
  const isCurrentlyDrawing = useSharedValue(false);
  const canvasWidth = width;
  const canvasHeight = width;

  const overlayX = useSharedValue(0);
  const overlayY = useSharedValue(0);
  const OVERLAY_WIDTH = 128;
  const offset = OVERLAY_WIDTH / 2;

  const updatePaths = (currentPathValue: SkPath) => {
    const newPath = {
      path: currentPathValue,
      blendMode: isDrawing ? 'color' : 'clear',
      strokeWidth: 30,
      id: `${Date.now()}`,
    };
    setPaths([...paths, newPath]);
    currentPath.value = Skia.Path.Make().moveTo(0, 0);
    hasUpdatedPathState.value = true;
  };

  const tapDraw = Gesture.Tap().onEnd(e => {
    currentPath.value.moveTo(e.x, e.y).lineTo(e.x, e.y);
    notifyChange(currentPath);
    runOnJS(updatePaths)(currentPath.value);
  });

  const panDraw = Gesture.Pan()
    .averageTouches(true)
    .maxPointers(1)
    .onBegin(e => {
      if (hasUpdatedPathState.value) {
        hasUpdatedPathState.value = false;
        currentPath.value = Skia.Path.Make().moveTo(e.x, e.y);
      } else {
        currentPath.value.moveTo(e.x, e.y);
      }
      notifyChange(currentPath);
    })
    .onChange(e => {
      if (hasUpdatedPathState.value) {
        hasUpdatedPathState.value = false;
        currentPath.value = Skia.Path.Make().moveTo(e.x, e.y);
      } else {
        currentPath.value.lineTo(e.x, e.y);
      }

      isCurrentlyDrawing.value = true;
      overlayX.value = -e.x + offset;
      overlayY.value = -e.y + offset - canvasHeight;

      notifyChange(currentPath);
    })
    .onEnd(() => {
      isCurrentlyDrawing.value = false;
      runOnJS(updatePaths)(currentPath.value);
    });

  const composed = Gesture.Simultaneous(tapDraw, panDraw);

  const magnifyViewStyle = useAnimatedStyle(() => ({
    transform: [{ translateX: overlayX.value }, { translateY: overlayY.value }],
  }));

  const overlayViewStyle = useAnimatedStyle(() => ({
    position: 'absolute',
    pointerEvents: 'none',
    top: 0,
    left: 0,
    width: OVERLAY_WIDTH,
    height: OVERLAY_WIDTH,
    overflow: 'hidden',
    backgroundColor: 'white',
    borderWidth: 1,
    opacity: isCurrentlyDrawing.value ? 1 : 0,
  }));

  const pointerInOverlayStyle = useAnimatedStyle(() => ({
    width: 30,
    height: 30,
    top: OVERLAY_WIDTH / 2 - 30 / 2,
    position: 'absolute',
    alignSelf: 'center',
  }));

  return (
    <View style={{ flex: 1 }}>
      <View
        style={{
          height: canvasHeight,
          width: canvasWidth,
          position: 'relative',
        }}
      >
        <GestureDetector gesture={composed}>
          <Canvas
            style={{
              width: canvasWidth,
              height: canvasHeight,
              backgroundColor: 'white',
            }}
          >
            {original && cutout && (
              <Mask
                mask={
                  <>
                    <Image
                      image={cutout}
                      fit="contain"
                      width={canvasWidth}
                      height={canvasHeight}
                    />
                    {paths.map(path => (
                      <Path
                        key={path.id}
                        path={path.path}
                        style="stroke"
                        strokeWidth={path.strokeWidth}
                        strokeCap="round"
                        blendMode={path.blendMode as any}
                        strokeJoin="round"
                      />
                    ))}
                    <Path
                      path={currentPath}
                      style="stroke"
                      strokeWidth={30}
                      strokeCap="round"
                      blendMode={isDrawing ? 'color' : 'clear'}
                      strokeJoin="round"
                    />
                  </>
                }
              >
                <Image
                  image={original}
                  fit="contain"
                  x={0}
                  y={0}
                  width={canvasWidth}
                  height={canvasHeight}
                />
              </Mask>
            )}
          </Canvas>
        </GestureDetector>
        <Animated.View style={overlayViewStyle}>
          <View style={{ width: canvasWidth, height: canvasHeight }} />
          <Animated.View style={magnifyViewStyle}>
            <Canvas
              style={{
                width: canvasWidth,
                height: canvasHeight,
                backgroundColor: 'white',
              }}
            >
              {original && cutout && (
                <Mask
                  mask={
                    <>
                      <Image
                        image={cutout}
                        fit="contain"
                        width={canvasWidth}
                        height={canvasHeight}
                      />
                      {paths.map(path => (
                        <Path
                          key={path.id}
                          path={path.path}
                          style="stroke"
                          strokeWidth={path.strokeWidth}
                          strokeCap="round"
                          blendMode={path.blendMode as any}
                          strokeJoin="round"
                        />
                      ))}
                      <Path
                        path={currentPath}
                        style="stroke"
                        strokeWidth={30}
                        strokeCap="round"
                        blendMode={isDrawing ? 'color' : 'clear'}
                        strokeJoin="round"
                      />
                    </>
                  }
                >
                  <Image
                    image={original}
                    fit="contain"
                    x={0}
                    y={0}
                    width={canvasWidth}
                    height={canvasHeight}
                  />
                </Mask>
              )}
            </Canvas>
          </Animated.View>
          <Animated.View style={pointerInOverlayStyle}>
            <View
              style={{
                borderColor: 'blue',
                borderWidth: 3,
                borderStyle: 'dashed',
                width: 30,
                height: 30,
                borderRadius: 30,
                backgroundColor: '#ffffff73',
              }}
            />
          </Animated.View>
        </Animated.View>
      </View>
      <View
        style={{
          justifyContent: 'center',
          flexDirection: 'row',
          alignItems: 'center',
          marginTop: 20,
        }}
      >
        <Text>Erase</Text>
        <Switch value={isDrawing} onValueChange={setIsDrawing} />
        <Text>Draw</Text>
      </View>
      <Button title="Undo" onPress={undo} />
      <Button title="Redo" onPress={redo} />
    </View>
  );
}

export default Editor;

Saving the image 💾

We can take a snapshot of the canvas by using makeImageSnapshot() method of a canvas ref, encode it to base64, store it again in our temporary directory and open a share dialogue:

import { shareAsync } from 'expo-sharing';

// ...

const ref = useCanvasRef();

const onSave = async () => {
  const skiaImage = ref.current?.makeImageSnapshot();
  if (!skiaImage) return;

  const base64 = skiaImage.encodeToBase64(ImageFormat.PNG, 30);
  const fileUri = await saveImageLocally({
    fileName: 'updated-cutout.png',
    base64,
  });
  shareAsync(fileUri);
};

<Canvas ref={ref}>
// ...
</Canvas>

Putting everything together 🚀

Here’s 3 more things I did to polish the feature:

  1. Make the UI pretty with Gluestack UI
  2. Add a slider to change the brush width (to modify the path’s strokeWidth)
  3. Add the original image in the background with a lower opacity so we know what we can erase/restore (make sure it’s not rendered inside the Canvas, otherwise it will be included in the exported image)

The final result

Repo with the full code: https://github.com/ludwighen/react-native-background-remover

Other use cases

Drawing with Skia and Reanimated can have numerous use cases, you can use it for example to

  • Make a painting app with different brush styles
  • Use it for generative AI models that rely on in-painting and creating masks

Happy coding ✨

Skia
React Native
Reanimated
Background Removal
Masks

--

--

Ludwig Henne

Written by Ludwig Henne

7 Followers

Product Manager @Mojo | Building things for fun

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams