Image rotation tool on SpriteKit

I think everyone of you used tools for changing image geometry. Basically all editors have that and the iPhone base image editor is not an exception. This tool helps you rotate an image around axes, mirror, crop and stuff like that. Today we are going to implement a similar tool but, to make it more interesting, we will do it with SpriteKit. Also if we ever want to add to our tool some cool photo editing features involving Metal we will have an advantage: MTLTexture from Metal easily converts to SpriteKit’s SKtexture and vise versa, so we will be able to use this to boost performance.

By the end of this article we will have an app capable of this:

Final app demo

Okay, let’s get started. For those of you who have never worked with SpriteKit i’ll introduce some new objects that are not present in UIKIt — SKScene, SKCameraNode and SKView. You can read general info in apple docs, here i’ll try to explain just how it works. I suppose in your childhood you played 2D games where your character appears at the beginning of a level and you need to move all the way to the right while overcoming different obstacles and fighting enemies. So, this whole level, together with our character, mobs and environment is SKScene. The portion of the screen we see at one time is determined by SKCameraNode which acts pretty much the same as a real camera would. It follows our character along a 2D scene, it can zoom in and out. In turn SKView is required to integrate SKScene into UIKit.

Our basic setup is very easy:

final class PhotoEditorScene: SKScene {
    
    private let imageNode: SKSpriteNode
    private let cameraNode = SKCameraNode()
    
    init() {
        self.imageNode = SKSpriteNode(texture: SKTexture(image: UIImage(named: "default_img")!))
        
        super.init(size: .zero)
        
        self.camera = self.cameraNode
        self.addChild(self.cameraNode)
        self.addChild(self.imageNode)

    }
    
    override func didChangeSize(_ oldSize: CGSize) {
        super.didChangeSize(oldSize)
        // Fit rect in self.size 
        self.imageNode.size = fittedSize
    }
    
    override public func didMove(to view: SKView) {
        // setup gestures
    }
}

Run this code (you will need to add some small things yourself) and you will see your image.

Handling gestures

To handle gestures we could either change size and position of an image node or scale and position of the camera. We will go with the second option. While these two approaches might seem similar now, camera will be much more convenient if nodes hierarchy gets more complicated with adding new features.

Implementing drag gesture is straightforward:

func handle(_ sender: UIPanGestureRecognizer) {
    guard let scene = self.scene else {
        return
    }
    // Convert gesture location from view to scene
    let location = scene.convertPoint(fromView: sender.location(in: sender.view))

    switch sender.state {
    case .began:
        self.startLocation = location
    case .changed:
        let translation = location - self.startLocation
        self.cameraNode.position += translation * -1
    case .cancelled,
         .failed,
         .ended:
        break
    default:
        break
    }
}

We are converting coordinates from view to scene, don’t forget to do that. Also keep in mind that in the SpriteKit coordinate system y axis is oriented from bottom to top, opposite to UIKit.

Zoom gesture just a little bit more trickier:

private var startScale: CGFloat = 1
private var startLocation: CGPoint = .zero

func handle(_ sender: UIPinchGestureRecognizer) {
    guard let scene = self.scene else {
        return
    }

    let location = scene.convertPoint(fromView: sender.location(in: sender.view))
    // scale for camera must be inverted.
    let scale = self.startScale * 1 / sender.scale

    switch sender.state {
    case .began:
        self.startScale = self.cameraNode.xScale
        self.startLocation = location
    case .changed:
        guard sender.numberOfTouches == 2 else { return }

        let translation = location - self.startLocation
        // Otherwise we will always zoom to the middle of the image
        // despite gesture center point 
        self.cameraNode.position += translation * -1
        self.cameraNode.setScale(scale)
    case .cancelled,
         .failed,
         .ended:
        self.onComplete?()
    default:
        break
    }
}

Besides scale we shift camera position to maintain zoom point.

Rotating

Firstly, let’s rotate an image around z axis (screen plane):

self.node.zRotation = radians

Well, that wasn’t hard! What about x and y axis? We can’t actually rotate an image in space with SpriteKit — this is a 2D engine after all, what we can do is fake it. The idea is to warp an image in a way that it would look like it was rotated in space. For this purpose, let’s take a close look at SKWarpGeometry. First we need to create a SKWarpGeometryGrid.

SKWarpGeometryGrid(
    columns: Int, 
    rows: Int,
    sourcePositions: [SIMD2<Float>], 
    destinationPositions: [SIMD2<Float>]
)

Source positions are coordinates of an image corners and destination positions are coordinates after warping. Regarding first two parameters, docs state “the number of vertices required for a given dimension is equal to (cols + 1) *(rows + 1)”, we have 4 vertices then columns = rows = 1. For example, let’s rotate around y-axis and find how vertices will change:

(0, 0) -> (-dx, -dy)
(1, 0) -> (1, 0)
(0, 1) -> (-dx, 1 + dy)
(1, 1) -> (1, 1)

where dx and dy numbers somehow depends on rotation angle α.

Normalised vertices before rotation(left). Vertices after rotating around y-axis (right).

We can try to use following formula (other options might be fine as well)

dx = sin(α) * 0.4
dy = sin(α) * 0.5

and see what happens. Making simular reasoning as above we can derive formula to find warped vertices from rotation angle.

private static func applyWarpToNormalizedSquare(
   to unwarpedVertices: [SIMD2<Float>],
   angleX: CGFloat,
   angleY: CGFloat
) -> [SIMD2<Float>] {
    var bottomLeft = normalizedSquare[0]
    var bottomRight = normalizedSquare[1]
    var topLeft = normalizedSquare[2]
    var topRight = normalizedSquare[3]
    
    let xw = Float(sin(angleY) * 0.4)
    let xh = Float(sin(angleY) * 0.5)

    if rotation.y > 0 {
        topRight.x += xw
        bottomRight.x += xw
        topRight.y += xh
        bottomRight.y -= xh
    } else if rotation.y < 0 {
        topLeft.x += xw
        bottomLeft.x += xw
        topLeft.y -= xh
        bottomLeft.y += xh
    }

    let yw = Float(sin(angleX) * 0.5)
    let yh = Float(sin(angleX) * 0.4)

    if rotation.x > 0 {
        topLeft.x -= yw
        topRight.x += yw
        topLeft.y += yh
        topRight.y += yh
    } else {
        bottomLeft.x += yw
        bottomRight.x -= yw
        bottomLeft.y += yh
        bottomRight.y += yh
    }

    return [bottomLeft, bottomRight, topLeft, topRight]
}

Generally it is done, however one caveat remains. To spot that you need to load an image that has straight lines and apply warp effect (rotate).

Source image (left). Warped image (right).

As you can see on the warped image straight lines become wavy. That’s not good! Looks like this thing approximates warping under the hood and does this not very smoothly. After a little bit of search you can find this guy

/* maximum number of subdivision iterations used to generate the final vertices */
var subdivisionLevels: Int { get set }

Let’s try to set extreme values to this parameter and see what hapens. Setting this value to 0 provides us with quite an interesting image which we can use to speculate about inner mechanisms. Experimentally I found that 6 is the maximum number, setting anything greater will cause the app to crash with unclear error. If someone can explain the reason it will be very interesting to hear.

subdivisionLevels 0 (left) and 6 (right).

Rotation of zoomed image & mirror

Almost done here, just one last improvement. Try to zoom an image to some point and then rotate it. Zoomed area will move out of sight because the image rotated around its center and not around the visible area center. There are several ways to fix this and after playing around i have concluded that the easiest one is to add an additional node that will contain an image node. Instead of rotating the image we will rotate the container but before that we can shift an image and container in the way to rotate around the observation point. This trick can be applied to any point related operation, for example mirroring an image relative to the vertical axis.

private func prepareForTransform(relativeTo point: CGPoint) -> CGPoint {
    let pointInNode = self.scene!.convert(point, to: self.imageNode)

    guard let scene else { return .zero }
    self.imageContainerNode.position = scene.convert(point, to: self)
    self.imageNode.position = scene.convert(.zero, to: self.imageContainerNode)

    return pointInNode
}

private func cleanUpAfterTranform(initialPointInNode: CGPoint, initialPointInScene: CGPoint) {
    guard let scene, let cam = self.scene?.camera else { return }
    self.imageContainerNode.position = .zero
    self.imageNode.position = .zero

    let newPointInScene = scene.convert(initialPointInNode, from: self.imageNode)
    let delta = newPointInScene - initialPointInScene
    cam.position = CGPoint(
        x: cam.position.x + delta.x,
        y: cam.position.y + delta.y
    )
}

// usage

func mirrorX(relativeTo point: CGPoint) {
    let pointInNode = self.prepareForTransform(relativeTo: point)
    self.xScale *= -1
    self.cleanUpAfterTranform(initialPointInNode: pointInNode, initialPointInScene: point)
}

You can find souce code here https://github.com/rodionov-andrei/rotationToolSpriteKit/