Elevate Your Testing Game with Swift Testing 🕊️

Introducing the new Apple’s Swift Testing Framework with Real-World Examples

Introduction

In this article, we explore Apple’s newly launched Swift Testing framework, unveiled at WWDC 2024, and examine how it differs from the well-established XCTest framework.

Swift Testing is specifically designed to align with modern Swift programming practices, offering a more streamlined and “Swifty” approach to testing. Get Ready to Dive In ! 🚀

Technical requirements 🛠️

• You will need XCode 16+ to use Swift Testing
• If you have existing tests written with XCTest, you can run them alongside new tests created with Swift Testing. This approach allows you to migrate your tests incrementally and at your own pace.
• Swift Testing works also in Linux, and Windows, so your tests can behave consistently when moving between platforms.

1- Syntax

The functions in XCTest should always start by test_ and all the XCTestCase assertions use the prefix XCTAssert :

    func test_createMergeRequest_givenSuccess_shouldNotif() {
        
        // XCTest
        XCTAssertTrue(sut.isSonarOK)
        XCTAssertFalse(sut.author == "Alice")
        XCTAssertEqual(sut.author, "Benoit")
        XCTAssertNil(sut.hasThreeApprove)
        XCTAssertLessThan(2,3)
        XCTAssertGreaterThan(5,2)
        XCTAssertLessThanOrEqual(2,3)
        XCTAssertGreaterThanOrEqual(5,5)
    }

In Swift Testing it’s different ! No need to start the name of the function by test_, it only needs to be declared by using the macro @Test :

@Test func createMergeRequest_givenSuccess_shouldNotif() {

        // Swift Testing
        #expect(sut.isSonarOK)
        #expect(sut.author != "Alice")
        #expect(sut.author == "Benoit")
        #expect(sut.hasThreeApprove == nil)
        #expect(2 < 3)
        #expect(5 > 2)
    }

All the XCTCase assertion are replaced by #expect API using Swift expressions and operators.

With XCTest we can only use XCTestCase classes, however, with Swift Testing you can use struct, final class, enum or actor :

final class XCTestMergeRequestTests: XCTestCase { 
  // ... XCTest 
}

struct SwiftMergeRequestTests {
  // Swift Testing  
}
actor SwiftMergeRequestTests {
  // Swift Testing  
}
enum SwiftMergeRequestTests {
  // Swift Testing  
}

In XCTestCase you can configure properties before and after each test run by using setUp & tearDown:

    override func setUp() {
        super.setUp()
        // Before testing function, configure something...
        sut = MergeRequest()
        sut.validators = ["Alain", "Alex", "Alice"]
        sut.author = "Benoit"
    }

    override func tearDown() {
        super.tearDown()
        sut = nil
    }

With Swift Testing these methods do not exist. Use init and deinit instead.

2- Parameterized tests 🔄

With XCTest we can re-run the same test using different values using loops :

 func test_verifyValidatorIdentityForAllDevs_givenSuccess_shouldAlert() {
        let iOSDevs = ["Soraya", "Teddy", "Antoine", "Julien", "Alice"]

        iOSDevs.forEach { dev in
            sut.author = dev
            XCTAssertFalse(sut.authorIsAValidator())
        }
}

Parameterized tests in Swift Testing framework allow you to execute the same test across a series of different values, enabling you to write less code. Additionally, all tests are fully integrated with Swift Concurrency.

@Test("Given different MR validation status type, should display alert",
      arguments: [MRValidation.rejected,.oneThreadIsOpen, .none])
func verifyMRValidatonStatus_givenSuccess_shouldAlert(validation: MRValidation) {
    if sut.status == .merged {
         #expect(sut.validationStatus !=  validation)
     }
}

Arguments are independent of each other, meaning that you can re-run individual test cases, and you can run them in parallel as well 🤩.

We can pass multiple arguments as parameters :

@Test(arguments: [MRValidation.rejected, .validated],[MRStatus.closed, .merged, .open, .draft])
  func verifyValidatorIdentity_multipleArgs(validation : MRValidation, status : MRStatus) {
        //... Tuples : (validation, status) 
    }

A test function with two arguments will test every possible combination, in our case eight in total :

(.rejected, .closed) (.rejected, .merged)(.rejected, .open) (.rejected, .draft) & (.validated, .closed) (.validated, .merged)(.validated, .open) (.validated, .draft)

The outer limits :

As you add more inputs to the two sets, the number of test cases will increase exponentially!! To help manage this rapid growth, test functions are designed to accept a maximum of two collections.

💡 You can also use the swift standard lib’s ZIP function to match up pairs of inputs that should go together instead of testing every combination !

@Test(arguments: zip([MRValidation.rejected, .validated], 
                      [MRStatus.closed, .merged, MRStatus.open, .draft]))
  func verifyValidatorIdentityForAll_multipleArgsWithZip(validation : MRValidation, status : MRStatus) {
    // use the tuple : (validation, status)
}

Zip will produce a sequence of tuples, where each element from the first collection is paired with its corresponding element in the second collection.

(.rejected, .closed) & (.validated, .merged)

3- Throwing and catching errors ⚠️❗

Error handling is often less tested but is an important part of the user’s experience.

When you are testing your code’s happy path ✅ and are expecting a throwing function to return successfully, just call it inside your test.

If the function does end up throwing an error, the test will fail.

Let’s take an exemple :

extension MergeRequest {

func merge() throws -> Bool {
        if isSonarOK == false { throw MRErrors.codeSmells }
        if isBitriseOK == false { throw MRErrors.failingPipeline  }
        return true
    }
}

enum MRErrors: Error {
    case codeSmells
    case failingPipeline
}

@Test mutating func mergeVerification_withThrows () throws {
        sut.isSonarOK = false
        let canMerge = try sut.merge()

        #expect(sut.status != .merged)
    }

When the test is executed, the try sut.merge() will throw an error and all the test will fail and be interrupted :

The #expect assertion will not even be executed !

You could add your own do catch statement around the code and examine the error :

 @Test mutating func mergeVerification_withTryCatch () throws {
        sut.isSonarOK = false

        do {
            let canMerge = try sut.merge()
        } catch {
            Issue.record("Unexpected Error") // Issue.record replaces XCTFail
        }
        #expect(sut.status != .merged)
    }

This code is verbose and doesn’t resolve the problem, it won’t let you know what went wrong !

Swift Testing is here to help with expect throws macro 🎯😎 :

  @Test mutating func mergeVerification_withExpectMacro () throws {
        sut.isSonarOK = false

        #expect(throws: (any Error).self) {
            let _ = try sut.merge()
        }
        #expect(sut.status != .merged)
    }

#Require macro :

Use try #require to halt the test if the condition evaluates to false.

The #require expectations function similarly to regular expectations but include the try keyword and will throw an error if the expression is false causing the test to fail and not proceed any further.

 @Test func mergeVerification_RequireMacro() throws {
      try #require(sut.isMRFlagActivated)
       #expect(sut.status != .merged)// will not be executed if isMRFlagActivated == false
  }

How to deal with optionals :

Another way to utilize the #require macro is for unwrapping optional values, allowing you to stop the test if the value is nil.

@Test func mergeVerification_RequireMacro_UnwrapUseCase() throws {
      let mySut = try #require(sut)
      #expect(mySut.status != .merged)// will not be executed if sut is nil
 }

Require expectation failed

4- Organizing tests

Suites :

Group related test functions or other Suites, their functionality can be documented by Traits (see next session), such as, a display name.

They could be annotated explicitly using @Suite although any type that contains @Test functions or @Suite is considerate as Suite itself implicitly.

Adding a sub Suite to organize the Tests by business logic : we will group all the merge request verification in a new sub suite for example :

@Suite struct SwiftMergeRequestTests {
  // some test func here ...

@Suite struct MergeRequestTestsSubSuite {
    var sut : MergeRequest!
    init() {
            sut = MergeRequest()
            sut.validators = ["Alice", "Alex", "Alain"]
            sut.author = "Benoit"
        }
        @Test mutating func mergeVerification_withThrows () throws {
            //...
        }
        @Test mutating func mergeVerification_withTryCatch () throws {
            // ...
        }
        @Test mutating func mergeVerification_withExpectMacro () throws {
            // ...
        }
  }
}

Tests with a sub Suite

Traits :

Traits enhance a test with descriptive information, allow customization of its execution, and modify its behavior.

Let’s take a look at Toggle Trait for example :

You might want to write a test that only runs on devices with particular hardware capabilities, or other particular conditions. Toggle traits allow you to force runners to automatically skip the test if conditions like these are not met :

 @Test(.enabled(if: RemoteFeatureFlag.MergeRequest.isActivated))
    func mergeVerification_ifFeatureFlagActivated() throw {
        #expect(throws: (any Error).self) {
            let _ = try sut.merge()
        }
        #expect(sut.status != .merged)
    }

It’s also possible to conditionally disable a test and to combine multiple conditions:

@Test(.enabled(if: RemoteFeatureFlag.MergeRequest.isActivated),
      .disabled("Currently we have a problem in Bitrise")))
    func mergeVerification_ifFeatureFlagActivated() throw {

        #expect(throws: (any Error).self) {
            let _ = try sut.merge()
        }
        #expect(sut.status != .merged)
    }

Here the test will be skipped :

If a test is temporarily disabled due to an issue with an associated bug report, you can use specific Bug Trait to establish the connection between the test and the bug report.

@Test(.enabled(if: RemoteFeatureFlag.MergeRequest.isActivated),
      .disabled("Currently we have a problem in Bitrise"),
      .bug(id :"AT-1524"))
    func mergeVerification_ifFeatureFlagActivated() throw {

        #expect(throws: (any Error).self) {
            let _ = try sut.merge()
        }
        #expect(sut.status != .merged)
    }

Tag Trait

Tags are used to group tests that are scattered across different files or simply not organized within the same @Suite of a single file.

This trait accepts a sequence of tags as its argument, which are then associated with the corresponding test at runtime.

Where Tags can be declared?
Tags must always be declared as members of the Tag type, either in an extension of that type or within a type that is nested inside Tag :

extension Tag {
  @Tag static var contains_Authentification: Self
  @Tag static var contains_SCA: Self
}
//                           ...

@Suite struct SwiftMergeRequestTests {
  // some test func here ...

@Suite(.tags(.contains_Authentification)) struct MergeRequestTestsSubSuite {
    var sut : MergeRequest!
    init() {
            sut = MergeRequest()
            sut.validators = ["Alice", "Alex", "Alain"]
            sut.author = "Benoit"
        }
        @Test mutating func mergeVerification_withThrows () throws {
            //...
        }
        @Test mutating func mergeVerification_withTryCatch () throws {
            // ...
        }
        @Test mutating func mergeVerification_withExpectMacro () throws {
            // ...
        }
  }
}

All the tests included in MergeRequestTestsSubSuite will be tagged under contains_Authentification tag :

Tags were been created in the Tests project

We can now run all the tests with a particular tag and we can also filter them in Test Reports 😍🥳.

Conclusion :

The introduction of the Swift Testing framework marks a significant advancement in the way developers approach testing in iOS applications.

With its focus on simplicity, efficiency, and seamless integration with modern Swift practices, Swift Testing empowers developers to write cleaner and more maintainable tests.

The ability to use parameterized tests, alongside the flexibility of tags and suites, allows for a more organized and effective testing strategy.

Moreover, the framework’s compatibility with existing XCTest tests facilitates a smooth transition, enabling developers to adopt this innovative tool at their own pace

References :

WWDC24 session : Meet Swift Testing

WWDC24 session : Go further with Swift Testing

Swift Github Repo

Swift Testing documentation