Writing test code using lit for Swift Compiler

Hi all, I’m @kitasuke, iOS Engineer.

I summarized how to write test code using lit(LLVM Integrated Tester) for Swift Compiler mainly for myself.

In this post, I’ll use my PR as example. I think this one is a good start to see how we can write test code using lit, because this one really does basic.

[SIL] Don't print duplicated import decls by kitasuke · Pull Request #17084 · apple/swift

See below link for more details.

lit - LLVM Integrated Tester - LLVM 9 documentation

What to test

I’ll explain what I did in the PR.

Background

I noticed that I got duplicated import declarations for specific ones when I emitted canonical SIL from raw SIL using swiftc like below.

$ swiftc -emit-sil input.sil
import Builtin
import Swift
import SwiftShims
import TestModule
import Builtin // duplicated
import Swift // duplicated
import SwiftShims // duplicated

Below imports are expected behaviour.

This didn’t break anything, but it’s nice to remove duplicated one. That’s what I did in my PR.

Test code using lit

Here is my test code using lit.

What we wanted to test is to check if there are any duplicated imports using lit.

// RUN: %empty-directory(%t)
// RUN: touch %t/empty.swift
// RUN: %target-swift-frontend -emit-module -module-name TestModule %t/empty.swift -o %t
// RUN: %target-sil-opt %s -I=%t | %FileCheck %s
sil_stage raw
import Builtin
import Swift
import TestModule
func foo() {}
// CHECK: import Builtin
// CHECK: import Swift
// CHECK-NOT: import Builtin
// CHECK-NOT: import Swift{{$}}
// CHECK: import TestModule

I’ll explain one by one.

RUN

lit executes specified command right after RUN:.

%empty-directory(%t) creates empty directry under tmp name which %t generates for test.

touch %t/empty.swift creates empty empty.swift file under the directory.

%target-swift-frontend -emit-module -module-name TestModule %t/empty.swift -o %t generates TestModulemodule which contains the empty file.

%target-sil-opt %s -I=%t | %FileCheck %s executes sil-opt with import search path added by -I option and pass the output to %FileCheck in specified %s path.

sil-opt

sil-opt is a module dedicated for SIL.

apple/swift

In this case, raw SIL is input.

sil_stage raw
import Builtin
import Swift
import TestModule
func foo() {}

If you run %target-sil-opt %s -I=%t without the fix, you get below raw SIL.

sil_stage raw
import Builtin
import Swift
import Builtin
import Swift
import SwiftShims
import TestModule
func foo() {}

FileCheck

FileCheck reads two files (one from standard input, and one specified on the command line) and uses one to verify the other. This behavior is particularly useful for the testsuite, which wants to verify that the output of some tool (e.g. llc) contains the expected information (for example, a movsd from esp or whatever is interesting). This is similar to using grep, but it is optimized for matching multiple different inputs in one file in a specific order.

FileCheck - Flexible pattern matching file verifier - LLVM 9 documentation

CHECK: checks if specified string is contained. CHECK-NOT: checks if specified string is NOT contained. {{pattern}}can be used for regular expression for the string.

First two CHECK:s check if there are import Builtin and import Swift.

Next two CHECK-NOT:s check if there are NOT import Builtin and import Swift{{$}}. In case there are duplicated imports here, this test is failed.

Last CHECK: checks if there is import TestModule.

How to test

Run below command to test specific file. In this case, build directory is Ninja-DebugAssert for Debug environment.

$ utils/run-test --build-dir ../build/Ninja-DebugAssert test/SIL/import-decls.sil

If you want to skip building again, add --build=skip option.

Then, you can see the test passed.

Expected Passes    : 1

Takeaway

This might be first time to see test code using lit, but it’s less difficult than expected. In this case, you see SIL file as input, but swift code can be input as well.

If you are interested in test code for Swift Compiler, I highly recommend to check out test directory. I think it makes us understand the compiler better if we could also understand what test code does.

Happy testing using lit!