By João Pedro de Amorim, Senior iOS Engineer, MEGA
Introduction
Here at MEGA, in order to build a consistent UI experience throughout all of our products on different platforms (Android, desktop, iOS, and our websites) it became clear that we needed to create a robust, consistent, and sustainable design system for our codebase.
With that goal in mind, we’ve decided to adopt an approach where design tokens (the basic building block of any design system: colours, radius, spacing, etc.) are defined in JSON specs given by the design team, in a way they can be used in a platform-agnostic manner.
In this article, we’ll deep dive into how we built MEGADesignToken - the Swift Package Manager (SPM) module responsible for creating and exposing Swift code based on those files, tackling topics such as: SPM Build Tool plugins, parsing strategies, data modeling, algorithms and code generation through SwiftSyntax.
Requirements
Our design team provided us with three JSON files named core.json, Semantic tokens.Dark.tokens.json and Semantic tokens.Light.tokens.json - each file has a distinct structure and represents a given type of design token.
core.json
As the name implies, this file represents the core tokens of our system — foundational tokens that have a fixed value and that are referenced by semantic tokens. Here’s a simplified version of how this file looks like:
{
"Colors": {
"Black opacity": {
"090": {
"$type": "color",
"$value": "rgba(0, 0, 0, 0.9000)"
},
"080": {
"$type": "color",
"$value": "rgba(0, 0, 0, 0.8000)"
}
},
"Secondary": {
"Orange": {
"100": {
"$type": "color",
"$value": "#ffead5"
}
}
}
},
"Spacing": {
"1": {
"$type": "number",
"$value": 2
}
},
"Radius": {
"--border-radius-circle": {
"$type": "number",
"$value": 0.5
}
}
}It consists of three main keys: Colors, Spacing and Radius:
Colorscan have categories - such asBlack opacity- containing an array of colour information, or have deeply nested categories - such asSecondary, that containsOrange, another category. The colour information has its value represented by aStringcontainingrgbaorhexvalues.SpacingandRadiusonly contains an array of number information.
Semantic tokens.Dark.json / Semantic tokens.light.json
In a similar fashion, as the name implies, both of these files contain semantic token information in their dark/light theme variation. Here’s a simplified version of the structure of these files:
{
"Focus": {
"--color-focus": {
"$type": "color",
"$value": "{Colors.Secondary.Indigo.200}"
}
},
"Indicator": {
"--color-indicator-magenta": {
"$type": "color",
"$value": "{Colors.Secondary.Magenta.500}"
},
"--color-indicator-yellow": {
"$type": "color",
"$value": "{Colors.Warning.500}"
}
},
"Button": {
"--color-button-disabled": {
"$type": "color",
"$value": "{Colors.Black opacity.010}"
}
}
}In it, we have different categories — Focus, Indicator and Button - containing an array of semantic color information. Values, instead of a String representing a rbga or hex code, reference a core color from the core.json.
Based on these files, the requirement is to create a build tool that will parse these files and generate Swift code that represents them (similar to SwiftGen with .strings files).
Another requirement is that for colour tokens, we must only generate code for their semantic values (in other words, we want to only publicly expose the semantic palette).
Now that we have a clear scope of what we want to build, how to achieve this?
Let’s build it!
Swift Package Manager (SPM) Build Tool plugins
In order to talk about SPM Build Tool plugins, it’s necessary to define in the first place what a SPM plugin is — for a formal definition, you should take a look at SPM’s official documentation about plugins.
But in other — less formal — words, you can think of SPM plugins as modern Swift way to create command line executables and build scripts. For instance, you could create a SwiftLint Build Tool plugin that lints all of your files during a build - this approach has some advantages when compared to the old-school Add Run Script build phase:
- You don’t need to make developers manually install the dependencies of your script (e.g.
swiftlint) - this will be automatically handled by SPM once you declare them as a dependency of your plugin - Your developer tooling infrastructure (linters, formatters, code generation tools, etc.) will all be written in
Swiftand totally integrated with Xcode
Therefore, in order to accomplish our goal to generate Swift code based JSON files, we've decided to create a build tool plugin called TokenCodegen. It's a build tool plugin that runs every time our MEGADesignToken module is built. TokenCodegen depends on an executable called TokenCodegenGenerator. To better understand this structure, let's take a look at MEGADesignToken's package.swift file:
// swift-tools-version: 5.9
import PackageDescription
let package = Package(
name: "MEGADesignToken",
platforms: [
.iOS(.v14),
.macOS(.v12)
],
products: [
.library(
name: "MEGADesignToken",
targets: ["MEGADesignToken"]
),
.plugin(
name: "TokenCodegen",
targets: ["TokenCodegen"]
)
],
dependencies: [
.package(url: "https://github.com/apple/swift-syntax", exact: "509.0.0")
],
targets: [
.target(
name: "MEGADesignToken",
plugins: ["TokenCodegen"]
),
.plugin(
name: "TokenCodegen",
capability: .buildTool(),
dependencies: ["TokenCodegenGenerator"]
),
.executableTarget(
name: "TokenCodegenGenerator",
dependencies: [
.product(name: "SwiftSyntax", package: "swift-syntax"),
.product(name: "SwiftSyntaxBuilder", package: "swift-syntax")
],
path: "Sources/Executables/TokenCodegenGenerator"
),
.testTarget(
name: "MEGADesignTokenTests",
dependencies: [
"MEGADesignToken",
"TokenCodegenGenerator"
]
)
]
)This file represents the following: a module, called MEGADesignToken and a build tool plugin, called TokenCodegen. TokenCodegen depends on the TokenCodegenGenerator executable, which itself has a dependency on SwiftSyntax (more on that later). We also have a test target called MEGADesignTokenTests.
Also, let’s take a look at TokenCodegen source file - TokenCodegenPlugin.swift - as well:
import Foundation
import PackagePlugin
@main
struct TokenCodegenPlugin: BuildToolPlugin {
func createBuildCommands(context: PluginContext, target: Target) throws -> [Command] {
guard let target = target as? SourceModuleTarget else { return [] }
let inputFiles = target.sourceFiles(withSuffix: ".json").filter { file in
file.path.string.contains("/Resources")
}
let inputPaths = inputFiles.map(\.path)
let executablePath = try context.tool(named: "TokenCodegenGenerator").path
let output = context.pluginWorkDirectory.appending(["MEGADesignTokenColors.swift"])
return [
.buildCommand(
displayName: "Generating Color design tokens",
executable: executablePath,
arguments: [inputPaths, output],
inputFiles: inputPaths,
outputFiles: [output]
)
]
}
}As you can see, TokenCodegenPlugin pretty much serves as a proxy for our executable (that's how it's intended to be: SPM plugins can only depend on executables and binary targets), that will write the Swift code based on JSON files in the MEGADesignTokenColors.swift file, located at the plugin work directory, which is the DerivedData folder.
Parsing strategies
Now that we’ve nailed how our module is structured, we need to get to the next steps: parse the JSON file, create Swift intermediate structures, and, based on them, generate code through SwiftSyntax.
The first step is to parse the core.json file. Remember, that file has three top-level keys: Colors, Spacing and Radius. As with all problems in computer science, let's divide and conquer - let's separate our JSON in three jsonObjects, each one representing the jsonObject keyed by each top-level key. Let's start by Spacing and Radius, as they're the most simple case and their parsing strategy is the same.
Spacing and Radius
Let’s use Radius as an example, as Spacing should follow suit. So after the step mentioned in the previous paragraph, for Radius, we should have a jsonObject representing this JSON:
{
"--border-radius-circle": {
"$type": "number",
"$value": "0.5"
},
"--border-radius-extra-small": {
"$type": "number",
"$value": "2"
}
// etc...
}This is very straightforward to model using Swift Decodable structures. Let's define:
struct NumberInfo: Decodable {
let type: String
let value: Double
enum CodingKeys: String, CodingKey {
case type = "$type"
case value = "$value"
}
}
typealias NumberData = [String: NumberInfo]And then, it’s just a matter of:
func extractNumberInfo(from jsonObject: [String: Any]) throws -> NumberData {
let jsonData = try JSONSerialization.data(withJSONObject: jsonObject, options: [])
return try decoder.decode(NumberData.self, from: jsonData)
}Ok, that was unsurprisingly easy. Now let’s get to the real deal.
Colors
Now, for Colors, we should have a jsonObject that represents the following JSON:
{
"Black opacity": {
"090": {
"$type": "color",
"$value": "rgba(0, 0, 0, 0.9000)"
},
"080": {
"$type": "color",
"$value": "rgba(0, 0, 0, 0.8000)"
}
},
"Secondary": {
"Orange": {
"100": {
"$type": "color",
"$value": "#ffead5"
}
}
}
}You can see that this is a fairly more complicated data structure, as it can be deeply and indefinitely nested. You can think of it in a tree-like manner.
Each key is either a leaf or a node:
- A leaf is when the key is responsible for keying a nested
JSONcontaining colour information directly (i.e. a$type/$valuepair) - A node is when the key is responsible for keying a nested
JSONrepresenting a category (i.e.Secondaryis a key forOrange)
Let’s start modeling a bit:
struct ColorInfo: Decodable {
let type: String
var value: String
enum CodingKeys: String, CodingKey {
case type = "$type"
case value = "$value"
}
var rgba: RGBA? {
value.starts(with: "#") ? parseHex(value) : parseRGBA(value)
}
}
struct RGBA {
let red: CGFloat
let green: CGFloat
let blue: CGFloat
let alpha: CGFloat
}Where parseHex(_:) and parseRGBA(_:) are helpers to parse value into the RGBA structure.
Now, there is one major aspect that will come into play in designing the algorithm — our requirement that:
For colours tokens, we must only generate code for their semantic values
This requirement hints to us that we must store core colour token information in some data structure that needs to be efficient for lookups, because we’ll need to use it while building the data structure that represents the semantic colours tokens.
One data structure that fulfills our requirements is a hash map (i.e. a dictionary — which has a O(1) lookup) that will contain flattened information about the core colours jsonObject.
In other words, while parsing the jsonObject, we want to produce something like this result:
let flatMap: [String: ColorInfo] = [
"{Colors.Black opacity.090}": .init(type: "color", value: "rgba(0, 0, 0, 0.9000)"),
"{Colors.Secondary.Orange.100}": .init(type: "color", value: "#ffead5"),
// ...etc
]Based on such ideas, this is the algorithm that will accomplish this task:
func extractFlatColorData(from jsonObject: [String: Any], path: String = "") throws -> [String: ColorInfo] {
var flatMap: [String: ColorInfo] = [:]
for (key, value) in jsonObject {
let fullPath = (path.isEmpty ? key : "\(path).\(key)").lowercased()
if let innerDict = value as? [String: Any],
innerDict["$type"] as? String != nil,
innerDict["$value"] as? String != nil {
let jsonData = try JSONSerialization.data(withJSONObject: value, options: [])
let colorInfo = try decoder.decode(ColorInfo.self, from: jsonData)
flatMap[fullPath] = colorInfo
} else if let innerDict = value as? [String: Any] {
let nestedMap = try extractFlatColorData(from: innerDict, path: fullPath)
flatMap.merge(nestedMap) { _, new in new }
}
}
return flatMap
}In this algorithm, each key in the jsonObject is visited exactly once and each corresponding value is also processed once, doing O(1) operations (serialization and deserialization or merging dictionaries). If n is the total number of keys in the input jsonObject (including nested keys), we can say that this algorithm will have a complexity of O(n).
Great, now we have a lookup table to build our semantic palette. Let’s proceed!
Regarding the semantic colours JSON, we'll have a jsonData object (same as before, but now we'll receive Data instead of [String: Any]) representing the following structure:
{
"Focus": {
"--color-focus": {
"$type": "color",
"$value": "{Colors.Secondary.Indigo.700}"
}
},
"Indicator": {
"--color-indicator-magenta": {
"$type": "color",
"$value": "{Colors.Secondary.Magenta.300}"
},
"--color-indicator-yellow": {
"$type": "color",
"$value": "{Colors.Warning.400}"
}
}
}This structure is only nested one level deep, so we can model it as a Decodable one:
typealias ColorData = [String: [String: ColorInfo]]Now we’ll make use of the flatMap generated previously to build this data structure in the following algorithm:
func extractColorData(from jsonData: Data, using flatMap: [String: ColorInfo]) throws -> ColorData {
var colorData = try decoder.decode(ColorData.self, from: jsonData)
for (categoryKey, var categoryValue) in colorData {
for (semanticKey, var semanticInfo) in categoryValue {
let sanitizedValue = semanticInfo.value.sanitizeSemanticJSONKey()
// O(1) lookup
guard let coreColorInfo = flatMap[sanitizedValue] else {
let reason = "Error: couldn't lookup ColorInfo for \(semanticKey) with value \(semanticInfo.value)"
throw ExtractColorDataError.inputIsWrong(reason: reason)
}
semanticInfo.value = coreColorInfo.value
categoryValue[semanticKey] = semanticInfo
}
colorData[categoryKey] = categoryValue
}
return colorData
}Being m the number of categories (e.g. Focus, Indicatorand so on) and n the average number of semantic keys containing colour information in each category, we'll have a complexity of O(mn).
Notice how crucial it was to determine an efficient lookup data structure to contain the core colours token data. If we were to choose a data structure that had a O(n) lookup (for instance, by creating a Decodable structure to directly decode the core colours JSON) this algorithm would be way less efficient, as it'd be multiplied by a factor of linear complexity upon every lookup instead of a constant factor.
Great! Now we have all parsed all the information contained in our input JSON files 🎉 - the next and final step is to generate Swift code from it.
Code generation
Here we’ll make use of Apple’s SwiftSyntax package to generate code. But what is SwiftSyntax? Quoting its GitHub about section:
A set of Swift libraries for parsing, inspecting, generating, and transforming Swift source code
Using SwiftSyntax we're able to create code that gives us access to Swift's Abstract Syntax Tree (AST) in a type-safe and fast API (by the way - you can check it out Swift's AST in code examples at this playground here).
In MEGADesignToken we use it to create Swift code from the parsed JSON data.
As code speaks more than anything, we invite you to take a look at our codegen.swift implementation at the TokenCodegen executable.
But, specifically in this article, we’d like to answer a question that might have popped up in your mind:
Why should I even bother using this library and its complex API? Can’t I just generate
Stringsthat representSwiftcode and write them to a.swiftfile?
func generateCode(with input: CodegenInput) throws -> String {
// Returns SwiftSyntax's `SourceFileSyntax` type
let code = try generateSourceFileSyntax(from: input)
guard !code.hasWarning else {
throw CodegenError.codeHasWarnings
}
guard !code.hasError else {
throw CodegenError.codeHasErrors
}
return code.description
}Using the power of SwiftSyntax's API, we can guarantee, at runtime, through the guard statements, that the code we're generating is warning and error-free.
Conclusion
In order to use the generated code, just import MEGADesignToken package into your target, build it, and use it as:
import MEGADesignToken
let darkColorExample = MEGADesignTokenDarkColors.Background.backgroundBlur // UIColor
let lightColorExample = MEGADesignTokenLightColors.Background.backgroundBlur // UIColor
let spacingExample = MEGADesignTokenSpacing._1 // CGFloat
let radiusExample = MEGADesignTokenRadius.small // CGFloatWe hope this article helps and inspires you to build your own custom build tools. Also, we hope it serves as an example of how data structures and algorithm design can appear in your day-to-day tasks.
All of our client code at MEGA is open-sourced. You can check MEGADesignToken and other client code at https://github.com/meganz.
