Automating Mobile Delivery at Scale
Best Practices for Fastlane
My journey to using Fastlane at scale began with building a fully automated continuous delivery system for an iOS application developed by a single team. This blossomed into two mobile applications; one Android app and one iOS app that were developed by multiple teams. Naturally, those teams employed different toolsets, various testing environments, and disparate application configurations.
With that feather in my cap, I was asked to do the same across many apps on both platforms with many development teams and many environments and many external resources. Did I mention “many”?
Automating the work of several hundred developers on dozens of teams contributing to dozens of mobile applications is difficult. Teams have different development approaches, architectures, tools, release cadences and gating requirements. Fulfilling these eclectic delivery goals requires removing manual processes, increasing automation reliability, and empowering teams with the flexibility to define and support their varied processes.
First, let’s lay the Fastlane foundation, then dig into implementation details that addressed many of these varied goals.
You can find the complete set of code samples on GitHub.
Visit 4 Lessons From Scaling iOS CI/CD for other aspects of this journey, including building a scaled iOS execution environment and three challenges we faced along the way (environmental inconsistency, limited capacity, and inflexible CI/CD).
Fastlane 101 — The Basics
Fastlane is a very active open source project implemented in Ruby, currently sponsored by Google. The project maintainers are quick to respond to issues opened by the community and warmly welcome outside contributions. Fastlane serves as a unifying framework for defining automated processes for both iOS and Android applications. It serves even better at moving process definitions into version control alongside the projects utilizing those processes.
“Fastlane is the easiest way to automate beta deployments and releases for your iOS and Android apps. 🚀 It handles all tedious tasks, like generating screenshots, dealing with code signing, and releasing your application.” — Fastlane documentation
Fastlane Terminology
- project : A codebase that utilizes Fastlane.
- Fastfile : The file containing the project’s codified processes using the Fastlane Domain Specific Language (DSL).
- action : A single configurable operation used within a Fastfile (documentation).
- lane : A collection of actions that constitute a process. Lanes can call other lanes.
- private_lane : A lane that can only be called by other lanes, not directly at the command line (documentation).
- function : A Ruby function that encapsulates logic and can be reused within a single Fastfile without changing lanes.
- plugin : A Ruby gem encapsulating additional actions not provided out-of-the-box.
Automation Reliability
Most processes evolve along a very similar trajectory. In the beginning, individual developers have the responsibility of building and releasing mobile applications. A developer accesses Apple and Google web consoles, updates configuration, manages code-signing assets, and manages the release artifacts and processes. These manual processes may operate on uncommitted code, follow different steps by different people, require special changes within an IDE, and require unnecessary elevated access within each web console.
The first goal of stabilizing and scaling should be removing all manual web-based operations and centralizing the management of the shared resources (Challenge #1 — Environmental Inconsistency). Fastlane provides these features out-of-the-box, so it is a natural fit. Each application simply needs a few lines in their Fastfile to unify the steps app developers use to configure, build, publish and manage binaries. The process is easily moved to the automated command-executor, removing the developer — and their local working copy — from the build and release process altogether.
Consider childhood building bricks; there are numerous ways to use the same blocks to construct a valid house. The colors of the blocks are purely aesthetic to the builder. Automated processes can be thought of similarly; the overall process (the house) needs to know very little about specific settings (colors) of various actions (bricks), rather it should focus on how those actions are organized to reach the desired results. The detailed configuration of the actions may change from execution to execution, but the flow of the process remains the same.
For Example:
Here is a common simplified automated process executed during development and again during a release. The details of the actions change significantly between the two executions of the identical process.
During development:
- Build: using debug build settings
- Binary: signed with a development certificate
- Deploy: a location for internal distribution to testers
- Report: send a message to a development chat channel
During release:
- Build: using the release build settings
- Binary: release-ready, signed with a production certificate
- Deploy: a production-like distribution channel
- Report: send a message to a release chat channel
Configuration Approaches (tl;dr: use dotenv and lane_context.)
In order to retain clarity at scale, it is imperative to keep the process and the configuration of that process decoupled. This also hardens the process by testing the process itself as frequently as possible according to the tenants of continuous integration. Fastlane provides several action configuration approaches; Direct action configuration, special configuration files, and ‘dotenv’. I’ll cover each below. tl;dr: use dotenv and lane_context.
Direct Action Configuration
Setting configuration values directly on the parameters to the action call within the Fastfile is effectively hard coding the values to be one value and one value only. The Ruby language can be used directly within the Fastfile to provide variation. However, this quickly becomes complex code rather than simple configuration.
See Code Sample on GitHub.
Special Configuration Files: Appfile, Deliverfile, Scanfile
To allow for a bit more flexibility, Fastlane provides a series of configuration “files” that provide some configuration to some settings of the main built-in actions. These files are fine when there is no complexity in your process. However, as the process grows, these files can quickly outlive their usefulness.
These files do not support all settings and are not able to configure a great number of actions and plugins. Ultimately, multiple configuration approaches would be required, leading to reduced clarity in the state of the configuration.
See Code Sample on GitHub
dotenv
For maximum flexibility, use the environment variable-based configuration and corresponding dotenv files to achieve the configuration variation. You can also set the values in the shell before calling Fastlane. However, the dotenv files allow the configuration to be versioned properly. To use environment-based configuration files, execute Fastlane with an additional argument: fastlane my_lane --env release. Even without the additional argument, you can use the default .env file to store your configuration values.
It is not necessary to refer to the variables configured via environment variables within the Fastfile, actions will pick them up automatically! Also, make note that any value configured directly on the action will supercede all of the nifty dotenv values.
Complex data-types, like arrays or maps, can be tricky using environment variables. Avoid developing actions that require complex inputs or implement a basic parser to keep configuration manageable. There are only a few instances of this in the out-of-the-box actions and existing plugins, but it is worth calling out.
See Code Sample on GitHub.
dotenv Hierarchy of Configuration Values
The first thing to be aware of is that the default .env file is always loaded, then additional files are loaded according to what is specified on the command-line.
When not passing any specific environment(s) files to load on the command-line (e.g. fastlane my_lane), the system environment variables of the same name take precedence over the values set in the default .env file according to dotenv rules about variable load order. This is convenient when you want the execution environment to control or change certain configurations.
Fastlane changes the rules slightly by using Dotenv.overload when passing specific environment(s) on the command-line. As before, the system environment variables of the same name take precedence over the values in the default .env file. Then the values in the last loaded .env file take precedence over all others.
Despite the nuances of these rules, they are finite. Other configuration models can have programmatic machinations of all sorts. Once the appropriately extensible configuration model is in place teams can begin to build and control their own processes.
See Code Sample on GitHub.
Embracing Innovation with Fastlane Plugins
Large organizations with large numbers of development teams and projects usually require very similar features (e.g. security scanning, code quality scanning, artifact delivery etc) for their individual automated processes. Furthermore, allowing teams to take the reins in building their processes can lead to divergent yet similar Fastlane implementations across an organization.
Teams create new projects with copies from other projects and add features using copy and paste. These common development practices lead to a variety of difficulties such as keeping projects up to date with best practices, remediating critical failures, and maintaining consistent support, just to name a few. Feature adoption is also impeded by an inability to discover available features developed by disparate teams.
Fortunately, Fastlane has it covered! Fastlane added a robust plugin architecture in the second half of 2016 and the community responded by adding 350 plugins to the 214 out-of-the-box fastlane actions (circa 2/1/2019). Developing further capabilities using Fastlane plugins helps alleviate the growing pains. If new features are very business specific, and/or a feature hasn’t already been created in the open-source Fastlane plugin library; the feature needs to be developed. Taking an evolutionary development lifecycle for Fastlane plugins led to the best results.
Fastlane Plugin Development Lifecycle
First, the developer in need of the functionality adds the feature directly in the Fastfile for the fastest feedback cycles. This is fairly limiting as features become complex and rely on other libraries or more detailed configuration.
As feature development continues, the code moves to an actions and helpers folder within the project’s fastlane folder. Here the code for the action is converted to actual action classes and the inputs and outputs are laid out and solidified. During this phase, code continues to be delivered alongside the application and called from the Fastfile as a formal action. Sharing this feature beyond the original codebase is effectively impossible. However, the code is now properly formatted to become a plugin and have the action(s) delivered via gems to be shared with other consumers.
See Code Sample on GitHub
As sharing the actions with other codebases becomes more desirable, the name and meta-data files are finalized using the Fastlane new_plugin command. Then the action and helper files move out of the project and into the boiler-plate structure. The developer follows the remaining steps in the documentation to complete the conversion into a publishable gem.
See Code Sample on GitHub
Lastly, the developer integrates the published gem into any project (including the original) just like any other Fastlane plugin.
When developing plugins that will not be open-sourced, it is quite valuable to ensure they are developed in an easily discoverable place; a single Github organization works quite nicely.
Version Drift
Packaging features into shareable plugins, Ruby gems in the Fastlane case, helps to centralize de-duplication efforts and provide a safe and purposeful update strategy. Valuable features that are maintained externally and generalized to fit the needs of many will inevitably lead to an increased number of consumers. The onus falls on the consumers to ensure they are up to date on a regular basis. This problem of version drift is not unique to Fastlane; it is a problem common to software development that produces or consumes external libraries.
Conclusion
The techniques detailed in this post have proven very successful in allowing large numbers of teams to contribute to a large number of applications, all while keeping it both flexible and easy to support. Codifying and decoupling processes from the command executor leads to experimentation with various tools without the need for significant refactoring. Finally, working towards even further centralization while still maintaining a stern focus on extensibility and ownership will create more opportunities for developers to automate their processes quickly and easily.
Appendix A: Fastlane Tips & Tricks
Logging
Be aware of your logging output. Some code and design choices can lead to complex and less readable logs. For example, lanes can call other lanes and pass variables like a method, but lane transitions include logging. You can also simply write a Ruby-esque method for the same and the logging is avoided.
Another logging pitfall is using the sh action to call shell commands. The associated logging can be a bit lengthy in some cases. There are several options for controlling this better.
See Code Sample on GitHub.
Helper.is_ci?
A large majority of application development and Fastlane process development occurs on a developer’s machine, but there are some operations and configurations that don’t make sense or present a security risk to run locally or in the CI environment. Lanes should still largely function as intended in both locations to support troubleshooting and ongoing development.
When used in plugin development or within the Fastfile, this flag is useful for blocking or allowing actions to occur only when running in a CI system. It is based on the existence of various known environment variables. See source for details. Power users can pose as CI when executing locally when absolutely necessary by setting one of the values in their environment.
Passwords and Sensitive Information
Action options can often contain passwords or API tokens. These option types use special handling to avoid logging and exposure of sensitive information. You can store safe tokens in your dotenv configuration files as detailed above. Safe tokens could include restricted/read-only authentication tokens and/or messaging integration tokens. Most CI systems provide password storage mechanisms that can properly expose the values as environment variables. There are also several standalone products that could be integrated as well.
See Code Sample on GitHub.
Using lane_context
Actions can return values in a couple of different ways. First, the call to them simply returns some value that can be assigned to a variable within the Fastfile — just like any method call. Second, they can choose to store salient information in a lane_context for this execution of Fastlane. This allows you to configure other actions using the outputs from previous actions. For example, when a build is performed the build actions store the path to the resulting binary. Often this value is the default value for other actions, like deployment. Avoid setting pre-conceived static paths to the configuration files when possible. Allow the action(s) to inform subsequent actions using the lane_context or return values.
See Code Sample on GitHub.
Filesystem Paths
Automated processes often deal with a variety of files. Fastlane execution can be tricky when dealing with paths, see https://docs.fastlane.tools/advanced/fastlane/#directory-behavior for all the details.
DISCLOSURE STATEMENT: These opinions are those of the author. Unless noted otherwise in this post, Capital One is not affiliated with, nor is it endorsed by, any of the companies mentioned. All trademarks and other intellectual property used or displayed are the ownership of their respective owners. This article is © 2019 Capital One.
