19 tips for Gradle in Android projects — 2019 Edition
At the Google I/O 2019 we had two great talks related with Gradle in Android projects: “What’s new in Android Studio Build System” and “Build Bigger, Better: Gradle for Large Projects”.
This article is a summary of the last one with examples explaining the different recommendations. I know the title is maybe a bit clickbait, but literally, there are 19 great tips in the presentation.
The session was presented by Xavier Ducrohet and Aurimas Liutikas:
The presentation is divided into three sections: Modularization, Configuration and Investigation. I strongly recommend you to watch it with your team. Let’s go through section by section:
Modularization
Nowadays, Modularization is everywhere in the Android World. If you still need more information, you have excellent resources here and here. Even at the I/O this year, there was a talk dedicated to good practices in Android Modularization.
Today we are working in teams by features or specific parts of the architecture, why we have the penalization of rebuilding parts of the project that we don’t touch? Besides the benefits from the architecture perspective(and of course Testing), Modularization matches very well with the Gradle Build System. With a big monolithic project, it is hard to get benefits like caching or compilation avoidance.
1- Create Pure Java/Kotlin Libraries
The first tip is to create pure Kotlin/Java modules when it’s possible. Since the adoption of concepts of Clean Architecture in the Android World, developers have more present being less dependent on the framework. For instance, in case you have your business logic in your domain layers, those modules don’t need to use Android dependencies. Usually, these modules will be pure Kotlin/Java implementations without references to the framework. In the end, talking from Architecture perspective, Android will be the implementation details of your business logic.
Regarding the Build system, having a pure Java/Kotlin has an impact on the build. Here you can see the Task Dependency Graph for a :libjava:assemble of the base java plugin:
Looks simple, and we have only 5 tasks, in case we want to show the Task Dependency Graph for a simple :lib1:assembleDebug of the com.android.library the graph is:
It’s quite clear which task will be faster. This recommendation doesn’t apply to the domain layers only. You can apply to your specific architecture when you do not depend on Android.
2- Apply only needed plugins
It’s quite normal the usage of plugins in our projects, but sometimes either because we don’t know or because we are copying and pasting, we include plugins in our configuration files that are not required.
apply plugin: 'com.android.application'
apply plugin: 'kotlin-android'
apply plugin: 'kotlin-android-extensions'
apply plugin: 'maven-publish'
apply plugin: 'kotlin-kapt'
apply plugin: 'talaiot'
apply plugin: 'com.github.triplet.play'In the configuration phase of Gradle, tasks are configured for each plugin applied to each module, adding an extra overhead of time and resources in our builds.
It’s important to double check if we need the plugins included in our configuration files and scope the use of the plugins in independent modules.
3- Scope annotation Processors
Annotation processing is still being one expensive task in our builds. Since Gradle 4.7, we can use incremental processing. Some of the most popular annotation processors libraries have been updated to be aligned with the incremental processing guidelines.
The recommendation in the presentation is to isolate the usage of the annotation processors in an independent module.
Just as a reminder, since Kotlin 1.3.30, KAPT now supports incremental annotation processors in an experimental mode, you can update the value in gradle.properties:
kapt.incremental.apt=true
4- Run Lint on final App module
The lint tool checks for structural code problems that could affect the quality and performance of your Android application. It is strongly recommended that you correct any errors that lint detects before publishing your application.
In a modularized project, we can waste time running lint in all modules. The advice here is to avoid repeated work by running lint on final app module. Lint will run over the final module and binary dependencies.
5- Use api and implementation configurations
Gradle 3.4 introduced the new configurations implementation/api. Implementation/api includes the term “Configuration avoidance”, that helps to prevent leaking transitive dependencies for multi-module projects.
Dependencies appearing in the api configurations will be transitively exposed to consumers of the library. Dependencies found in the implementation configuration will not be exposed to consumers.
In What’s New in the Android Studio Build System, they explained in detail the impact of using api/implementation in a use case of 4 modules:
Use api/implementation in your dependencies modules, anyways Gradle is showing a warning in case you are using compile
6- Extra configurations for versioning dependencies
In a world of Modularization, using the same libraries for different modules is a typical case. For example, in your feature-ui modules, you are using the same recyclerview dependency androidx.recyclerview:recyclerview:1.0.0. If you have 20 modules and you want to update the version of one dependency you can notice is not very optimal process to update every module with the new version.
You can centralize this versioning with extensions for the dependencies and versions in a configuration file visible by all modules:
ext.versions = [
'compileSdk' : 28,
'minSdk' : 23,
'targetSdk' : 28,
'appcompat' : '1.1.0-alpha04',
'androidx' : '1.0.0',
'androidxCollection' : '1.0.0'
]This tip explained in the presentation is vastly used in the Android World since 4–5 years ago. I’m sure that all of you have worked on some project using this technique.
7 — BuildSrc for versioning dependencies
A better approach of versioning dependencies is using buildSrc. By convention, Gradle automatically compiles and tests buildSrc code and puts it in the classpath of your build script.
You only have to create a module called buildSrc in your root folder. Of course, you can use Kotlin
For our example, we will create our Dependencies (com.dependencies.Dependencies.kt) class and will define the dependencies and versions for the different modules:
Once it’s compiled, the class is included in the classpath of our project, and we can use it in our build.grade configuration files(yes, with autocomplete):
implementation(Dependencies.supportAppcompat)
implementation(Dependencies.supportRecycler)
implementation(Dependencies.supportCardView)
implementation(Dependencies.workManager)8- Implementing custom Plugins
After viewing the benefits of including in the classpath the dependencies, we can go further.
One of the coolest ideas cited in the presentation is to define and customize plugins in the buildSrc to encapsulate logic and reduce the configuration files size.
Here, I want to mention Anton Malinskiy, last year we were pioneers in Agoda implementing this approach, reducing the complexity of the build configurations with different Plugins.
Let’s see an example, we want to create a Plugin that encapsulates the logic of Android library plugins in our modules( com.android.library ). The first thing is to create the plugin:
The plugin will apply the Android Library plugin, and then we are applying a configuration when we retrieve the Library Extension, configure is an extension function of Library Extension:
Finally, we need to register the plugin in our build.gradle.kts of the buildSrc folder:
This gives us the possibility to define the configuration files for our android libraries modules like:
Beautiful right? This is only a basic example, but you can implement more advanced plugins with implementations like abstracting multi-variant configurations or reducing configurations related to verification tasks.
Confs done Right
The second part of the presentation is related to the configuration of our builds in the project. The general advice here is “do as little as possible”, keep simple the configuration files. This makes absolute sense, but when we are trying to orchestrate deployments, aggregations, reporting and build configurations is easy to end messing up without noticing that we can improve the overall process.
9-Build relevant variants
The first advice in this section is “build only relevant variants”. Build Variants are a combination of buildTypes + Flavors. In projects defining multiple build types and flavors, we are generating more build variants, increasing the cost of our build.
The advice is to separate the responsibilities of different buildTypes. A Debug build should be light, usually is going to be used by the developers. Also, for CI executions, only build the relevant variants(avoid using general assemble).
10-Use Lazy tasks configurations
Gradle provides Lazy configuration, which delays the calculation of a property’s value until it’s required. Among the benefits of using Lazy properties, we find: avoiding resource working during the configuration phase, automatically determining tasks dependencies based on their connections and wiring together Gradle models without worrying when a particular property’s values will be known.
On the presentation What’s new on Android Build Tools Studio they showed the impact of using Lazy configuration in 100 modules during the configuration phase:
Gradle represents lazy properties with two interfaces:
- Provider: represents a value that can only be queried and cannot be changed.
- Property: represents a value that can be queried and also changed.
Let’s see an example of how to apply Properties in one Task:
The Task AndroidMessage is using a Property<String> to represent a value that later we will print in the console. The Provider<String> represents the calculated, read-only, message. The Lazy property will be passed and only will be queried when is required, usually in the execution phase. Additionally, we are adding one Extension to set up the value.
Later we will register our task:
getting the following output:
> Task :what
Android Rules!!!BUILD SUCCESSFUL
The value is only retrieved when it’s queried in the execution of the task and not in the configuration phase.
The Lazy API offers more types to suit the common use cases in the definition of Tasks like DirectoryProperty or RegularFileProperty and works with Collections and Maps too.
Finally, maybe you have noticed that we are using the open identifier in the task. However, one thing that called my attention in the presentation was that you can define the tasks and extensions as abstract classes. This will help us to simplifies the declaration of our tasks, and we will let Gradle handle the initialization and decoration of our tasks:
Of course, the recommendation is to use Lazy configuration in our tasks and plugins.
11-Task wiring
When we are creating different tasks in the project, it‘s possible that we create some dependencies between them. The first way that may come to our mind is by using Task.dependOn. This can lead to undesired dependencies.
In the previous point, we have seen one of the benefits of using Lazy Properties is that it determines the tasks dependencies based on their connection. Let see how it works, the next example is based on the one mentioned in the presentation:
We have two tasks, and we want to depend on one of the tasks. Notice that on both, we are using the abstract way mentioned before and the usage of Property types(DirectoryProperty). Now we have only to register the tasks:
The flatmap method returns a new Provider from the value of this provider transformed using the given function. It propagates the value and dependency on a lazy way, detaching the explicit relation on the TaskConsumer. Once we set the input of the TaskConsumer with the artifact Provider we create the dependency between both tasks.
The Task Dependency graph for TashConsumer:
In summary, don’t use dependsOn and keep in mind Lazy configurations, all are advantages.
12- Use Gradle Worker API
The Gradle Worker API provides the ability to break up the execution of a task action into units of work and then to execute that work concurrently and asynchronously. The Android team has worked very closely with Gradle to bring this feature and improve the multithreading in the AGP.
In What’s New in the Android Studio Build System, it was one of the main topics explained:
The team has converted more than 90% of the AGP tasks to use Workers. It’s enabled by default in 3.5( android.enableWorkers).
If you are developing plugins or using custom tasks, the recommendation is using Workers with them.
Let’s follow a simple example of the Official documentation, we need to create a custom task that generates MD5 hashes of a configurable set of files. The Runnable is:
We are adding a sleep of three seconds to show more clearly the benefit of the feature. The task is:
The action of the task is iterating a source folder, and generating the MD5 hash for each file. The action is scheduled in the WorkerExecutor defined in the Task. Finally, we need to register our Task:
You can execute the task with ./gradlew md5 and try to create the same task without Workers to understand the benefits of using Workers.
Finally, mention that using the Property lazy configurations with the abstract mode will help you to abstract the complexity of initializing and scheduling tasks with WorkerExecutor, again one more benefit.
13- Custom Types For Gradle tasks
This advice is related to declare tasks with custom type. Of course, we can create tasks in a simple way like:
task("customTask"){
doWhatever()
}However, the problem here is during the configuration phase of the task doWhateverwill be executed. If you are thinking about using doLast is not a solution either.
Use custom types, define inputs/outputs and use Workers:
abstract class TaskProducer : DefaultTask() {
@get:OutputDirectory
abstract val output : DirectoryProperty
}14- Don’t apply computation on Configuration
Finally, in this section, the recommendation is to don’t apply computation on configuration phases, this looks widespread but still is being one of the main sources of slowness in the projects. Avoid using expensive computation operations like fetching git information in the configuration sections or using the same expensive operations to define BuildConfig values.
Guide To Investigations
After reviewing tips of Modularization and configuration now is time to check the investigation tips. Our decisions relating to the build system should be driven by measurements, regressions and experimentation.
15- Build Scans
Build-Scans is an awesome tool provided by Gradle Inc. It gives you valuable information about the build, you need to apply the plugin and when the build is done it will upload the information to the Gradle server. If you are enjoying the Gradle Enterprise service, you can aggregate the information.
Build Scan is a perfect tool to find some valuable data like finding the tasks that are not lazy or even check the time of the CG.
15.b- Talaiot
Another tool to help with the measurements is Talaiot:
Talaiot is perfect for applying regressions and experimentations, the results can be published to your time-series systems(some of the graphs in this article were generated by the plugin).
15.c- Gradle Profiler
Another tool coming from Gradle Inc is Gradle Profiler. It’s a tool to automate the gathering of profiling and benchmarking information for Gradle builds. A nice concept is Scenarios which you can define more complex scenarios to benchmark or profile. The scenario file is described in Typesafe format:
assemble {
tasks = ["clean"]
}
clean_build {
versions = ["5.1","4.10.2"]
tasks = ["assembleDebug"]
gradle-args = ["--parallel"]
cleanup-tasks = ["clean"]
run-using = cli
warm-ups = 20
}16- Gradle memory Limits
In our projects, we are continuously adding more features, dependencies, modules and plugins. Wrongly we can think that increasing values of the JVM like max heap memory(-Xmx) will result in faster builds. This is not strictly true. At the session, it was mentioned how adding more memory doesn’t mean faster builds:
When you’re analyzing the build times is important to keep in mind the time spent in GC. High GC pressure is a problem, following this great webcast by Gradle team, the recommendation is to not spend more than 1–2% on GC time in your build.
Next example shows that increasing memory for the build it doesn’t mean improvement in the build time:
17- Worker Gradle limits
Like the previous point, using more Workers is not synonymous of faster builds, by default Gradle will use the available CPU’s available for parallelizing your build increasing the usage of memory. Running out memory will force GC.
Measure and experiment the values that work better in your project. To update the number of workers available for your builds, you will usegradle.properties with:
org.gradle.workers.max
In our experiment we are applying for same task different number of workers:
In this case, using 2 workers is translated to better times in the build. In the next experiment, we want to measure the usage of 2 workers against 4 workers for the same generic task:
18- Different Values for CI and local machines
During our investigations, we have to experiment with different values depending on the environment. We shouldn’t apply the same configuration for CI environments and development in local machines. That’s mean that our regressions and experiment environments should apply for different contexts reporting and taking decisions independently.
19- Tracer Agent
And last but not least, in both presentations, they announced Tracer Agent:
Tracer allows displaying in a very visual way all the tasks that are being scheduled per Thread in the build:
This tool is not only available for tasks and is perfect to profile plugins too.
You can use Tracer attaching it to the JVM, or you can integrate it into your project using events with the API. It will generate a JSON file that you can export to the Chrome Trace Tool.
Final words
Thanks for reading until the end, as I mentioned at the beginning, I strongly recommend to watch the presentation with your team and start optimizing your builds.
Thanks to the teams of Android Tools and Gradle to help us with the new features and resources like these presentations.
You have all the different examples explained in this repository:
