Android gradle plugin tricks around application build time

Sergiy Grechukha
Jul 30, 2017 · 8 min read

Sometimes we meet huge projects with massive structure, a bunch of technologies and features. Quite always such projects tend to become monsters in many ways.

  • Massively linked code which leads to hardly maintainable process.
  • Library zooLand with lots of overlapping dependencies, conflicts and old versioned libs.
  • Almost each project uses annotation processing which appears to be a cake with surprise as soon as your project grows in something serious. Annotation processing breaks down the concept of an incremental build. So any tiny change in code calls annotation processing. In big projects this operation may be quite time consuming.
  • Tons of resources stored in one heap (and duplicates of these resources) and so on, and so on.

One can be surprised: what is so bad about this stuff? Ok, I’ve got a big project, even exceeding 65k of methods and it’s ok!? And it’s true. Here we are not going to discuss an average project (something like REST client for news feed) we’re going to speak about projects like more than million lines of code, with heaps of generated code (maybe), tons of resources, big dev teams and we will try to squeeze max performance in build process.

One of the real problems in such big projects is huge compile time, it can take over 5–7 minutes of cold start and 3–5 minutes for incremental builds and it’s tremendously insufficient.

So let’s consider an example: imagine that there is a project which combines in it feature sets for 3–5 separate dev teams (chat, news feed, hardware communication, geo features etc).

Here are some requirements we’d like to meet:

  • Teams must cause a minimal effect at each other’s workflow but still use common core functionality, styles, resources etc.
  • We’d like to have an opportunity to build separate builds for each let’s name it stream.
  • Code must meet SOLID principles and be reusable
  • Architecture must support easy testing (unit and UI)
  • Minimal possible build time without influence on runtime (here will have the biggest focus)

So let’s dive deep into a process. To make it like a huge project I’ve generated 8k classes (2k for each stream) which refer to each other by chain (so that they all will influence an incremental compile time). Something like this:

Then divide all code into flavors (to meet the requirements):

App contains few screens with routing between them, some stub feature (named SomethingUsefull) and all this is linked with DI.

Now is the show time — let’s measure some stuff.

First of all, I must say that I excluded multidex situation in the wake of the experiment (it was not much representative — just long ride). By the way there’s an awesome plugin for method counting https://github.com/KeepSafe/dexcount-gradle-plugin . It gives needed info plus visualisation like this:

> Task :app:countFullDebugDexMethods
Total methods in app-full-debug.apk: 52157 (79.59% used)
Total fields in app-full-debug.apk: 23616 (36.04% used)
Total classes in app-full-debug.apk: 12664 (19.32% used)
Methods remaining in app-full-debug.apk: 13378
Fields remaining in app-full-debug.apk: 41919
Classes remaining in app-full-debug.apk: 52871

The given image is a screenshot of an interactive web page where you can see the amount of methods for each package in your app (just hovering cursor over the segments).

Next tool is gradle-scan plugin. This one gives you a full build info (especially with enterprise license). It’s easy to use and pretty informative:

Total time distribution:

And a lot of other helpful info which may give you a hint what is wrong with your project’s build time.

And last but not least is https://github.com/gradle/gradle-profiler this one I’ve used to get real figures of build time for warmed up JVM and in bunch of builds:

Run using: ToolingApi
Cleanup Tasks: []
Tasks: [clean, assembleFullDebug]
Gradle args: []
Build changes: none
Warm-ups: 6
Builds: 10
* Running scenario using Gradle 3.5 (scenario 1/1)
* Stopping daemons
* Running warm-up build 1 with tasks [clean, assembleFullDebug]
Execution time 60924ms
* Running warm-up build 2 with tasks [clean, assembleFullDebug]
Execution time 30091ms

Also, you can combine last two tools and get realistic and pretty build report.

So, being equipped with all this stuff I started profiling. As an entry point I’ve used android gradle plugin 2.3.3 with gradle 3.5 (which is out of the box for currently stable Android Studio) for single module application with 8k stubbed classes + annotationProcessing for 2k classes (generating builders). This is the report:

build,3.5
tasks,clean assembleFullDebug
build 1,19746
build 2,25880
build 3,22192
build 4,22437
build 5,27784
build 6,28752
mean, 24465.16667
median, 24158.5

Let me explain it a bit. This report says that it warmed up JVM (I’ve deleted that part — it was of no interest) and time needed to perform sequentially two gradle tasks (clean and assemble).

The median value is the most relevant time for the real live process. So it will take about 24 seconds for clean build.

By the way, this is gradle.properties settings for now:

org.gradle.configureondemand=true
org.gradle.jvmargs=-Xmx1536m

Let’s make it faster! What can be done!? Ok, google and gradle guys say that whatever you try to build with gradle do it with the latest one version! When this article was written the newest one gradle version was gradle-4.1-milestone-1 which was used with android gradle plugin (AGP) 3.0.0-alpha6. Time for benchmarking…

build,4.1-milestone-1
tasks,clean assembleFullDebug
build 1,25061
build 2,24898
build 3,23100
build 4,23677
build 5,24808
build 6,21733
mean, 23879.5
median, 24242.5

Well… it’s even slower or I’d say the same ((

Heads up! Going on! Another thing that google and gradle guys say is that we should divide our projects into separate modules. There are several reasons for this:

  • One can reuse modules (they are already quite independent and can be moved/copied/injected into other projects)
  • Being independent and separated modules do not much influent each other especially if they do not depend on each other (that’s obvious). So the work on a big project can be split into dev streams who will work with their modules only thus, the impact onto the other streams will be minimized
  • Introduced last year instant app feature is available with the module concept only
  • And most interesting in context of this article — modularized application is built in parallel workers which in theory leads to build time cut

Initial app was broken into modules equivalent to previous flavors but with some peculiarities:

  • Each stream has its own module
  • All stream modules depend on core module with common code base (utility classes, rest client etc.). Core also can be split into modules.
  • There is a shell module which combines stream modules providing navigation (separated via flavors to exclude any of module if needed)
  • There is a main app module which downstreams project settings and flavour combinations

It looks like this

All the codebase is identical (but gradle settings). Also I’d like to say a few words about DI in this project. Trying to avoid annotation processing I’ve used Kodein which provides DI without generating code. It turned out to be really flexible and nice to use. Give it a shot.

Application module build.gradle

One may notice that the application module has only one dependency (app-shell) and creates flavor policy

And here is the app-shell build.gradle:

First of all, it repeats flavor policy (which is downstreamed from the main application module) and resolves dependencies due to flavor configurations. Pay attention to the new gradle dsl “implementation” and “api” which substitute deprecated in the new gradle 4 “compile” (find here more) and of course dependencies could be resolved in some awesome gradle method but it’s out of the scope of this article and possibly would not be so representative.

build,4.1-milestone-1
tasks,clean assembleFullDebug
build 1,23142
build 2,19861
build 3,20159
build 4,19912
build 5,18319
build 6,19007
mean, 20066.66667
median, 19886.5

And I’ve squeezed 4 seconds. Speaking about seconds is not so obvious but translating seconds into per cents it will be more than 17% — cool!

I’d like to bring your focus on build tasks timeline from gradle-scan tool

Timeline for single module application:

Timeline for multimodule application

Can you see the difference?

Tasks in multiModule application are performed in parallel workers and as a result the whole build is quicker. Nevertheless, you can find out that dex tasks still work one by one in line (multidex as well), but gradle team says they are working in this field and we can expect some significant improvements here (they want to exclude dex task in separate daemon for instance).

Go on squeezing more!

One of the main features of new gradle is caching it became really efficient. Now, gradle won’t perform a task until its input changes. Let’s see it in work. Adding

org.gradle.caching=true

in gradle.properties we enable this feature and run again profiling:

build,4.1-milestone-1
tasks,clean assembleFullDebug
build 1,10153
build 2,10176
build 3,9951
build 4,9735
build 5,9774
build 6,9022
build 7,9106
build 8,9401
build 9,9309
build 10,9330
mean,9533.777777777777
median,9401.0

WhAAAAT!? Less than 10 seconds! It’s 59% faster than the starting point!

This result is good enough but one must know:

  • Gradle new caching feature is still in prerelease state
  • Caching can bring out some weird stuff so be prepared for errors and switching on/off this feature

Here is the resultant table of build times:

And the graph:

Conclusions:

  • All of this stuff can easily be ignored on powerful PCs but some of us do prefer keeping in mind scaling possibility and brute force is not an option for us.
  • Modularizing the app is not so easy (especially for existing apps) and time-consuming process so estimate your profits
  • Be careful with unreleased new features (new AGP has bunch of bugs — aapt2 errors for instance)
  • Once you designed your multimodule architecture do not forget about tests (it’s also little tricky)
  • Switching to modules is a question of time and sooner or later you’ll switch

All the measurements were performed on the same machine:

2.7 GHz Intel Core i5
8 GB 1867 MHz DDR3
Flash Storage

Here are the links on git to overview the projects:

Here is modularised version of the application, where you can find tests concept in B stream

This is a single module app

Few links to consider:

Gradle summit 2017
Migrate to android gradle plugin 3.0

Comments and questions are quite welcome

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade