Practical ProGuard rules examples

In my previous article I explained why everyone should use ProGuard for their Android apps, how to enable it and what kind of errors you might encounter when doing so. There was a lot of theory involved, as I think it’s important to understand the underlying principles in order to be prepared to deal with any potential problems.

I also talked in a separate article about the very specific problem of configuring ProGuard for an Instant App build.

In this part, I’d like to talk about the practical examples of ProGuard rules on a medium sized sample app: Plaid by Nick Butcher.

Lessons learned from Plaid

Plaid actually turned out to be a great subject for researching ProGuard problems, as it contains a mix of 3rd party libraries that use things like annotation processing and code generation, reflection, java resource loading and native code (JNI). I extracted and jotted down some practical advice that should apply to other apps in general:

Data classes

public class User {
  String name;
  int age;
  ...
}

Probably every app has some kind of data class (also known as DMOs, models, etc. depending on context and where they sit in your app’s architecture). The thing about data objects is that usually at some point they will be loaded or saved (serialized) into some other medium, such as network (an HTTP request), a database (through an ORM), a JSON file on disk or in a Firebase data store.

Many of the tools that simplify serializing and deserializing these fields rely on reflection. GSON, Retrofit, Firebase — they all inspect field names in data classes and turn them into another representation (for example: {“name”: “Sue”, “age”: 28}), either for transport or storage. The same thing happens when they read data into a Java object — they see a key-value pair “name”:”John” and try to apply it to a Java object by looking up a String name field.

Conclusion: We cannot let ProGuard rename or remove any fields on these data classes, as they have to match the serialized format. It’s a safe bet to add a @Keep annotation on the whole class or a wildcard rule on all your models:

-keep class io.plaidapp.data.api.dribbble.model.** { *; }

Warning: It’s possible to make a mistake when testing if your app is susceptible to this issue. For example, if you serialize an object to JSON and save it to disk in version N of your app without the proper keep rules, the saved data might look like this: {“a”: “Sue”, “b”: 28}. Because ProGuard renamed your fields to a and b, everything will seem to work, data will be saved and loaded correctly.

However, when you build your app again and release version N+1 of your app, ProGuard might decide to rename your fields to something different, such as c and d. As a result, data saved previously will fail to load.

You must ensure you have the proper keep rules in the first place.

Java code called from native side (JNI)

Android’s default ProGuard files (you should always include them, they have some really useful rules) already contain a rule for methods that are implemented on the native side (-keepclasseswithmembernames class * { native <methods>; }). Unfortunately there is no catch-all way to keep code invoked in the opposite direction: from JNI into Java.

With JNI it’s entirely possible to construct a JVM object or find and call a method on a JVM handle from C/C++ code and in fact, one of the libraries used in Plaid does that.

Conclusion: Because ProGuard can only inspect Java classes, it will not know about any usages that happen in native code. We must explicitly retain such usages of classes and members via a @Keep annotation or -keep rule.

-keep, includedescriptorclasses 
            class in.uncod.android.bypass.Document { *; }
-keep, includedescriptorclasses 
            class in.uncod.android.bypass.Element { *; }

Opening resources from JAR/APK

Android has its own resources and assets system that normally shouldn’t be a problem for ProGuard. However, in plain Java there is another mechanism for loading resources straight from a JAR file and some third-party libraries might be using it even when compiled in Android apps (in that case they will try to load from the APK).

The problem is that usually these classes will look for resources under their own package name (which translates to a file path in the JAR or APK). ProGuard can rename package names when obfuscating, so after compilation it might happen that the class and its resource file are no longer in the same package in the final APK.

To identify loading resources in this way, you can look for calls to Class.getResourceAsStream / getResource and ClassLoader.getResourceAsStream / getResource in your code and in any third party libraries you depend on.

Conclusion: We should keep the name of any class that loads resources from the APK using this mechanism.

In Plaid, there are actually two — one in the OkHttp library and one in Jsoup:

-keepnames class okhttp3.internal.publicsuffix.PublicSuffixDatabase
-keepnames class org.jsoup.nodes.Entities

How to come up with rules for third party libraries

In an ideal world, every dependency you use would supply their required ProGuard rules in the AAR. Sometimes they forget to do this or only publish JARs, which don’t have a standardized way to supply ProGuard rules.

In that case, before you start debugging your app and coming up with rules, remember to check the documentation. Some library authors supply recommended ProGuard rules (such as Retrofit used in Plaid) which can save you a lot of time and frustration. Unfortunately, many libraries don’t (such as is the case with Jsoup and Bypass mentioned in this article). Also be aware that in some cases the config supplied with the library will only work with optimizations disabled, so if you are turning them on you might be in uncharted territory.

So how to come up with rules when the library doesn’t supply them?
I can only give you some pointers:

1. Read the build output and logcat!
2. Build warnings will tell you which -dontwarn rules to add
3. ClassNotFoundException, MethodNotFoundException and FieldNotFoundException will tell you which -keep rules to add

You should be glad when your app crashes with ProGuard enabled — you’ll have somewhere to start your investigation :)

The worst class of problems to debug are when you app works, but for example doesn’t show a screen or doesn’t load data from the network.

That’s where you need to consider some of the scenarios I described in this article and get your hands dirty, even diving into the third party code and understanding why it might fail, such as when it uses reflection, introspection or JNI.

Debugging and stack traces

ProGuard will by default remove many code attributes and hidden metadata that are not required for program execution . Some of those are actually useful to the developer — for example, you might want to retain source file names and line numbers for stack traces to make debugging easier:

-keepattributes SourceFile, LineNumberTable

You should also remember to save the ProGuard mappings files produced when you build a release version and upload them to Play to get de-obfuscated stack traces from any crashes experienced by your users.

If you are going to attach a debugger to step through method code in a ProGuarded build of your app, you should also keep the following attributes to retain some debug information about local variables (you only need this line in your debug build type):

-keepattributes LocalVariableTable, LocalVariableTypeTable

Minified debug build type

The default build types are configured such that debug doesn’t run ProGuard. That makes sense, because we want to iterate and compile fast when developing, but still want the release build to use ProGuard to be as small and optimized as possible.

But in order to fully test and debug any ProGuard problems, it’s good to set up a separate, minified debug build like this:

buildTypes {
  debugMini {
    initWith debug
    minifyEnabled true
    shrinkResources true
    proguardFiles getDefaultProguardFile('proguard-android.txt'), 
                  'proguard-rules.pro'
    matchingFallbacks = ['debug']
  }
}

With this build type, you’ll be able to connect the debugger, run UI tests (also on a CI server) or monkey test your app for possible problems on a build that’s as close to your release build as possible.

Conclusion: When you use ProGuard you should always QA your release builds thoroughly, either by having end-to-end tests or manually going through all screens in your app to see if anything is missing or crashing.

Runtime annotations, type introspection

ProGuard will by default remove all annotations and even some surplus type information from your code. For some libraries that’s not a problem — those that process annotations and generate code at compile time (such as Dagger 2 or Glide and many more) might not need these annotations later on when the program runs.

There is another class of tools that actually inspect annotations or look at type information of parameters and exceptions at runtime. Retrofit for example does this by intercepting your method calls by using a Proxy object, then looking at annotations and type information to decide what to put or read from the HTTP request.

Conclusion: Sometimes it’s required to retain type information and annotations that are read at runtime, as opposed to compile time. You can check out the attributes list in the ProGuard manual.

-keepattributes *Annotation*, Signature, Exception

If you’re using the default Android ProGuard configuration file (getDefaultProguardFile('proguard-android.txt')), the first two options — Annotations and Signature — are specified for you. If you’re not using the default you have to make sure to add them yourself (it also doesn’t hurt to just duplicate them if you know they’re a requirement for your app).

Moving everything to the default package

The -repackageclasses option is not added by default in the ProGuard config. If you are already obfuscating your code and have fixed any problems with proper keep rules, you can add this option to further reduce DEX size. It works by moving all classes to the default (root) package, essentially freeing up the space taken up by strings like “com.example.myapp.somepackage”.

-repackageclasses

When to use @Keep and -keep

@Keep support is actually implemented as a bunch of -keep rules in the default Android ProGuard rules file, so they’re essentially equivalent. Specifying -keep rules is more flexible as it offers wildcards, you can also use different variants which do slightly different things (-keepnames, -keepclasseswithmembers and more).

Whenever a simple “keep this class” or “keep this method” rule is needed though, I actually prefer the simplicity of adding a@Keep annotation on the class or member, as it stays close to the code, almost like documentation.

If some other developer coming after me wants to refactor the code, they will know immediately that a class/member marked with @Keep requires special handling, without having to remember to consult the ProGuard configuration and risking breaking something. Also most code refactorings in the IDE should retain the @Keep annotation with the class automatically.

Plaid stats

Here are some stats from Plaid, which show how much code I managed to remove using ProGuard. On a more complex app with more dependencies and a larger DEX the savings can be even more substantial.