Global renaming in an Android project

Hey! Let’s avoid dragging the feet and get started as quickly as possible.

But first, let me introduce myself. My name is Evgeny — I’m an Android developer at hh.ru’s features team. I write all kinds of interesting features that you use daily (if you use hh’s applications, of course).

In this article, I will talk about how our Android application went through a global renaming of features and packages and their structuring. What worked and what didn’t. And whether you should do the same is up to you.

For those who aren’t the fans of reading.
I have a solution for you! You can watch this article as a video on youtube in one of the episodes of “hhella cool stories” ;)

Let’s imagine we’re in 2018. Imagine? Common, stop crying. There were tons of problems in 2018, too.

We have an awesome project, in which, according to the laws of the genre, a huge monolith with 1 app gradle-module and several flavors. The main ones are applicant and employer apps, respectively.

Project structure

There’s nothing wrong with that, just so long

• until the team gets too big and it’s hard to make up the features
• until old developers quitted and newcomers don’t know everything
• until hype-train with multi-modularizing joined the team
• until the cows come out of the woodwork

Underline a correct point.

In our case, there were all sorts of bugs that were hard to fix, and the old developers quit, while the new ones found it hard to find their bearings in what was left. Because of this, it was difficult to decompose tasks and development became “expensive”.

Write in the comments what problems you faced in the projects :)

Project structure before renaming

After a while, when a lot of things had been refactored and debugged, we came up with a new kind of project structure. This is exactly the structure that we had until recently.

Throughout all this time the following has happened in terms of structure:

• We got rid of flavors and divided the apps into two app gradle-modules — modules that lean on the “com.android.application” plugin.
• Split the monolith into separate library gradle-modules (and further, there’re named just gradle modules) — modules that lean on the “com.android.library” plugin.

There were also several stages in the structure’s composition. And before the last one, which I will talk about at the end, the structure was like this:

• Some of the features were in the feature folder at the root of the project. The feature gradle-modules of the applicants’ app with the prefix “feature-” in the naming were stored there.
  For example, the resume feature was named feature-resume. This is a rudimentary solution that appeared almost at the beginning of refactoring.

Feature folder with app features

• Some of the features consisted of several sub-features, like in point 1, but we created not a gradle-module, but a folder with the prefix, too. We put the necessary features there. Too complicated? Here is an example. A resume feature, feature-resume, with gradle-modules inside it: feature-resume-profile, feature-resume-network, etc.

Typical feature structure

• The general gradle-modules that were used in both applications were in the shared folder. In addition, the features in the shared folder were divided into core and feature. Correspondingly, in which folder the gradle-module was located, the name of the folder was prefixed to the name of the module. As you can see in the image, something was wrong with the core. As the rules require that the prefix for the name of features should be “shared-core-”.

Shared (between apps) modules structure

• The main app logic lay in the folders corresponding to the names of the applicant and employer apps.
  The structure was similar to the shared folder, and the features were named as follows:

Applicant app modules structure

Employer app modules structure

Naming problems

• The location of the modules did not match their names, which made it difficult to connect the modules, and sometimes confusing, especially for new developers
• The mess in packages names. Of course, there were some rules, but over the past 3 years, they have also changed, and the packages could be named differently, sometimes the names were even the same for some of the different features

The solution was renaming!

Retrospective renaming

And here’s the most interesting thing. I will tell you about the new structure of the project.
Let’s start with a little retrospective and look at the problems we encountered.
Before we started refactoring, we decided on what new structure we wanted to see.
During the holidays in February, we started migrating files to new folders and changing package names via various scripts.
The time was specially picked so that nobody would do anything else at the same time. The work took a long time. Back then we had little understanding of the scale of the task assigned to us.

So we decided to start by manually making a giant checklist of what needs to be done. It was in the format “as the module was named -> as it will be named”. And also we renamed packages according to the name of the modules.

And even after that, we didn’t fully realize that the hole we crawled into wasn’t a rabbit hole.
Therefore, we decided to manually transfer the first modules in Android Studio. After moving 10 modules, it became clear that the git history of these files disappeared. We definitely didn’t want to lose the history, so we recalled such a command as git mv, which allows you to transfer files from one folder to another without losing the history.

When we tried to transfer some modules using git mv, we got really sad. Because it was very long and bothersome to write these commands manually.

It was necessary to look for some kind of automatic solution. So we wrote a simple bash script that generated a ton of git mv commands for two specified folders that we could copy and run immediately.

Things got a little more fun. But migrating folders is only one chapter of the story. The second one was that, in addition to just moving folders, we also wanted to REMOVE some modules, as I wrote above, by adding structures not only to the folder hierarchy, but to the package hierarchy as well.

To make it look like this:

ru.hh.feature_chat_list -> ru.hh.applicant.feature.chat.list
ru.hh.feature_chat_network -> ru.hh.applicant.feature.chat.core.network
…

Therefore, in addition to generating git mv commands, we needed to generate commands to rename some packages. To do this, we also wrote an additional script, which generated commands to call the rename script.

The script of running the rename script

The package renaming script

As a result, we spent the whole weekend generating commands for each module one by one and checking if the application was built and running.
Three days and two nights passed…

The people responsible for it headed off to rest, and the rest of the developers continued the work, but it didn’t work out for us.
It didn’t work out for these reasons:

• there were many changes in the big developer’s feature branches and we had multiple such branches. If everyone merged on their own, it is more likely that someone could have made a mistake at some point
• also, some branches had overlapped changes and it was difficult to merge them all together with the renaming

After thinking about it, we decided that it would be nice if someone would merge renaming everywhere!

So we decided to merge feature branches in the develop branch. Those who do not want to merge them now, and want to work more, will then resolve conflicts with the new develop branch on their own.

And we were not fully aware of the consequences…

After we had merged everything we could into the develop branch, we announced a code-freeze on Friday, and the elected developer started merging the renaming into the develop branch.
Over the week of teamwork a bunch of changes had accumulated, and a lot of conflicts came up during the merge.

For example

After merging the branch with the renaming to the old (a week old) develop branch, the old features changed name and package, in fact, this meant that they were moved to a different folder.
Note the files are in a new location. Everything is fine.

Old feature location -> New one

But when merging the renaming into a new develop branch after a one-week effort, such phantom structures appeared:

Feature phantom “old” structures

In the old package routes there were files that had been changed by the developers, but they had been changed in the old structure, and there were a lot of such locations… Our minds were boggled.

Naturally, it would take a long time to manually fix all this… To speed up the process, we chose the rsync utility as an auxiliary tool, because it can recursively merge folders with each other and can determine the conflict resolution strategy ( rewrite, leave new, leave old, etc.).
In the console, it was used to recursively move the feature folders. From the folder with the old name to the folder with the new one.

Then, the problems with the imports were fixed with the magic Android Studio settings — Optimize imports on the fly and Add unambiguous imports on the fly. Yes, we manually opened each file.

Android Studio settings

Ideally, we should have gone from the root modules (shared/core) first and made a regular sync of the project in the IDE. This would have led to much less headaches with imports when transferring files, because Android Studio would have automatically renamed them in all places it could reach.

But this idea came to our bright minds after all this work and experience.

After a couple of days of struggling merge, the develop branch was updated and contained the new folder structure and new package names.

And the guys who didn’t merge their branches were instructed on how to get into the new develop branch painlessly.

But it wasn’t as easy as we would have liked it to be.

Here is a list of the main points, in case you need them:

1. To merge the develop branch into your branch and save a log of conflicts in memory to understand the field of work
2. To resolve conflicts such as modified — modified by yourself, manually, but such conflicts were minimal.
3. The rest of the conflicts must be resolved for your benefit.
4. You need to move all the folders you have created to the new structure.
5. It is better to do each transfer as a separate commit, to prevent any losses and to better control the process
6. Resolve all incorrect imports with Android Studio
7. Delete all unused folders

The theory sounds easy but in reality:

• if you move a module/modules within your branch, the refactoring will be applied to the module that was in the old location. I had to repeat everything I had already done, but for the migrated module (it was a double job) + delete the old module, and do it carefully, so that the history of the git remained
• if you migrate files within your branch, refactoring will be applied to the old files, of course appeared irrelevant phantom structure of files that have already been migrated, so we had to carefully merge them with the new ones

The total number of files with conflicts was ~500 in ~50 modules.

Renaming summary

• The names of all modules correspond to the folder structure.
  And now you can enable gradle-module in settings.gradle via include(‘:shared:feature:location’), because the route matches the naming.
  For example, it used to be done like this:

• All of the modules from the feature folder in the project root (as I wrote about in the beginning), have been moved to their correct places in the folder applicant/feature.

• The package names have been correlated with the feature location.
  For example, the :shared:feature:location geolocation feature got the package ru.hh.shared.feature.location.
• Gradle-modules got rid of the prefixes feature, shared, core, etc.
  But it was decided to leave the prefixes for sub features… And then we decided not to write them either :)
• It is now possible to statically validate module connections.
  In the future we will check the module and package naming, as well as the correctness of links between different types of modules.

Our recommendations

Before you go into this story, you need to write scripts that automate most of the work. You can use our know-how, but first you need to check whether they are valid for your project.

And the main advice — do not do it manually, do it directly through the scripts. This will save a lot of nerves and time.

You should also make a checklist for migrating modules/files. And after each migration stage you have to check the checklist to see if the project is being built. Yes, it takes a long time and slows down the process, but greatly simplifies life in the future. At least there will be an understanding that “because of this my project crashed”.

For such global changes it is necessary to notify the entire team and agree in advance on how the development will go during this time to minimize conflicts. The most radical tool for this is feature-freeze/code-freeze etc. If you’re doing this kind of refactoring, keep in touch with the team regularly, report problems and potentially difficult places for merge right away.

And don’t underestimate this task if you have a large codebase. It’s definitely going to take longer than you think.

And that’s it, if you still have any questions or can share your own experience, feel free to write in the comments.

Thank you for your attention ;)