GoogleCast, GitHub, Git-LFS, and You

My main project these days bundles Google’s GoogleCast / Chromecast framework in it to allow users to stream video content from their iOS devices to Google’s little streaming sticks. Overall, I like the Chromecast devices. They seem to work a bit more reliably than Airplay in a lot of ways. What I don’t like, however, is the framework that Google provides for iOS to enable casting support in third-party apps.

We have had the framework in our git repository for the last year without much issue. Only recently when we tried to update to the latest version (2.10.4 as of this writing) did the issues really start to pop up. Seemingly overnight, the GoogleCast framework ballooned from 17MB to 439MB. That is not a typo. It really did grow 25x in size. I opened an issue and Google is quick to blame Bitcode, though I’ve never seen a framework grow 25x. So our app is now a little bigger because we have this giant framework embedded in our project. No big deal, right?

The problem is that we use GitHub for our version control system, and it has a hard limit of 100MB for a file size that can be pushed to a repository. If it’s over that 100MB size, the file is rejected and GitHub suggests adding it to GitHub’s Large File Storage system. Git-LFS is an extension to the Git protocol that replaces giant files like a 439MB iOS framework with a pointer to a location on a remote storage location that is better designed to house such large files (think Amazon S3).

Easy fix, I thought. I installed the Git-LFS extensions and tried adding the GoogleCast framework to LFS, but was met with an error:

Long story short, to get a file on GitHub LFS it has to have zero history in your repository. Otherwise, the pre-push hook won’t recognize it as an LFS file, and you’ll be in a jam. GitHub has a page that explains this and how to get around it. It still took a bit of trial and error for me to get GoogleCast to behave with Git LFS. These are the steps I went through in Terminal:

1. Analyze the Damage

Using gitk, I was able to see how far back and how many commits were around with the GoogleCast Framework as a part of it. To get gitk working on El Capitan, however, you need to update the copy of tcl through Homebrew.

brew cask install tcl
gitk --all -- External/GoogleCast.framework

In the output, you should see the commits that include the GoogleCast framework and its children inside.

2. Sweep It All Away

Here comes the painful part. You’re going to need to go through and remove every reference of the file in your repository history on every branch and tag. You’ll then need to force push the changes to your repo, which is always a little scary. There are two ways to do this: git filter-branch or a third-party tool called BFG Repo Cleaner. The BFG is supposed to be faster, but I was unable to get it to work for what I was trying to accomplish, so I went the slow and time consuming route.

You’ll need to repeat this on every branch you have open in your repository. Grab a drink:

git filter-branch --force --index-filter 'git rm -rf --cached --ignore-unmatch External/GoogleCast.framework' --prune-empty --tag-name-filter cat -- --all

This will go through and remove the GoogleCast framework files from every commit in your history. After you’ve finished cleaning up a branch you can run gitk again with this command to ensure it’s no longer in your history:

gitk -- External/GoogleCast.framework

Clean Up

After this, it’s probably worth running the Git garbage collector.

git gc --aggressive --prune=now

And finally, you’ll want to force push all your changes back to your repository.

git push origin --force --all; git push origin --force --tags

On to LFS

Now that we’ve got a cleaned up copy of the repository on GitHub, we can finally start re-adding the GoogleCast framework back to your repo and making sure it’s under LFS. Go ahead and add a fresh copy of the GoogleCast framework and xcassets files to where you were storing them previously. Make sure your app builds at this point just for safety.

First we need to install the Git LFS pre-push hook in our repository:

git lfs install

Next, we’ll go ahead and tell LFS about our oversized GoogleCast files.

git lfs track External/GoogleCast.framework/Versions/A/GoogleCast
git lfs track External/GoogleCast.framework/Versions/Current/GoogleCast
git lfs track External/GoogleCast.framework/GoogleCast

You’ll notice I am doing this three times. For some reason only known to Google, they are bundling the same 139 MB binary three times in their “framework,” so you have to make sure you hit all three instances of it.

Commit all your changes, including the new .gitattributes file additions that were made to track the new oversized GoogleCast files. You should now be able to push your changes to your master or develop branch and have the oversized files added to GitHub’s LFS system.

Finally you can go back to getting actual work done.

Originally published at carpeaqua.