How to Beta Test your Unity Game using Firebase App Distribution
Using App Distribution to test out an internal project
Intro
In my time making games and toys, I’ve learned that the single biggest driver of success early in a game’s lifecycle is testing. Not unit testing, not necessarily testing a piece of software, but as little as setting aside an hour to play a sample round of a new idea on a piece of paper can affect massive positive change during the most formative stages of a game’s development. My favorite saying has become “a prototype is worth 1000 meetings.” Through that experience, I will drag people into a room and make them play a game where all the sprites are solid boxes and levels are wireframe if I can muster up the influence to do so.
Just after the Firebase Summit, I started working on a fun little game that I really wanted to push out to some of my coworkers. Presenting… Sword Chucks!
Immediately I ran into the typical issues: “How do I distribute the binary?” “Who even wants to play this, and who will consider this annoying spam?” “When I update this game, being a prototype, how do I notify only those interested when a new build is out?” “Do I really want to deal with setting up a build server?”
It turns out that Firebase just released App Distribution into beta. This seems like a great time to put it through its paces!
App Distribution setup
To start, I need to enable App Distribution for my game:
I’m not sure who’s interested in playing my game, so instead of inviting people directly I want to generate invite links to get my coworkers in. When you generate invite links, you can have people who follow them go directly to different groups. I decided to create three different groups. One group is just all of my Firebase Developer Relations coworkers, another are coworkers that sit near me in Boulder. I also decide to pull in the “Fun Propulsion Labs,” they’re the team maintaining the Unity plugins for Firebase:
First, I add the three groups:
This is admittedly a mildly contrived use of this feature for this article, but there are plenty of real world reasons why I would have wanted this in my past!
- If your QA team has a lot of temp contractors (or perhaps is outsourced), you can have them self-sign up to play the game as part of their on-boarding process (rather than an email you have to remember to send out) and cut off access to new builds to that whole group when they’re no longer on the project.
- You can also drop a link in your project’s Discord to get a bunch of testers from your community or auto-generate links to send out to the press ahead of your game’s release (note that on Apple platforms, you may be required to use TestFlight for this use case).
- Even internally with full time employees, different team members with different familiarity with your product may necessitate different levels interaction with development. You may want to invite executives or investors less familiar with your project or processes to play stable builds of your game, but do not want to notify them of the version you rushed out the door with a free form hand drawn loading bar (true story). Your QA team probably needs every new build as fast as possible, especially the week of a major milestone delivery.
But I’m going to use it to see who really pays attention when I send an email.
To actually upload my build, I jump back over to the Distributions tab.
On Android, I just open up the directory where I built my apk file from Unity and drag it into the window to start uploading.
On iOS, the only difference is that the Unity editor opens XCode rather than generating an ipa (a packaged iOS application). Once you’ve exported an ipa, you can drag and drop this into Firebase App Distribution. If you use Unity Cloud Build (or any other “build server”), this should be the artifact that it emits for iOS.
I now just have to type some notes, and I’m off to the races!
To invite some testers to play my game, I go over to the Invite links section:
And create links to my three competing groups:
Now I just copy my invite links, and wait for feedback.
btw, your UI is not notch friendly 🙂 the score is hidden behind the notch (can’t see in this screenshot) I demand my money back.
I guess I already have some action items for my next sprint!
Why distribute?
Other than showing off, what do I want to get out of distributing my game? The most obvious benefit of shipping small internal alphas is to really start honing in on the gameplay. To help with this, I decided to add some basic analytics so I can gather feedback as my players engage in the game.
If this is your first time exploring Firebase, check out this video to help you get started. But this seems like a good time to go over how I integrated Analytics. First, I make sure that I call CheckAndFixDependenciesAsync:
I drop this into my main menu scene. I’ll make it so that the “Start Game” button can’t be clicked until this process finishes by just hooking up a UnityEvent:
And, in case you were wondering. If OnInitializationFailed is ever triggered, I load a simple scene to inform my testers:
How you handle this is up to how you built your game. If you cannot work without Firebase, you’ll have no choice but to dead-end your players here. But if you plan on shipping your games on Android devices without Google Play Services, you can listen for a failure case and start your game in a reduced functionality mode.
Right now, I don’t have much to track in my game. I do want to get a handle on how long a play session is, how many points the player got, and how they died. I really want it to feel kind of hectic, so a really high score with a moderate duration (maybe 30 seconds to a minute) each session. I also think it would be hilarious if the most common death reason is that you hit yourself with your sword chucks, so I’m going to throw that in as well.
I’m not particularly proud of how GameOverPanel got its hands on DeathData, but let’s just say that I resorted to some pretty sketchy code to hand large bits of binary data across scene boundaries so that I can later upload it to something like a high score board. But that’s a post for a different day!
Once I ship a build off to my testers, it can take a few hours before analytics data starts populating. Therefore, I test locally by running this game in a special development mode.
On Android, this is a simple: adb shell setprop debug.firebase.analytics.app com.Google.SwordChucks
And on iOS it’s a command line argument you pass to your application:
If I click on over to debug view:
I can see events from my test device in seconds rather than hours.
I cannot stress how important it is that you test this. Events typically take up to 24 hours to start propagating. By setting up this debug device, I was able to instantly catch an issue where “death_reason” wasn’t being reported because my parameter value had a space in it!
Crashlytics
I cover Crashlytics in depth in these two posts. Since I’m gathering testing data from my peers, I definitely want to know if this game breaks on their phones!
It’s a simple matter of enabling Crashlytics:
and importing the plugin. The integration is otherwise automatic!
Automate all the Things
Eventually you’re going to want to automate this whole process. I’ll give you a brief overview of how to use the Firebase CLI to upload a build for testing, but I’ll leave it to you to figure out what works with your setup. Note that with a small adaptation, this is trivial to move from your local machine to your continuous integration server. There’s also fastlane integration already available, and some functionality to distribute via gradle if you don’t care about iOS builds.
I start by first Installing the CLI, then logging in.$ firebase login
I can do a test distribution by first, updating my build version and making a new build:
Then going over to where I built the APK:$ cd ~/Builds
Now, I need a few key pieces of information. First, my Firebase App ID. You can find this under Project Settings:
In the “Your apps” section:
Second, I need to list who to share it with. Since I’m driving everything via groups, I run over to my groups page from before and make sure I pull out the “group alias” (boulder-devrel in this screenshot):
Now I can construct my command line:
There are a ton of useful command line arguments I could use, but here are the ones I chose and why:
firebase— just the firebase cli invocation.appdistribution:distribute— as you might expect, I’m askingappdistributiontodistributea build.swordchucks.apk— the Android application I’m uploading. This can be anipaas well, but be aware that you’ll have to make sure you copy the proper App ID when switching between platforms.--app— this specifies the App ID for which I’m providing a build.--release-notes— this takes a string of release notes. It may be more convenient to provide a--release-notes-fileargument, but I plan to auto-generate this command line and it’s easier for me to generate a string like this.--groups— the list of groups I’ll share this build out to. Again, I could use the--groups-filecommand, but this is less useful for how I plan to automate this process. There are also similar--testersand--testers-filearguments, but I plan on controlling all my distribution through groups.
And when I go back to the distributions tab, I see my new Build!
Of course, all players in the groups boulder-devrel, firebase-devrel, and fun-propulsion-labs get a nice email letting them know that they have something new and shiny to play with:
If they need an older build, the download screen also lists all versions that that player has access to.
Wrap Up
There’s plenty more to cover here, and I feel like automation could itself be its own article for integrating with a continuous integration tool and even running builds through Firebase Test Labs.
Now that I’m tracking basic user metrics, a great next step game wise might be to integrate Remote Config so that I don’t have to rely on user updating their app to pull down my latest changes. Also, now that I have a Firebase project created for App Distribution, it’s worth seeing what other fun integrations I can figure out.
