Getting Started with Crashlytics and Unity, Part 2
Witness the power of this fully operational crash aggregation system
If you haven’t read it yet, check out my first post on getting started with Crashlytics. I’ve already covered how Crashlytics is great for simple crashes, but reality isn’t always as clean as those nice isolated demo applications. Crashes often result from chains of seemingly unrelated events in the embarrassingly messy codebases that result from late night crunches. What I want to highlight now is how you can augment Crashlytics data with other information and metadata you have about your game.
Custom Keys
The easiest way to get more information about a bug is to explicitly add it yourself. Games all have key points where the state shifts in a major way or particularly sketchy bits of logic that give off a bad code smell the moment you write them. For any state where you might think, “I’d love to know x if this ever breaks,” Crashlytics allows you to store information in a key value store to be uploaded with any future crashes.
Perhaps the easiest bit of data you can possibly add is the level that the player is on when an exception occurs. There are many ways your game may represent the current level, with one scene to one level being the simplest in Unity. To track this, I wrote a small script called CrashlyticsLogCurrentLevel:
The most important part here is Crashlytics.SetCustomKey(“CurrentLevel”, next.name);. This adds a custom key called “CurrentLevel” to the crash reports.
Another detail worth noting is my yield return FirebaseInit.WaitForInitialization();. If you don’t wait for FirebaseApp.CheckDependenciesAsync() to complete, occasionally some things may break. I’ve modified the sample firebase initialization script accordingly with a boolean to indicate that Firebase is Initialized as well as a CustomYieldInstruction named WaitForInitializationYieldInstruction to turn this into coroutine logic:
How you perform this bit of initialization is, like all things, specific to your game’s needs and architecture. Treat this as an illustrative example rather than an idyllic implementation.
With these scripts in hand, Crashlytics begins logging the current level whenever a crash occurs. I can click over from “Stack trace” to “Keys”, and see all the ancillary data I’ve associated here:
What you log is entirely up to your game’s needs. We’ve all had those bits of code where we think “if this takes more than 50 iterations, it’s probably going to break” or “this may crash if a player drops connection during the loading screen”. If you know a condition that has crashed in the past or some assumption is made before a bit of code runs, this is a great way to catalog it.
Debugging the Undebuggable
Sometimes an exception happens that makes you say, “huh?” Consider this crash report:
And the corresponding overly-complicated and questionably optimized scoring script:
This bit of code is small enough that you can probably spot the error after reading TallyFinalScore two or three times. You can also see that this will probably turn into a pile of poorly isolated code and a magnet for a less experienced me to throw in well-intended but not-well-thought-out bits of optimization. The good news is that the design team wants to tune the scoring during the testing phase of our game and has some well targeted analytics to track all of the variables that will go into tabulating a player’s score:
It would be awesome if we could cross correlate our Firebase Analytics and Crashlytics data sets!
Cross Correlating Datasets
In addition to everything else Crashlytics tracks, you can add a custom User ID to all your crash logs by calling Crashlytics.SetUserId(currentUserUserId);. It’s important to keep this ID as anonymous as possible, so I choose to generate and store a random GUID on first launch and send that to Crashlytics and Firebase Analytics:
Some important notes on this code:
I’m using ContinueWithOnMainThread to make sure that the continuation from CheckDependenciesAsync executes on the Unity main thread. This is an extension method provided by the Firebase plugin to make working with threads a little easier in Unity. Without that, accessing PlayerPrefs will typically cause an error. It’s also possible to wrap tasks in CustomYieldInstruction to achieve the same effect, especially if I’m working with a team that’s more familiar with Unity than typical .NET development.
I check for a user id first with HasKey. This way I ensure that the user’s id remains consistent through multiple runs of the current app installation.
I’m using Guid.NewGuid() to generate my key. This ensures that this identifier random and that no one else knows what it is until my game client reports it. It would also be much easier for me to regenerate the key if I needed to comply with local privacy legislation.
Asking the big queries
Now that we have a broken score tally event and we know that the design team has hooked in analytics around scoring, my next trick is to use a Google Cloud product called BigQuery.
A good thing to note is that Crashlytics will try to automatically capture what I’m about to do under Logs. Look there first, but BigQuery will still be invaluable for giving you a deep and complex insight into your game.
So I jump over to “Events” under “Analytics”:
and click the “Link to BigQuery” button at the bottom:
I can then click “View” to see all of the data for my game:
You may also get here by going over to the Google Cloud Console, and finding your Firebase project there:
Then either selecting BigQuery from the main dashboard:
Or from the side bar:
If you look in the sidebar now, you should see your Crashlytics and Analytics tables already imported and ready to go:
So, let’s get debugging!
A note: if you’re in fact following along and not debugging a live game, it may take up to 24 hours for any new analytics events to propagate. It may take up to another 24 hours after that for BigQuery to update. Take a nap, play some games, and come back ready to go in the morning!
I like to get started with any query by selecting the table, and clicking “Query Table”. This gets some of the starter boilerplate out of the way:
There’s one vital piece of information I need now: how do I find my bug? The exported table has a column named issue_id. This seems promising, but it appears to be a bunch of nonsense at a glance:
If we jump back into Crashlytics, it may not be obvious where to find this value:
But if you look at your URL, the issue_id is actually the string of characters immediately following /issues/:
In the case of our exception in “ScoreKeeper.TallyFinalScore,” the issue id is 080c5d3cc5ef219325ef09d775cb2843.
Therefore, to find my crash, I can type:
Let me tease this apart a little bit real fast. BigQuery uses a SQL like syntax to search potentially enormous datasets. So, we use SELECT * to pull out all of the column in the table com_firebase_test_crashlytics_ANDROID (and note that there’s a separate iOS table as well) that match the first of issue_id = "080c5d3cc5ef219325ef09d775cb2843".
I also filter out any issues without a user.id field. This is because I need to use this field in my next step, which is to associate these with analytics events. To do this, I create a new query:
I’m introducing some new syntax here.
First I use WITH and AS to turn these table names into readable variables. I can get these from clicking the “Query Table” button like I did before or copying from the side bar. Note that Firebase Analytics events are grouped by day, but you can use * to query all available tables:
Then, since there’s a lot of data to sift through, I chose some fields that I think are important with SELECT.
My JOIN statement joins the variables crashlytics and analytics together if their user ids match.
When I run this, I’m starting to see some useful information:
Which is kind of useful, but it just shows me every event that a user who experienced a crash. Feel free to sift this by hand, but I have 720 results on my own small test app. Let’s make one more small tweak:
I want to compare when crashlytics’ and analytics’ timestamps match, and to do this I need to convert them to a common format. Analytics uses microseconds since 1970 whereas crashlytics uses a string describing the time of the event. Since I’m doing some mathy things, I’ll use UNIX_MICROS to turn this string into an integer to make it easier to work with. I divide this by 1,000,000 to convert to seconds (since it’s really hard for me to think about microseconds).
I also order this by the absolute value, so values closer to 0 move to the top of my list. Now I’m starting to see analytics events close to when my crash occurred:
As I scroll through this, I notice a lot of tally_score events with ‘0’ as the enemy count. Armed with this data, I can run back to my crashing function, and see that int remainingEnemyValue = remainingEnemies[0].PointValue; will crash if FindObjectsOfType returns an empty list.
Onward and Upward!
Thanks for sticking around, I’ve had a lot to cover and still just barely scratched the surface of what Firebase can do to help you improve and iterate on your games. Games are large and complex objects that are always evolving with their users. In today’s gaming culture, players expect games to grow with them and improve over time. I hope I’ve given you a few new tools with which you can ask about and inspect the health of your game and help it grow to meet not only your standards, but your players’ as well!
