Case Study
How BallFields uses Firebase
A ritual of passage for being a parent in the US is experiencing the complete absorption of your weekends by youth travel sports when your child comes of age. For me, that started with travel baseball when my son turned ten — and continues to this day, four years later. Weekend after weekend you drive your child (and sometimes their friends) to games, tournaments, practices and social events often starting early on a Saturday and finishing (if you’re good) late on a Sunday.
More often than not, these facilities are far from home and are complete unknown to you before you get there. Will they have toilets (and toilet paper)? Do they sell food? Should I bring a chair? Is there shade? What are the fields like? Where should I park? Are all questions you find yourself asking again and again. This ritual keeps up from early spring to late fall (or autumn where I grew up).
I decided during the COVID-19 pandemic that I was going to build an app and a service for long suffering travel sports parents that “Helped them know what they were getting into, before they got there”. The idea for the BallFields App was born.
I had not written any meaningful amount of code for over 20 years and so I spent months sitting in a mini-van outside of kids sports facilities (capacity limits were in effect due to the pandemic and parents weren’t allowed in), stealing WiFi from wherever I could and learning SwiftUI, iOS development and investigating different platforms for the backend.
I learned so much along the way. In this particular post, I want to give a sense of how my solution took advantage of Cloud Firestore as a platform solution.
Why I chose Cloud Firestore
I am a team of one. I wanted to spend most of my time on creating a full featured user experience, not on managing a deep tech stack. So when I was thinking about my app and imaging the requirements both in the beginning and at scale, it became important for me to choose a platform which was robust, could scale, was well supported and had a great development and support community around it. It was also important to me that the SDK for it would be robust in the SwiftUI environment.
After building small pilot apps on three different platforms, Cloud Firestore clearly stood apart from the others. It’s performance, seamless offline capabilities, really robust and well documented SDK set it apart. The default offerings like Cloud Storage, Cloud Functions, Cloud Messaging, Authentication flexibility, Crash monitoring and comprehensive (and growing set of extensions) made it a clear and obvious choice. I have not regretted that choice for a moment.
A shout out to Freelancers
This story wouldn’t be complete if I didn’t acknowledge the freelance community in the world. I had no idea this community existed before I started this app. I needed to learn so much. I didn’t need people to write code for me, I needed people to teach me contemporary concepts. I learned how to write and structure Cloud Functions from someone in Pakistan. I learned how to write Javascript from someone in Australia. I learned about Icon development from some folks in India and I worked with a single mom in Macedonia for a year to research and create the initial detailed database of 5000 baseball fields in New Jersey with all their attributes. Sometimes it really does take a village.
How I use Cloud Firestore
The backend is a really important part of my app. A big part of my app allows parents to give to-the-minute ratings and impressions about different features of a particular baseball or softball facility. This includes ratings, hints and comments, game-day reports (text or video) and other like/dislike mechanisms. It’s a social app. In New Jersey alone, over 100,000 families participate in Travel Baseball on most weekends. At large tournaments, you will have thousands of visitors over the course of a weekend all (hopefully) interacting with the app at the same time. Those requirements mean that a lot of the work has to be handled by the backend and has to be structured really well for the client to be responsive.
A basic searchable store of information
Of course, the most important thing I need to do is keep track of my basic information Fields, Playing Surfaces, comments, ratings etc.
This was bread and butter for Cloud Firestore of course, but I found structuring the main data (ballFields — physical facilities and playingSurfaces — actual pitches that kids play on) as separate collections helpful. Technically, they are related to each other with one ballField having many playingSurfaces, however structuring them as separate collections allowed for different query and indexing options that wasn’t as easy with a sub-collection.
Search and Geo-lookup
Two critical features of the app are that users can search for a facility by any part of its name or address and they can also search for facilities which are nearby.
There is no full-text search feature available straight out of the box (note, you can use extensions for this for Algolia and other search engines) so I had to build one. To do this, I build a separate collection of search terms, based on the name, description, address etc. of the facility with a pointer from each term, back to an array of document ID’s for each facility. That collection is then indexed and allows for features like this.
There is no facility database maintenance from within my app. All of that is handled through a Javascript Web interface I built separately. To keep these indexes up to date any time facility information changes, I use the magic of a cloud function which looks at the update or creation of a ballField and updates the search index directly.
exports.updateBallfield = functions.region("us-east4").firestore.document('ballFields/{ballFieldID}').onUpdate(async (change, context) =>{
var oldData = change.before.data();
var newData = change.after.data();
var oldNameField = oldData['name'].toUpperCase();
var newNameField = newData['name'].toUpperCase();
var oldDescriptionField = oldData['description'].toUpperCase();
var newDescriptionField = newData['description'].toUpperCase();
if (oldNameField == newNameField && oldDescriptionField == newDescriptionField) {
return null;
}
var jointArray = newNameField.toUpperCase().split(" ").concat(newDescriptionField.toUpperCase().split(" "));
var resultField = jointArray.filter(exclusionFunction).filter(onlyUnique)
const lat = newData['entranceLat'];
const lng = newData['entranceLong'];
var hash = "";
if (lat != 0.0 && lng != 0.0){
hash = geocommon.geohashForLocation([lat, lng]);
} else {
hash = "";
}
const geoHash = hash;
const searchTerms = resultField;
return change.after.ref.set({
searchTerms,
geoHash
},{merge: true});
function exlusionFunction(word) {
let excludedList = ["FIELD", "BASEBALL","LITTLE","THE", "OF", "FROM", "IS", "BALL","HOME"];
return excludedList.findIndex(element => element == word) == -1;
}
function onlyUnique(value, index, self) {
return self.indexOf(value) === index;
}
});This cloud function has the advantage of creating the index. It also updates the geoHash for each field, based on a stored latitude and longitude — more on that later. Then attached to a .searchable field in my SwiftUI view, I asynchronously query that index.
func fetchBallFields(matching: String) async {
self.isSearching = true
let currentSearchTerm = matching.trimmingCharacters(in: .whitespaces)
let searchTerms = currentSearchTerm.uppercased().components(separatedBy: " ")
if currentSearchTerm.isEmpty {
locations = []
hasResult = false
} else {
let result = await BallFieldRepository.asyncMainReturnSearch(searchTerms: searchTerms)
locations = result
hasResult = true
}
self.isSearching = false
}
@MainActor
static func asyncMainReturnSearch(searchTerms: [String])async ->[BallField] {
var finalResults:[BallField] = []
for f in 0..<searchTerms.count {
if (searchTerms[f].count > 0) && (searchTerms[f] != " "){
do {
let results = try await COLLECTION_BALLFIELD_INDEX.document(searchTerms[f]).collection("ballFields").getDocuments()
let snapResults = results.documents.compactMap { document in
try? document.data(as: BallField.self)
}
finalResults.append(contentsOf: snapResults)
} catch {
defaultLogger.error("BALLFIELDS: failed in async get")
}
}
}
return finalResults
}You can also see in the Cloud Function that a geoHash is created or updated with every document update. One of the important value points of the database behind the app is that each physical facility in the database was studied for the best actual navigation point. I did this to make sure that when a user said “Take me there” that they navigated directly to the entry point of the facility. If you just rely on street addresses alone, particularly for parks, you can end up 1/2 a mile away from where the best place to enter the facility is. These latitude/longitude combinations are then geoHashed with 10 digits of precision. I used the geoCommon core library to do that. On the client side then, I have geoHash query code to allow location search behavior. This does two main things for a user: it allows a user to ask what facilities are near them and, if permissions are granted, keeps track of when a user arrives at the facility, records their visit (for gamification reasons later) and swaps all the views to the current facility.
For this functionality, I’ll use the SwiftGFUtils library that I published here. This library has no restrictions on which Firebase version is supported. The Swift code below does the search against Cloud Firestore for nearby locations.
static func getNearbyBallFields(latitude: Double, longitude: Double, radius: Double, completion: @escaping([BallField])->Void){
var interimResults: [BallField] = []
let center = CLLocationCoordinate2D(latitude: latitude, longitude: longitude)
let distanceMiles = Measurement(value: radius, unit: UnitLength.miles)
let distanceMeters = distanceMiles.converted(to: UnitLength.meters)
let dispatchGroup = DispatchGroup()
let queryBounds = GeoHashUtils.geoHashUtils.queryBoundsForLocation(location: center, radius: distanceMeters.value)
let queries = queryBounds.map {bound -> Query in
return COLLECTION_BALLFIELDS
.order(by: "geoHash")
.start(at: [bound.startValue])
.end(at: [bound.endValue])
}
func getDocumentsCompletion(snapshot: QuerySnapshot?, error: Error?) -> () {
guard let documents = snapshot?.documents else {
defaultLogger.error("BALLFIELDS: Unable to fetch snapshot data. \(String(describing: error))")
dispatchGroup.leave()
return
}
for document in documents {
var bf: BallField
let lat = document.data()["entranceLat"] as? Double ?? 0
let lng = document.data()["entranceLong"] as? Double ?? 0
let coordinates = CLLocation(latitude: lat, longitude: lng)
let centerPoint = CLLocation(latitude: center.latitude, longitude: center.longitude)
let distance = GeoHashUtils.geoHashUtils.distanceFromLocation(startLocation: centerPoint, endLocation: coordinates)
if distance <= distanceMeters.value {
bf = try! document.data(as: BallField.self)
bf.distanceFrom = distance
interimResults.append(bf)
}
}
dispatchGroup.leave()
}
for query in queries {
dispatchGroup.enter()
query.getDocuments(completion: getDocumentsCompletion)
}
dispatchGroup.notify(queue: .main) {
interimResults.sort { (lhs: BallField, rhs: BallField)->Bool in
return (lhs.distanceFrom ?? 0.0) < (rhs.distanceFrom ?? 0.0)
}
completion(interimResults)
}
}The ratings system, distributed counters and the use of Listeners
Two really useful parts of the BallFields App are it’s rating system and comment system. The rating system lets a user tell other users their point of view on how good or bad specific parts of the facility are on any particular day. Ratings reset at the end of the day, however the totals for the day are stored as trends for the long term. If I am successful, thousands of people will be providing ratings at the same facility at almost the same time. There are separate ratings systems for the facility (the physical premise) and each playing surface (where you play the game).
To make this work, I chose distributed counters which use a sharding scheme to make sure that high throughput updates can be captured without causing backend performance or data loss. This is one of the extensions which is freely available via Cloud Firestore. I confess that I originally tried to create this same functionality using transactions. The solution was hopelessly incapable of achieving the distributed performance that was necessary. Going with distributed counters made the solution robust, simple to implement, fast and reduced the amount of code I had to write. Counter updates are done asynchronously so don’t burden the client app. Distributed counters also let me store summary ratings by hour and month. You can see how easy this is to do in Swift from this partial code snippet below. I can’t stress enough how much value Cloud Firestore extensions were able to bring here.
static func addRating(whichFieldID: String, rating: BallFieldRating, completion: @escaping((Error?)->Void)){
let date = Date()
let dateFormatter = DateFormatter()
dateFormatter.dateFormat = "yyyy"
let yearString = dateFormatter.string(from: date)
let calendar = Calendar.current
let components = calendar.dateComponents([.hour,.month], from: date)
let monthString = String(components.month ?? 0)
let hourString = String(components.hour ?? 0)
let ballFieldDoc = COLLECTION_BALLFIELD_HOURLY_RATINGS.document(whichFieldID)
let parkingDoc = COLLECTION_BALLFIELD_HOURLY_RATINGS.document(whichFieldID).collection("ratings").document("parking")
let bathroomDoc = COLLECTION_BALLFIELD_HOURLY_RATINGS.document(whichFieldID).collection("ratings").document("bathroom")
let foodDoc = COLLECTION_BALLFIELD_HOURLY_RATINGS.document(whichFieldID).collection("ratings").document("food")
let shadeDoc = COLLECTION_BALLFIELD_HOURLY_RATINGS.document(whichFieldID).collection("ratings").document("shade")
let dayDoc = COLLECTION_BALLFIELD_HOURLY_RATINGS.document(whichFieldID).collection("ratings").document("day")
let monthDoc = COLLECTION_MONTHLY_BALLFIELD_RATINGS.document(whichFieldID).collection(yearString).document(monthString)
let whichUserID = Auth.auth().currentUser?.uid ?? ""
let gamificationDoc = COLLECTION_GAMIFICATION.document(whichUserID)
let parkingControllerSentimentPositive = FirebaseDistributedCounter(docRef: parkingDoc, field: "dailySentimentPositive")
let parkingControllerSentimentDenominator = FirebaseDistributedCounter(docRef: parkingDoc, field: "dailySentimentDenominator")
let parkingControllerNone = FirebaseDistributedCounter(docRef: parkingDoc, field: "dailyNoParkingDenominator")
let parkingControllerStreet = FirebaseDistributedCounter(docRef: parkingDoc, field: "dailyStreetParkingDenominator")
let parkingControllerLot = FirebaseDistributedCounter(docRef: parkingDoc, field: "dailyLotParkingDenominator")
let dayParkingPositiveController = FirebaseDistributedCounter(docRef: dayDoc, field: "parkingSentimentPositive")
let dayParkingDenominatorController = FirebaseDistributedCounter(docRef: dayDoc, field: "parkingSentimentDenominator")
let monthlyParkingSentimentController = FirebaseDistributedCounter(docRef: monthDoc, field: "parkingSentimentPositive")
let monthlyParkingSentimentDenominatorController = FirebaseDistributedCounter(docRef: monthDoc, field: "parkingSentimentDenominator")
switch(rating.parkingRating){
case 1:
parkingControllerNone.increment(by: Double(1))
case 2:
parkingControllerStreet.increment(by: Double(1))
case 3:
parkingControllerLot.increment(by: Double(1))
default:
break
}To manage the realtime nature of this, I also use Firestore listeners against these documents to provide a near realtime update as users everywhere provide their ratings. This code snippet shows the traditional and simple way to install a listener. My current user base loves the up-to-date nature of the rating system.
func addParkingListener(whichField: String){
if whichField == "" {
Self.defaultLogger.error("BALLFIELDS: Empty field ID passed to listener add.")
return
}
if parkingListener != nil {
removeParkingListener()
}
let query = COLLECTION_BALLFIELD_HOURLY_RATINGS.document(whichField).collection("ratings").document("parking")
parkingListener = query.addSnapshotListener { (snapshot, error) in
if let error = error {
Self.defaultLogger.error("BALLFIELDS: Error setting a listener message: \(error.localizedDescription)")
} else {
let result = try? snapshot?.data(as: BallFieldHourlyParking.self)
self.parkingResults = result ?? BallFieldHourlyParking(id: "parking")
}
}
}Comment system and @FirestoreQuery
Another important feature of the app is it’s comment system. Users can provide persistent comments which give hints to other users about what to expect or bring. They can categorize those comments for easy sorting and users can upvote and downvote them, based on their usefulness. Comments have an expiry date (more on that later) but that expiry date changes depending on how useful other users have voted that comment.
These comments also flow in in realtime. This happens using Cloud Firestore listeners again. Recently the Firebase team added a new property wrapper @FirestoreQuery which works a lot like the Apple core data @FetchRequest property wrapper. Adding this property wrapper makes adding and managing listeners really easy and allows you to change their snapshot criteria as the user interface changes. This code snippet shows how I do that.
import Firebase
import FirebaseFirestoreSwift
import os
struct FacilityCommentsScrollableView: View {
@EnvironmentObject var schemeTheme: Theme
@EnvironmentObject var bfRepository: BallFieldRepository
@EnvironmentObject var bfCommentRepository: BallFieldCommentRepository
@Environment(\.presentationMode) var pm
@StateObject private var commentListModel = FacilityCommentsListOO()
@State private var selectedArea = "All"
@State private var commentSort = "Default"
@Binding var isExpanded: Bool
@FirestoreQuery(collectionPath: "ballFieldComments") var comments: [BallFieldComment]
let dateFormatter : DateFormatter = {
let formatter = DateFormatter()
formatter.dateFormat = "EEEE"
return formatter
}()
let timeFormatter : DateFormatter = {
let formatter = DateFormatter()
formatter.dateFormat = "h:mma"
return formatter
}()
private static let defaultLogger = Logger(
subsystem: Bundle.main.bundleIdentifier!,
category: String(describing: FacilityCommentsScrollableView.self)
)
var body: some View {
ZStack (alignment: .bottomTrailing) {
if isExpanded {
Color("ModernHighContrastBackground").edgesIgnoringSafeArea(.all)
}
VStack {
...
}
...
}
.onChange(of: bfRepository.currentBallField.id, perform: { value in
let compareDate = Calendar.current.date(byAdding: .day, value: -30, to: Date())
if let ballFieldID = bfRepository.currentBallField.id {
if selectedArea == "All" {
$comments.predicates = [.whereField("whichField", isEqualTo: ballFieldID), .whereField("commentWhen", isGreaterThan: compareDate ?? Date()), .order(by: "commentWhen", descending: true)]
} else {
$comments.predicates = [.whereField("whichField", isEqualTo: ballFieldID), .whereField("commentWhen", isGreaterThan: compareDate ?? Date()), .whereField("commentCategory", isEqualTo: selectedArea), .order(by: "commentWhen", descending: true)]
}
}
}
)
}
}A Summary System
One of the features I wanted to provide to my users was the ability to see how a facility was rated over time — not just on the current day. The idea was to see what people thought of it during peak seasons, off seasons when busy, etc. To do that, I keep track of all the ratings on a daily basis and roll them over into month averages. These averages are then available to users in nice charts in the app. Users can choose which 30 days they want to look at over the past year.
Clearly, this is not work that you have a client app do and it’s not work you do in the moment. The Cloud Firestore environment shines here again. I was able to build a scheduled Cloud Function which took ratings each day, summarized them and reset for the next days rating to make sure everything was set for the next day.
The combination of the distributed counter extensions, Cloud functions and robust listeners create a great user experience.
exports.resetPlayingSurfacesSummary = functions.region("us-east4").runWith({ memory: '1GB', timeoutSeconds: 180 }).pubsub.schedule('0 0 * * *').onRun(async (context) => {
var endDate = new Date();
var startDate = new Date();
startDate.setHours(startDate.getHours() - 24);
await firestore.collection('playingSurfaceHourlyRatings').where("todayRatingsCounter", '>', 0).get().then(async (snapshot) => {
for (var i in snapshot.docs) {
let fieldID = snapshot.docs[i].id
await firestore.collection('playingSurfaceHourlyRatings').doc(fieldID).collection('ratings').doc('day').get().then(async (doc) => {
if (doc.exists) {
var data = doc.data();
// let theDate = data['lastUpdateTimestamp'].toDate();
let theDate = startDate;
const convertedDate = convertTZ(theDate, "America/New_York")
var dd = String(convertedDate.getDate()).padStart(2, '0');
var mm = String(convertedDate.getMonth()+1).padStart(2, '0');
var yyyy = convertedDate.getFullYear();
var today = ""
today = yyyy + "-" + mm + "-" + dd;
var newRecord = {};
newRecord['lastUpdateTimestamp'] = admin.firestore.FieldValue.serverTimestamp();
data['lastUpdateTimestamp'] = admin.firestore.FieldValue.serverTimestamp();
var rootRecord = {};
rootRecord['todayRatingsCounter'] = 0;
await firestore.collection('playingSurfaceHourlyRatings').doc(fieldID).set(rootRecord);
await firestore.collection('playingSurfaceHourlyRatings').doc(fieldID).collection('ratingsHistory').doc(today).set(data);
await firestore.collection('playingSurfaceHourlyRatings').doc(fieldID).collection('ratings').doc('outfield').set(newRecord);
await firestore.collection('playingSurfaceHourlyRatings').doc(fieldID).collection('ratings').doc('infield').set(newRecord);
await firestore.collection('playingSurfaceHourlyRatings').doc(fieldID).collection('ratings').doc('mound').set(newRecord);
await firestore.collection('playingSurfaceHourlyRatings').doc(fieldID).collection('ratings').doc('dugout').set(newRecord);
await firestore.collection('playingSurfaceHourlyRatings').doc(fieldID).collection('ratings').doc('overall').set(newRecord);
await firestore.collection('playingSurfaceHourlyRatings').doc(fieldID).collection('ratings').doc('day').set(newRecord);
} else {
functions.logger.info('Day document doesnt exist');
}
});
}
});
function convertTZ(date, tzString) {
return new Date((typeof date === "string" ? new Date(date) : date).toLocaleString("en-US", {timeZone: tzString}));
}
})Media, Messaging and Reporting
Travel sport parents share a lot. They share a lot of chatting, they share a lot of media and they share a lot of insights in what they’ve found at any particular location. I wanted to build in team messaging, team image sharing and “report” sharing into the app for parents in teams.
One of the great developer advantages of using Firebase as a platform is that you also get access to Cloud Storage and that Cloud Storage works seamlessly with Cloud Firestore documents and listeners where needed. The first two functions I built were real time messaging and image (video and photo sharing) in streams for teams.
The messaging was straightforward with @FirestoreQuery making coding easy and creating a realtime user experience for chat with individuals or in a whole team. The “BallRoll” image sharing and liking feature makes use of Cloud Storage, including the extension to resize the image once it’s loaded, Firestore listeners to make all users experience realtime as people post in their rolls and very importantly, makes use of TTL functions (more in a moment) to manage the storage needs and clutter of any teams environment.
I should say, underpinning all of this is a robust user privileges system that makes sure only a user can control data which they post — and that they can remove it at any time, including if they delete their account. Once again, Cloud Firestore really made the development experience easier here by providing background services to manage user data — all keyed off a user ID.
A relatively new app feature is a GameDay report. These are short (either text or maximum 30 second videos) which can be recorded live with the camera or keyboard and shared instantly. They appear in the main BallField screen as they are submitted. They are meant to be instantaneous condition reports. They expire three days after they are submitted so that the user experience is never cluttered. This was also managed using a combination of Cloud Firestore documents, Cloud Storage files, Listeners and TTL policies — all of which are part of the Cloud Firestore platform.
Managing Data
As the app went into production, it was important that daily backups could easily take place. Once again a simple cloud function, triggered to run daily and given reporting roles in Google Cloud console allowed for a robust backup experience with very little effort from a development perspective.
exports.scheduledFirestoreExport = functions.region("us-east4").pubsub
.schedule('every 24 hours')
.onRun((context) => {
const projectId = process.env.GCP_PROJECT || process.env.GCLOUD_PROJECT;
const databaseName = client.databasePath(projectId, '(default)');
console.log(databaseName);
return client.exportDocuments({
name: databaseName, outputUriPrefix: bucket, collectionIds: []})
.then(responses => {
const response = responses[0];
console.log('Operation Name: '+response['name']);
})
.catch(err => {
console.error(err);
throw new Error('Export operation failed');
});
});Finally, perhaps one of my favorite features which was recently introduced is Time To Live (or TTL). I’ve removed hundreds of lines of Javascript and Typescript code and reduced my backend footprint with this feature. It allows you to set up a Google Cloud policy that takes a document field as a parameter (deleteWhen or whatever you call it) as a date and then any document in that collection or subCollection with that field in it, will be deleted approximately when that date occurs. I once told a Google developer advocate that approximately 70% of my coding effort had been on functionality that wasn’t part of the core experience. It’s features like TTL which continue to be introduced that give a developer a chance to spend more time on user experience, and let the magic just happen on the platform.
And So…
There are so many more features of my app which I would love to share, however the real purpose here was to highlight how I used Cloud Firestore to bring something valuable to parents like me. The app really relies on a Network effect (like all social apps) to be at it’s highest value — and I have a long way to go to get there — but I do hope that some of this description gives other developers, particularly indie developers, ideas on how they can accelerate their solutions using Cloud Firestore.
