How We Gaymoji: An Exercise in XP

In March of this year, Grindr introduced Gaymoji, enabling a user to send various manifestations of eggplant emojis to other users.

Fear of complexity can often hinder forward progress, particularly for development of new features. There are a host of unknown unknowns that commonly need to be tackled. A new feature like this would require an application server, discussions about modeling, and a database schema. And, of course, no feature is ready before a lengthy discussion about which new database is best. There are 3rd party SDKs to consider with contracts, metrics, costs, approvals, frameworks, etc… You know the drill. You have sat through these meetings before. It’s the elephant in the room, the intractable problem; it keeps you up at night, sick and stressed out.

I’m going to tell you a story about how none of this happened. I’m going to tell you how the Gaymoji feature was created without writing any server side application code at all.

Serverless architecture? No. We didn’t write any server side application code, not even lambdas in a serverless architecture.

Third Parties? No. We don’t use any third party to serve Gaymojis.

This is not a trick. What we did is obvious, yet rarely practiced. We were able to do this by adhering to one constant — build what you need.

Simply put, it is best to not write code

At Grindr, we use really cool technologies. In particular, we have been making use of Erlang and its younger cousin, Elixir. We would have been elated to use Elixir and its “sort-of-like-rails” framework, Phoenix, had we needed to write server side code. But we could only write new code if it adhered to two basic best practices.

  1. YAGNI (you ain’t gonna need it)
  2. KISS (keep it simple stupid)

I would like to suggest that these are not merely best practices; rather, they are codes of conduct. It is best to conduct oneself in a way which does best by your employer, your product, your customers, and your fellow engineers. Distilled to their ultimate meaning — your best code is the code you don’t write.

Jeff Atwood puts it like this:

It’s painful for most software developers to acknowledge this, because they love code so much, but the best code is no code at all. Every new line of code you willingly bring into the world is code that has to be debugged, code that has to be read and understood, code that has to be supported. Every time you write new code, you should do so reluctantly, under duress, because you completely exhausted all your other options.

We gave ourselves a challenge. Could we launch an entirely new feature without writing any application server code at all?

An aside about client code

I am focused on server side code, but the same rules apply to the client code. New features require client code because the client is the user interface for the user. Nevertheless, we do our best to remain backwards compatible — we attempt to develop features that work as well as possible without new client code. Solving for this problem follows the same rules that we are following on the server side, namely, aiming to write the simplest code possible. The part that is backwards compatible is the portion of the feature that requires no new code.

First Iteration

An effective way to tackle a large feature is to break it down into the smallest piece that still results in an effective product and then implement only that piece. Without practice, this is actually quite uncomfortable for an engineer and the organization. It requires deliberately not building for any other iteration. In our case, for example, this meant not building that fancy new application server in Elixir.

The original requirements for Gaymoji had been somewhat defined for over a year, but after a few meetings we boiled them down to some easy to understand points:

  • Users can send Gaymojis in messages as well as receive them.
  • We might want to update Gaymojis over time.
  • We might want to sell Gaymojis to advertisers.
  • We might want users to be able to click a Gaymoji and be directed to a website.

The next step was to define the first iteration. Notice that three of the points are mights — we don’t have enough data to know if the business will want to do any of these. We can throw out each might and focus only on those things which we know. Furthermore, each might is dependent upon the ability of a user to send and receive Gaymojis. The user story practically writes itself.

As a user, I want to be able to send and receive Gaymojis in messages.

Now, the scope is greatly simplified. Without thinking about any other considerations, could we make this user story happen? Could we do it without writing code?

First, we examined which features were supported by the client. Users could already send images in chat messages. Sending an image simply consists of a specifically formatted message that contains the image id. The image, in turn, is retrieved with a API call by the receiver. Gaymojis are just images, so we could send their ids and the clients would display them. Keeping it simple gave us our backwards compatible implementation.

While anybody could receive Gaymojis without new client code, code would need to be developed to send Gaymojis. The client would need to know about Gaymojis, have a selector, and allow the user to send them in messages. Additionally, new clients could know about Gaymojis and optimize the display in messages. We could maintain backwards compatibility by adding an extra JSON field indicating that the message is a Gaymoji type of image which the new clients could parse, but old clients would ignore.

Backend

Now that the client implementation was solved, what about the backend? How would the client reference the Gaymoji images?

Normally, this would be solved with an application server. The application server would perform CRUD operations on Gaymoji resources and the corresponding attributes (id, name, location) and expose these resources with a RESTful API.

Application server development is exactly what we wanted to avoid, however — after all, it requires significant code, servers, and a database to manage.

“you ain’t gonna need it”

Remember, we were developing only for our user story. That was our commitment. The big revelation was that we did not actually need an application server in order to implement our user story. If we don’t need it now, we ain’t gonna need it. All we needed was an index of the Gaymojis and the images to be accessible over our RESTful image API.

We came up with a son index file of Gaymoji that looks something like the following:

{
"gaymoji": [
...
{
"name": "2_Eggplant_1_soft",
"id": "2_Eggplant_1_soft.png",
"category": "dating+sex"
},
{
"name": "2_Eggplant_1_under_magnifying_glass",
"id": "2_Eggplant_1_under_magnifying_glass.png",
"category": "dating+sex"
},
{
"name": "2_Eggplant_1_with_piercing",
"id": "2_Eggplant_1_with_piercing.png",
"category": "dating+sex"
},
{
"name": "2_Eggplant_1_with_ring_on_it",
"id": "2_Eggplant_1_with_ring_on_it.png",
"category": "dating+sex"
},
...
]
}

The client could then access the gaymoji with an API call like cdn.grindr.com/path/to/gaymoji/2_Eggplant_1_soft.png.

Keeping it simple.

  1. By using a predictable id, we avoided the need to have a database manage the id.
  2. Ordering is just based on the alphanumeric order of the name.
  3. There is no functionality for deletion or updating because the Gaymojis available to the client are just what is defined within the json file.

Putting it all together

We decided to leverage Amazon S3 for serving our index file and Gaymoji images because it is simple, cheap to expose over HTTP, and is easily automated with the command line tool.

aws s3 cp index.json s3://${S3_BUCKET}/${PATH} \
--content-type "application/json" --cache-control max-age=300 \
--acl public-read

We managed the images in a git version-controlled project and added scripts to generate the index file directly from the image files, as well as for uploading the index file and images to S3. This process, in turn, is managed and executed with a makefile.

Simple. To change our Gaymoji pack and put it into production, all it required was:

  1. Add, remove, or rename an actual image file in the Gaymoji directory.
  2. Run make all S3_BUCKET=our_bucket

Even our product manager was able to make changes.

From start to finish, including planning and client changes, the feature took one month to get into the hands of users.

Rolling it out the clients, we had high user engagement (3 million sent) and an article on the front page of the New York Times!

That’s real value.

Second Iteration

Almost immediately after the success of our Gaymoji feature, Ad Sales wanted to run a campaign.

Let’s break this down into a user story:

As an advertiser, I want to add a Gaymoji to the client so that users can click through to my site.

There were two things we needed to do:

  1. Update the Gaymoji index and images to add a new Gaymoji for the advertiser.
  2. Create a clickable Gaymoji.

We already had the ability to update the Gaymoji and images; simply update the image file and run make.

Creating a clickable Gaymoji would require a change on the client, but the server side change was simple because we didn’t have application server code to update. All we needed to do was add a field for a click-through url in the index as well as a signature to make sure it was our click-through.

{ "gaymoji": [
...
{
"tag": "t9dFTH/ziixNdrKCTZJjycCO1JSuMeSGMYUFCq2nvKM=",
"href": "https://intomore.com",
"advertiser": "INTO",
"name": "1_6_1_into_logo_2",
"id": "1_6_1_into_logo_2.png",
"category": "featured"
},
...
]
}

In order to generate this index file, we added a metadata file to our Gaymoji repository that allows us to specify the click-through url for any given Gaymoji. This feature was simple to add precisely because it didn’t require any production runtime code or application server changes. All we had to do was update a utility script.

For the client, adding the change was simple because we had only implemented what was needed. The old clients ignored the new field, so they would display Gaymoji without the click-through. New clients would read the new field, and allow the click-through action.

The feature only took two weeks — plenty of time for us to successfully run the advertisement campaign!

That’s real value.

Third Iteration

The third iteration is a story about the Fourth of July and a great idea from Marketing.

“We have Fourth of July Gaymojis and would like them to be in the application. Next week would be great! Oh… and please make them visible for selection only in the United States.”

This was the real test of our minimalist approach so far. We had built an implementation to only meet a given user story and nothing else. Now, we were faced with targeting Gaymojis by country and asked to do it in a single week, no less. We didn’t have an application server to return a dynamic index or query based on location. Would this be the end of our code-less backend solution? Would this prove our entire approach wrong? Did we have to drop everything and implement an application server with a database?

We developed the user story…

As a user, I want to send Fourth of July Gaymojis beginning a week before the Fourth of July.

…and agreed to get it done in one week.

Why did we think we could get it done? Our approach so far was solid — all we needed was to return a different index dependent on country. We suspected we could rig some way of accomplishing this without redoing everything. Perhaps there was still an iterative solution hiding in plain sight…

All of our Gaymoji requests are proxied through Cloudflare. It turns out that Cloudflare can add a CF-IPCountry header to a request which specifies the country code for a requester's IP address.

Amazon S3 can not act dynamically on a country code, but it is simple to configure nginx to return a specific asset based on an arbitrary header.

Now we needed servers, but still didn’t need application code, because Nginx would take care of the work. We set up two EC2 instances (configured with automated ansible scripts in our Gaymoji repository, of course) and moved the index file from S3 to Nginx. An entry was added to our metadata file to specify country targets when needed, and the scripts generated an index per country and a universal index for any non-matching country.

In a single week, we were able to go live with country-targeted Gaymoji and everybody had a wonderful 1 million Gaymoji-filled Fourth of July.

That’s real value.

Future Iterations

Although the deployment process for updating Gaymoji is automated, if changes needed to happen frequently, it could possibly be a burden for our team. For example, if Ad Sales was frequently starting and stopping campaigns at awkward hours, we would have to manually run the deploy for each change at that time.

Originally this was the expectation, but reality is almost always different than expectations. Currently, Gaymojis are updated infrequently, and this is precisely why building only what was needed versus what might be needed paid off quicker.

Nevertheless, there could come a day when there are many Gaymoji based advertisement campaigns. If that day comes we can add a field to the metadata file and add functionality to our scripts to create an index per day. Nginx can then be leveraged to select the static index file by matching on date just as it currently does for country code.

Until then, there are other items of real value that need to get done.

The Takeaway

Although it seems counter-intuitive, doing the most minimal thing to accomplish a story really does get to value faster. We have been challenging ourselves regularly to only build what we need, and not only did it give us super powers, but it resulted in better design.

If you enjoyed reading this and want to go wild with XP programming practices, check out our jobs.


Ben Brodie, Senior Software Engineer, Chat Team, Grindr