My Revenge on the Emojis

This week I received an email from a user that he was not able to import his browser bookmarks to Bookmark Ninja. I checked the log on the server and saw immediately the exception in the MySQL database. The exception was caused by a special character in the title of a website. It was an emoji! A “nail polish” emoji! But why did it cause an exception? My database is configured to support unicode UTF8 encoding, so it should work, right? NO! Emojis are 4 bytes, UTF8 supports max 3 bytes/character. I didn’t know this about emojis. You cannot squeeze 4 bytes into 3 bytes… Shit.

I replied him, that I was not aware of this issue and would fix it in the future. I was very thankful for finding this problem, because nobody has run into this issue, yet. Yes, this is a bug, but I didn’t find it a high priority issue, it’s not very common to have emojis in the website titles.

A few days later the same user contacted me again. He couldn’t add a bookmark to Ninja on his mobile device. I checked the log again. Guess what? The website title had an emoji again… This was the point when I decided to fix this issue not in the future but next week.

Option #1

An obvious fix could be to change the encoding in the database from UTF8 to UTF8MB4, that supports a maximum of 4 bytes per character. It sounds very simple, but making this change require a lot of configuration changes in the database. So it’s risky and would require a lot of testing to make sure no errors were introduced by these changes. Also I read that the 4 byte support has some negative impact on the performance of the database, which I didn’t like at all.

Option #2

Another fix could be to simply replace the emojis in the website titles with some character that is supported by the current database configuration. This is a safe fix. In this case you will lose the emojis but at least you can add the bookmark to Ninja. But even if the database supported the emojis the display of the emojis on the website would bring up new problems again…

So I decided I would choose option #2. Will do the fix next week and will destroy all the emojis!