String Encodings Decoded With Elixir

Know the working of strings

Is UTF-8 some kind of unbreakable code?

It is time you gained membership in the secret society of the dark question marks �����. This may be the last explanation of encodings you will ever need to read.

I had run afoul of string encodings for years, but it wasn’t until I started working with Elixir that I finally really understood them. It’s been a few years since my string-enlightenment, but let me share with you all lessons I learned so you can put the misunderstandings behind you. Here follows my tale of hard-won bits of knowledge about strings and encodings that I wish somebody had told me. ☯⚡❁♛

Before you can really understand encodings, you need a greater appreciation for the string data type — yes, the backstory is necessary, and if you really want to blow your mind, I highly recommend reading “The String Type is Broken” by Edaqa Mortoray.

The particular koan to meditate on is that strings are different than any other data type. Booleans are simple on/offs; numbers can be constrained by their size (e.g. an 8-bit number). But a string? It could be infinitely long, comprised of virtually anything. Curious curious… (brew that confusion for a moment)…

I always felt that the academic studies surrounding software development were completely backwards: you learn early on…