Playing with Typelevel Programming

Krisztian Lachata
Feb 18, 2017 · 5 min read

Some while ago I came across Typelevel Programming and I wanted to know more about it. I tried to dig deeper in the subject and I found a number of interesting articles dealing with the topic:

I especially enjoyed the second one since it covered more concepts from the area with easily understandable examples. This was the first time I had an AHHHAAAA!! feeling with the Aux pattern

Meanwhile I was exploring the subject I felt like more and more trace leads back to Shapeless which is a Generic Programming library for Scala. Despite the fact that is has a fairly good documentation I was still struggling to put into practice what I read and learnt there.

Then a game changer book was published by Dave Gurnell and Underscore about Shapeless: The Type Astronaut’s Guide to Shapeless Book

This was the second time I had AHHHAAAA! feeling and step by step I started to feel that this is something I might be able to use even in my daily job confidently — assuming my team mates are not committing suicide immediately after seeing my PR.

Armed with my new book/knowledge I started to think about a potential pet project what I can use to sink my teeth into the topic more. One of my colleagues — Tomek — suggested me to play with the machine readable travel documents format and try to write a parser for it. It seemed to be a brilliant idea. I literally spent days and nights to come up with a reasonable solution. It gave me good opportunity to pull in Cats as well to start get used to it in this post scalaz world.

My solution was nearly ready but one feature was missing. I wanted to use hlist.Intersection to remove items from my generic representation before it is converted to the final case class. The modification was a minor change but the impact was tremendous. My test class didn’t seem to compile. I mean there wasn’t any error from the compiler but instead it was compiling the test class for hours turning my Mac into a central heating system… I was experimenting with the solution and the number of fields in the target case class and it turned out that every additional field roughly doubles the compile time. By the time I had 5 fields it was already 9–10 minutes!! And I needed 11 fields… Good luck!

I was very disappointed. Then I attended Miles’s presentation at the Scala eXchange 2016 and I saw the light at the end of the tunnel. I didn’t know that time if it was really the end of the tunnel or just a fast train running towards me. I thought that instead of using Lightbend Scala, I should give Typelevel Scala a try. It meant to contain some fixes and improvements on top of Lightbend Scala. Unfortunately it couldn’t solve the issue. For the real break through I had to wait, until last week…

I had high expectations against Typelevel Scala 2.12.1 fast implicit resolution. I felt that this time it will be different! And it turned out that it helped a lot. The compilation time dropped from ~6hours to 3 minutes although the cpu still behaves like a heater :)

So now I have reached the point where I can publish my MRD solution based on Shapeless.

Custom Typesafe MRD parser with Shapeless

Typesafe parser for machine readable documents.
The library allows to define different formats of documents encoded
using ICAO 9303 standard (passports/identity cards/others) and to define
parsers that parse these documents into case classes in typesafe manner.

Sample code:

Usage

  1. Define your case class what serves as a target container for the parser. In this case it is PassportMetadata
  2. Use predefined field parsers to define your MRD format by creating an HList with parsers in the proper document field order. Tag parsers based on its target field name in the case class:
    StringField which can extract an arbitrary long string from an MRD document
    DateField which can be used to extract dates
    SexField which can extract gender
  3. Run the parser against a given string

    Please note that you can have less field in your target case class than parser in your MRD format HList. If a tag doesn't have a field in the case class then it will be dropped.
    Please also note that the case class field order doesn't matter as long as the field name and the parser tag is in align!!

Custom Typesafe MRD parser vs Scodec MRD encoder

Recently I have started to play with scodec in my other pet project, after getting my parser working with 2.12.1 Typelevel compiler I thought it would be cool to implement the same MRD parser with scodec as well to see how my solution works compared to a mature binary parser scala library and vica versa.

Bear in mind that there is more than likely better and nicer solutions for both of the implementations. If you have suggestions how I could improve them feel free to share with me!

Scodec MRD encoder

Sample code:

Usage

  1. Define your case class what serves as a target container for the parser. In this case it is PassportMetadata
  2. Use predefined scodec encoders/combinators to define your MRD format.
  3. Run the encoder against a given string

Please note that you can have less field in your target case class than encoder if you use Unit encoders like ignore. Adding name to the encoder is not directly binding the encoded value to a case class field. It is just for helping the user in case of encoding error.
Please also note that the case class field order does matter!!

Conclusion

Apart from my core parser implementation, both solutions have roughly the same size of domain related code:

Custom solution

  • My solution has just a handful of parsers
  • Harder to extend and less generic
  • However my solution supports arbitrary case class field ordering as long as the parser tag has the same name!
  • case class may have less fields than parsers in your MRD format HList. If a parser tag doesn’t exists in the case class field name list, it will be just simple removed.

Scodec solution

  • Loads of different combinators to deal with data encoding. I really like the checksummed encoder. It is really easy and convenient to use
  • Easily extendable
  • It would be amazing to use encoder names to bind the encoded value to the case class field via name. With this solution we could have arbitrary case class field orders!

For more details check my github repository!!

Making Gumtree

This publication is made up of posts from the product and development team members at Gumtree UK. The opinions we share here are our own. We are based in Richmond, London.

Krisztian Lachata

Written by

senior dev @ https://www.ovoenergy.com

Making Gumtree

This publication is made up of posts from the product and development team members at Gumtree UK. The opinions we share here are our own. We are based in Richmond, London.

More From Medium

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade