The PureScript logo by Gareth Hughes is licensed under the CC-BY 4.0 license

PureScript and Haskell at Lumi

Note: This post and our new engineering blog can be found at lumi.dev

At Lumi, we take the correctness of our code seriously. “Code correctness” is a tricky thing to define fully, but we can begin by trying to eradicate things like type errors and unhandled exceptions. This sort of correctness is important for understanding and communicating data confidently and accurately, especially when that data is related to large financial transactions or the manufacture of large numbers of physical goods.

Previously, I talked about how we were able to ensure correctness while successfully migrating our database from RethinkDB to Postgres by leaning heavily on Haskell. That was almost one year ago, and since then we’ve been able to build on the work done there, to develop a large and stable backend application.

In this blog post, I’m going to tell a similar story about how we’re improving correctness on the front-end by moving from JavaScript to PureScript.

The need for types on the front-end

We get a lot of benefit from using Haskell on the backend at Lumi. All of our API servers and clients are written in Haskell using Servant, and we use esqueleto and persistent (with a generous helping of custom DSLs) to write our SQL queries. It is extremely rare for us to encounter a runtime error in production on our servers, and when we do, it is usually of the “business logic error” variety. It’s hard to overstate the value of having types everywhere on the backend.

Our front-end is a large web application written using React, with plenty of logic of its own. So why then, if types are so useful to us, were we using untyped JavaScript with React on the front-end, when JavaScript offers comparatively few guarantees? There are a few answers:

1. When our product was new, there were fewer options available, and the options we might have chosen were immature.
2. Everyone on the team was able to write JavaScript effectively, but not everyone was familiar with compile-to-JavaScript languages. Now we have enough developers that we don’t need to worry about that, but originally, it was useful for everyone to be able to contribute easily.
3. JavaScript was a suitable choice for a small application initially, allowing us to iterate quickly, but is less suitable now that the project has grown in scope. It is not particularly difficult to build new code in JavaScript, but it is difficult to maintain a large JavaScript application, requiring extensive tests for things which could be covered by types.

Of course, types are not just a tool for guaranteeing correctness. A common complaint is that a type system might decrease developer productivity by requiring the user to add type annotations to provide any guarantee of correctness, but a sufficiently expressive type system like the one in GHC is able to actually increase developer productivity by providing tools like parametric polymorphism, type classes, type-level programming and datatype generic programming.

Finally, types are also a wonderfully succinct language for expressing ideas and processes, and we wanted that language to be available on the front-end as well.

The decision to use PureScript

I had personally used typed front-end languages (TypeScript and PureScript) extensively, and other team members had similar experience with other options like Flow and Elm. As the original developer of the PureScript compiler, I had an obvious interest in seeing it adopted at Lumi, but I was also aware that it might not be the optimal choice of front-end language for a few reasons:

• Other team members might not enjoy using it as much as I do, or might find themselves to be less productive.
• Using a relatively uncommon programming language might make it more difficult to hire developers.
• Without language-level support for things like JSX and CSS, we would need to find some other way to work with our design team, who previously were able to modify our React components directly if necessary.
• The library ecosystem is relatively small, and we would need to build some things of our own in order to be productive.

As a team, we were in agreement that JavaScript was causing a lot of problems for us, but we weren’t sure about the best approach to fix them. To my pleasant surprise, a majority of the team were enthusiastic about trying PureScript as the first option, because it fits our needs well.

You might be asking how we chose PureScript out of the many available options for typed front-end development. Aside from the in-house experience and my own preference for demonstrating its use on a large real-world application, there are a few technical reasons. It is simplest to just say that PureScript was the unique solution to the following set of constraints:

• The setup process should be simple and unintrusive. It should be trivial to set up an environment to quickly test out ideas.
• The language should integrate smoothly with JavaScript, its libraries and its build tools.
• The type system should be expressive, supporting things like sum types, row polymorphism, type classes and higher-kinded types.
• It should be easy to build simple solutions, but still possible to experiment with more advanced ideas. As Justin Woo puts it, the language should have a culture of “the sky is the limit” and should not limit your creativity.
• The development of the language itself should be open enough that we would be able to modify any part of the toolchain if it became necessary.

Getting started

Eventually, we decided to jump in and try out PureScript by replacing one of our existing JSX-based React components. We chose a simple, pure component with no side-effects or API calls. After setting up the PureScript compiler in our existing Webpack-based build, we were off to a good start.

I was a little concerned about the suitability of the existing React bindings for our purposes (they were a little too complicated, as they tried to support the full React API, which we didn’t need), so I cobbled together a simplified set of React bindings for us to use. We’ve since polished and released those bindings as a separate library called purescript-react-basic.

Now that we had proven that the approach could work, we started porting more and more of our pure components over to react-basic. We knew, however, that we wanted to be able to replace our API calls and page-level components as well. For this, we decided to use code generation, since our API is large and changes reasonably frequently, and we wanted to ensure as much correctness as we could.

Generating types

The first step was to generate a complete set of PureScript types to correspond to the types we used in our Haskell API. To solve this problem, we turned to GHC Haskell’s support for datatype-generic programming. We created a simplified representation of PureScript data types (PursTypeConstructor), and a type class for Haskell record types which would be converted into PureScript types (ToPursType):

data PursRecord = PursRecord
  { recordFields :: [(Maybe Text, PursType)]
  }
data PursTypeConstructor = PursTypeConstructor
  { name      :: Text
  , dataCtors :: [(Text, PursRecord)]
  }
class ToPursType a where
  toPursType :: Tagged a PursTypeConstructor
  default toPursType 
    :: ( Generic a
       , GenericToPursType (Rep a)
       )
    => Tagged a PursTypeConstructor
  toPursType = retag $ genericToPursType @(Rep a) id

The toPursType member of the type class creates a representation of the type class, tagged with the original Haskell type it originated from.

It would be tedious and error-prone to write these instances out by hand, so we provide a default implementation of the type class for record types which implement the Generic interface. Luckily, GHC will derive Generic instances for us if we turn on the -XDeriveGeneric extension, so generating these representations of our Haskell types is practically free.

Once we have a list of PursTypeConstructor structures, we can turn them into PureScript code fairly easily — nothing fancy needed here, just simple string templating. For convenience, we also emit a little extra code to make our generated types usable on the PureScript side: serialization boilerplate (itself derived using PureScript’s own version of datatype-generic programming!), Lenses and Isos for all fields and data constructors, functions for debugging, and so on.

Generating API clients

The next step was to turn our Haskell API definitions into usable, safe PureScript clients. Fortunately, Servant is perfect for this task — since our API definitions are represented at the type-level, we can turn those definitions into PureScript code and know for sure that the resulting code will be compatible with the server implementation.

The servant-foreign library was perfect for solving this problem, since it generates the data structures we need, including lists of API endpoints with reasonable names and all of the types of query parameters, request bodies and response bodies. From there, it’s just a question of assembling the PureScript code from those data structures.

The only tricky bit is providing the necessary HasForeignType instances which are necessary in order to convert names of Haskell types into names of PureScript types. We chose to reuse the same names, and then a dash of Template Haskell magic is all that’s needed to traverse the graph of types and generate all of the necessary instances, thanks to the incredibly useful th-reify-many library:

$(do names <- reifyManyWithoutInstances ''HasForeignType
       [ ''Order
       , -- ... a list of other top-level types goes here
       ] (const True)
     let toInstance nm =
           let tyCon = TH.ConT nm
               nmLit = TH.LitE (TH.StringL (TH.nameBase nm))
           in [d| instance HasForeignType
                             Purs
                             PursType
                             $(pure tyCon)
                    where
                      typeFor _ _ _ = PursTyCon $(pure nmLit)
                |]
     concat <$> mapM toInstance names
 )

This needs a little explanation:

• reifyManyWithoutInstances traverses the type graph looking for types without HasForeignType instances
• For each type it finds, the toInstance function turns its name into a HasForeign type instance using a Template Haskell splice. The tyCon and nmLit nodes are in scope, so we can use anti-quotation $(...) to use them in the splice.

Types as a tool

As I mentioned earlier, a good type system should increase productivity by reducing busywork. In addition to making our API clients free from boilerplate, we’ve been able to increase productivity in a number of other areas since implementing PureScript. I hope to be able to write about each of these in some detail in future:

• We have built a collection of completely generic UI components on top of our typed API clients. For example, we have one table component which is parameterized by the API which provides its data and search capabilities. If we change the API, the compiler reminds us to update the table!
• We have built a combinator library for assembling forms which are compatible with our API types. By using lenses and a handful of basic functions, we are able to build type-safe forms in a fraction of the time it would take by hand.
• We have also built a type-level DSL in the style of Servant for deriving forms from types for certain data collection tasks. Just as Servant allows us to repurpose our type-level API definitions for the generation of API clients and documentation, we can reuse our type-level form descriptions for all sorts of things like storage in the database, indexing and querying.

We have plans to implement more type-directed tools in the future:

• As I described in my blog post about our migration to Postgres, we have implemented a completely generic backend solution on top of our Postgres database, including filtering, search, and computed fields. We are working towards implementing the same level of reusability on the front-end, by abstracting over common API client patterns.
• Now that our API clients are represented on the front-end, we would like to find new ways of representing our API calls, and find ways to implement features like batching and caching in a generic way.

If this sort of work sounds appealing, we are hiring!

Conclusion

While we still have a way to go, our experience with PureScript so far has been very positive. Instead of touting its benefits myself, I’ll leave you with a few quotes from the team:

The main thing I notice is that it’s easier to expand abstractions. You can take something really specific and incrementally make it more general.

— Brady

[I’ve had] such a positive experience, with little mental overhead, and total trust in the compiler. I implemented an entire page with a list of data, filters, search, and pagination which worked first time.

— Brandon

I’ve been a fan of Javascript’s “loose” style; but as the applications I’ve worked on got larger, fear of the “null” or “undefined” began to take over. Using PureScript ensures any incorrect logic is caught or contained. I can feel more confident in the code I push out.

— Kimi

After a few months in PureScript, having to edit the occasional JavaScript file fills me with anxiety — a typo or incorrect assumption anywhere could break half the site! In PureScript, even if I’ve gotten the business logic wrong somewhere, the problem is always contained.

— Madeline