Image for post
Image for post
The PureScript logo by Gareth Hughes is licensed under the CC-BY 4.0 license

PureScript and Haskell at Lumi

Phil Freeman
Aug 6, 2018 · 9 min read

Note: This post and our new engineering blog can be found at lumi.dev

At Lumi, we take the correctness of our code seriously. “Code correctness” is a tricky thing to define fully, but we can begin by trying to eradicate things like type errors and unhandled exceptions. This sort of correctness is important for understanding and communicating data confidently and accurately, especially when that data is related to large financial transactions or the manufacture of large numbers of physical goods.

Previously, I talked about how we were able to ensure correctness while successfully migrating our database from RethinkDB to Postgres by leaning heavily on Haskell. That was almost one year ago, and since then we’ve been able to build on the work done there, to develop a large and stable backend application.

In this blog post, I’m going to tell a similar story about how we’re improving correctness on the front-end by moving from JavaScript to PureScript.

The need for types on the front-end

Our front-end is a large web application written using React, with plenty of logic of its own. So why then, if types are so useful to us, were we using untyped JavaScript with React on the front-end, when JavaScript offers comparatively few guarantees? There are a few answers:

  1. When our product was new, there were fewer options available, and the options we might have chosen were immature.
  2. Everyone on the team was able to write JavaScript effectively, but not everyone was familiar with compile-to-JavaScript languages. Now we have enough developers that we don’t need to worry about that, but originally, it was useful for everyone to be able to contribute easily.
  3. JavaScript was a suitable choice for a small application initially, allowing us to iterate quickly, but is less suitable now that the project has grown in scope. It is not particularly difficult to build new code in JavaScript, but it is difficult to maintain a large JavaScript application, requiring extensive tests for things which could be covered by types.

Of course, types are not just a tool for guaranteeing correctness. A common complaint is that a type system might decrease developer productivity by requiring the user to add type annotations to provide any guarantee of correctness, but a sufficiently expressive type system like the one in GHC is able to actually increase developer productivity by providing tools like parametric polymorphism, type classes, type-level programming and datatype generic programming.

Finally, types are also a wonderfully succinct language for expressing ideas and processes, and we wanted that language to be available on the front-end as well.

The decision to use PureScript

  • Other team members might not enjoy using it as much as I do, or might find themselves to be less productive.
  • Using a relatively uncommon programming language might make it more difficult to hire developers.
  • Without language-level support for things like JSX and CSS, we would need to find some other way to work with our design team, who previously were able to modify our React components directly if necessary.
  • The library ecosystem is relatively small, and we would need to build some things of our own in order to be productive.

As a team, we were in agreement that JavaScript was causing a lot of problems for us, but we weren’t sure about the best approach to fix them. To my pleasant surprise, a majority of the team were enthusiastic about trying PureScript as the first option, because it fits our needs well.

You might be asking how we chose PureScript out of the many available options for typed front-end development. Aside from the in-house experience and my own preference for demonstrating its use on a large real-world application, there are a few technical reasons. It is simplest to just say that PureScript was the unique solution to the following set of constraints:

  • The setup process should be simple and unintrusive. It should be trivial to set up an environment to quickly test out ideas.
  • The language should integrate smoothly with JavaScript, its libraries and its build tools.
  • The type system should be expressive, supporting things like sum types, row polymorphism, type classes and higher-kinded types.
  • It should be easy to build simple solutions, but still possible to experiment with more advanced ideas. As Justin Woo puts it, the language should have a culture of “the sky is the limit” and should not limit your creativity.
  • The development of the language itself should be open enough that we would be able to modify any part of the toolchain if it became necessary.

Getting started

I was a little concerned about the suitability of the existing React bindings for our purposes (they were a little too complicated, as they tried to support the full React API, which we didn’t need), so I cobbled together a simplified set of React bindings for us to use. We’ve since polished and released those bindings as a separate library called purescript-react-basic.

Now that we had proven that the approach could work, we started porting more and more of our pure components over to react-basic. We knew, however, that we wanted to be able to replace our API calls and page-level components as well. For this, we decided to use code generation, since our API is large and changes reasonably frequently, and we wanted to ensure as much correctness as we could.

Generating types

data PursRecord = PursRecord
{ recordFields :: [(Maybe Text, PursType)]
}
data PursTypeConstructor = PursTypeConstructor
{ name :: Text
, dataCtors :: [(Text, PursRecord)]
}
class ToPursType a where
toPursType :: Tagged a PursTypeConstructor
default toPursType
:: ( Generic a
, GenericToPursType (Rep a)
)
=> Tagged a PursTypeConstructor
toPursType = retag $ genericToPursType @(Rep a) id

The toPursType member of the type class creates a representation of the type class, tagged with the original Haskell type it originated from.

It would be tedious and error-prone to write these instances out by hand, so we provide a default implementation of the type class for record types which implement the Generic interface. Luckily, GHC will derive Generic instances for us if we turn on the -XDeriveGeneric extension, so generating these representations of our Haskell types is practically free.

Once we have a list of PursTypeConstructor structures, we can turn them into PureScript code fairly easily — nothing fancy needed here, just simple string templating. For convenience, we also emit a little extra code to make our generated types usable on the PureScript side: serialization boilerplate (itself derived using PureScript’s own version of datatype-generic programming!), Lenses and Isos for all fields and data constructors, functions for debugging, and so on.

Generating API clients

The servant-foreign library was perfect for solving this problem, since it generates the data structures we need, including lists of API endpoints with reasonable names and all of the types of query parameters, request bodies and response bodies. From there, it’s just a question of assembling the PureScript code from those data structures.

The only tricky bit is providing the necessary HasForeignType instances which are necessary in order to convert names of Haskell types into names of PureScript types. We chose to reuse the same names, and then a dash of Template Haskell magic is all that’s needed to traverse the graph of types and generate all of the necessary instances, thanks to the incredibly useful th-reify-many library:

$(do names <- reifyManyWithoutInstances ''HasForeignType
[ ''Order
, -- ... a list of other top-level types goes here
] (const True)
let toInstance nm =
let tyCon = TH.ConT nm
nmLit = TH.LitE (TH.StringL (TH.nameBase nm))
in [d| instance HasForeignType
Purs
PursType
$(pure tyCon)
where
typeFor _ _ _ = PursTyCon $(pure nmLit)
|]
concat <$> mapM toInstance names
)

This needs a little explanation:

  • reifyManyWithoutInstances traverses the type graph looking for types without HasForeignType instances
  • For each type it finds, the toInstance function turns its name into a HasForeign type instance using a Template Haskell splice. The tyCon and nmLit nodes are in scope, so we can use anti-quotation $(...) to use them in the splice.

Types as a tool

  • We have built a collection of completely generic UI components on top of our typed API clients. For example, we have one table component which is parameterized by the API which provides its data and search capabilities. If we change the API, the compiler reminds us to update the table!
  • We have built a combinator library for assembling forms which are compatible with our API types. By using lenses and a handful of basic functions, we are able to build type-safe forms in a fraction of the time it would take by hand.
  • We have also built a type-level DSL in the style of Servant for deriving forms from types for certain data collection tasks. Just as Servant allows us to repurpose our type-level API definitions for the generation of API clients and documentation, we can reuse our type-level form descriptions for all sorts of things like storage in the database, indexing and querying.

We have plans to implement more type-directed tools in the future:

  • As I described in my blog post about our migration to Postgres, we have implemented a completely generic backend solution on top of our Postgres database, including filtering, search, and computed fields. We are working towards implementing the same level of reusability on the front-end, by abstracting over common API client patterns.
  • Now that our API clients are represented on the front-end, we would like to find new ways of representing our API calls, and find ways to implement features like batching and caching in a generic way.

If this sort of work sounds appealing, we are hiring!

Conclusion

The main thing I notice is that it’s easier to expand abstractions. You can take something really specific and incrementally make it more general.

— Brady

[I’ve had] such a positive experience, with little mental overhead, and total trust in the compiler. I implemented an entire page with a list of data, filters, search, and pagination which worked first time.

— Brandon

I’ve been a fan of Javascript’s “loose” style; but as the applications I’ve worked on got larger, fear of the “null” or “undefined” began to take over. Using PureScript ensures any incorrect logic is caught or contained. I can feel more confident in the code I push out.

— Kimi

After a few months in PureScript, having to edit the occasional JavaScript file fills me with anxiety — a typo or incorrect assumption anywhere could break half the site! In PureScript, even if I’ve gotten the business logic wrong somewhere, the problem is always contained.

— Madeline

Lumi

Find us at https://www.lumi.com/blog

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store