Benchmarks: GHCJS (Reflex, Miso) & Purescript (Pux, Thermite, Halogen)
Table of contents
1. Motivation
2. Benchmark results
2a. “Keyed” implementations
2b. “Non-keyed” implementations
3. Comments about the benchmark results
3a. Comments: Purescript (in general)
3b. Comments: Purescript/Halogen
3c. Comments: Purescript/Pux
3d. Comments: Purescript/Thermite
3e. Comments: GHCJS/Reflex-DOM
3f. Comments: GHCJS/Miso
4. Personal Opinions
4a. Thoughts about GHCJS/Reflex-DOM — by Alex
4b. Thoughts about Purescript (in general) — by Thomas
4c. Thoughts about Purescript/Halogen — by Thomas
4d. Thoughts about Purescript/Pux — by Thomas
4e. Thoughts about Purescript/Thermite — by Thomas
4f. Thoughts about GHCJS/Miso — by Saurabh
5. What next?
6. Would you like to contribute?
Motivation
Once we were over the infamous Haskell learning-curve, we began looking for functional programming, immutability, and types everywhere! Given that one-third of our code runs in the browser (via Angular v1 — for now!), it is only a matter of time before we make the switch to typed-FP for front-end development as well.
But, which language/framework/library should we use? That’s a million dollar question which doesn’t have a clear answer. While we are still looking for the right-fit for our use-case & team, we have some interesting data to share.
As part of the the Haskell Bounty Program, Alex Esteves and Thomas Honeyman spent a couple of weeks getting benchmarks for the following libraries merged into JS Framework Benchmarks repository.
- PureScript/Halogen — merged upstream
- PureScript/Thermite — merged upstream
- PureScript/Pux — merged upstream
- GHCJS/Reflex-DOM — PR awaiting review
- GHC/MisoJS — PR rejected
While we managed to get a few of them merged, the conversation around the others was “interesting” (to put it mildly!)
CONTRIBUTION OPPORTUNITY: We encourage readers to dig into these PRs and make either the benchmarks faster, or contribute to areas where the FP libraries are lacking in performance. Wrt benchmarks, we recommend having an “i
diomatic"and a "tuned"version, to give users a sense of “out-of-the-box” vs “hand-tuned” performance of the library.
Unofficial benchmark results
BIG DISCLAIMER: These are unofficial benchmark results. The benchmarks have been run on our development machines. They probably mean nothing in absolute terms. Please use these numbers only to compare the frameworks amongst themselves.
Please look at the official results as well. However, due to various back-stories, not all FP benchmarks were merged into the main repository.
Benchmarks for “keyed” implementations
Benchmarks for “non-keyed” implementations
Comments about the benchmark results
PureScript: In General
PureScript is a language focused on correctness, power, and safety. It’s a joy to work in, but that emphasis means PureScript is slower than optimized JavaScript. As a relatively new language, the compiler still has plenty of quick-win optimizations to be made.
On these benchmarks, PureScript can get as little as a 2x slowdown as compared to vanilla JS. Nate Faubion implemented an Elm-like library called Spork that achieved this result. The three frameworks used most often in production — Pux, Thermite, and Halogen — came in closer to a 4x slowdown on these benchmarks.
If speed is your #1 consideration, PureScript may not be the right choice. Whether speed is a reasonable trade-off for the power of the language will depend on your use case.
Purescript: Halogen
Halogen is the second-fastest PureScript framework by a hair, and is about 3x slower than React and Angular. It was the only framework in our test that relies on a virtual DOM written in PureScript.
Halogen optimizes for efficient deep updates rather than updates from the root, which is the opposite case from what this benchmark is designed to test.
Comment by Nate Faubion: PureScript will never be as fast as optimized JS, but if you build a test that optimizes for the cases this test runs, it can be pretty fast. Halogen uses components to thunk [instead of referential thunking, like Elm]. There’s not a reason to implement referential thunking, which is only an optimization when you are diffing the tree from the root on every change.
Halogen applications can optimize for speed by focusing on the changes made in components and avoiding overly flat & broad structures.
Purescript: Pux
Pux was the slowest framework we tested. The primary reason for this is Smolder, the monadic HTML DSL that Pux uses. While perfectly fine for a small number of nodes, this monad dramatically slows down as the number of DOM nodes in the page grows. For full details on Smolder and how this problem affects Pux, see this issue.
Pux’s author, Alex Mingoia, had a few extra notes: Pux is not focused on performance yet. The slow performance arises from translating Pux’s (smolder) virtual DOM to React’s virtual DOM. The goal is to write a purescript virtual DOM module for smolder, which would avoid that translation step and could be optimized for a monadic data-structure. I suspect this would achieve performance on par with Halogen.
Elm = Virtual DOM -> DOM patch
Halogen = PureScript Virtual DOM -> DOM patch
Pux = Smolder Markup -> React Virtual DOM -> DOM patch\Once Pux cuts out that unnecessary step performance should significantly improve.
That said, Elm-like frameworks in PureScript show promise of similar performance. While Spork isn’t production-ready, it’s built atop a PureScript virtual DOM and reaches near-Elm speeds. With changes, Pux should be capable of the same.
Purescript: Thermite
Thermite, written by PureScript’s creator, Phil Freeman, was the fastest framework in the benchmarks. It’s a high-level wrapper for React, and I’d hoped to see comparable speeds. But Thermite is about 3x slower than React in these benchmarks.
The primary reason for this seems to be that Thermite has had no performance tweaks since its release. As the compiler improves, Thermite improves, but it doesn’t appear Thermite is the answer for blistering-fast code in PureScript.
GHCJS/Reflex-DOM
The swap/remove benchmarks come very close to Vanilla performance, with select/update coming in at < 3x. It’s the “churning out DOM” operations that perform badly. The performance issues seem to be mostly from non-FRP stuff — there’s a bunch of inlining/specialisation opportunities that GHC isn’t picking up for whatever reason.
The benchmark slowdown was around ~8x before “handing it over” to Ryan Trinkle (Reflex FRP’s author), who made it twice as fast. Adding a preload optimization used by all frameworks brought us down to ~3.5x.
Closure compiler was run on the resulting JS file, but only with simple optimizations (removed about 33% of the total file-size) due to what appears to be a GHCJS bug. This likely contributes to startup time.
GHCJS/Miso
Since Miso is essentially a Haskell implementation of “The Elm Architecture”, it would be fair to compare it against Elm itself. Unfortunately, the Miso benchmark implementation is categorised as “non-keyed” whereas Elm is “keyed”. Therefore the numbers are not directly comparable. Given that caveat, Miso is within 2–3x overall performance of Elm. In fact, had it not been for one particular benchmark (“select rowDuration to highlight a row in response to a click on the row”) in which Miso is 4x slower than Elm, the numbers would be looking even better for Miso.
This is a significant achievement for Miso/GHCJS! In fact, after the initial hiccups, the GHCJS/Reflex-FRP numbers also look promising (compared to all the Purescript frameworks). We’re not sure why this is the case — is GHCJS emitting well-optimised code, or is Purescript emitting really non-performant code?
Comment from David (Miso’s author): A few core algorithmic optimisations, like the keys patch, are being worked upon. The benchmarks for Miso should improve significantly once this work is completed. Right now, benchmarking seems premature.
BIG DISCLAIMER: What follows next, are highly personal opinions on each library/language. We have tried linking to relevant Github issues for two reasons: (a) offering some context for why we are making a particular claim, and (b) to allow future readers to easily see if a particular issue has been solved, thus making the claim invalid.
We had also reached out to the framework authors for their comments, and have published their comments in the relevant sections below.
Thoughts about GHCJS/Reflex-DOM (by Alex)
Reflex is an FRP implementation, whereas Reflex-DOM is a lib/framework built on Reflex & GHCJS.DOM (lets you target both the browser and ‘native’ applications through WebkitGtk).
There’s quite a learning curve to it (kind of like with Haskell), especially if you’ve never done FRP before, but learning it can be quite enlightening (kind of like with Haskell).
Tooling
Reflex-DOM avoids dependency hell by pinning down everything through Nix. Actually trying it out is pretty simple. Either:
- Use the playground at http://hsnippet.com/ — has some samples that one can tinker with, and run right in the browser
- Clone https://github.com/reflex-frp/reflex-platform and run ‘try-reflex’ — this drops you in a nix-shell where GHC + GHCJS + Reflex + Reflex-DOM are all available, so imports just work and simple scripts compile with no hassle. Running this script with an empty nix store took me 6 minutes of mostly binary downloads from https://nixcache.reflex-frp.org. Subsequent runs only take 10 seconds henceforth due to the local Nix cache.
Architecture
Disclaimer: parts of this might also apply to other FRP implementations, but I don’t really have experience with them
Working with Reflex feels a lot like the original message-passing promise of OO, haskell-style. Your ‘widgets’ are basically functions which happen to receive and/or return signals in addition to pure values (well, technically, the return signal is wrapped in Reflex widget monad). This makes widgets very self-contained and reusable.
Example: a button with a label that shows the amount of clicks
main = mainWidget $ display =<< count =<< button “ClickMe”-- ormain = mainWidget $ do
clicks <- button “ClickMe”
current <- count clicks
display current
One consequence of this is that there’s a lot of freedom on the way you plug things together. There’s no “One True Way” of doing things in Reflex-DOM (at least not yet), as opposed to The Elm Architecture (TEA) approach.
Like TEA, state is plumbed down and events are plumbed up. However, you can consume the events to transform the state at whatever scope you want and not only at top level.
There are a lot of combinator functions for multiplexing/demultiplexing signals (e.g. lists, tables). Some force your event payloads to yield the new state, others want you to be more granular and provide a diff. The more granular you are, the more your code looks like a specification of the system’s, erm, reactions. On the other side, having nested layers of signals can make the types quite nasty.
Reflex-DOM doesn’t do DOM-diffing (at least not yet), so sometimes this choice has performance implications (might not matter enough outside of benchmarks). Reflex-DOM comes with a bunch of widgets and there’s some more experimental ones at https://github.com/reflex-frp/reflex-dom-contrib
Related discussions:
- https://www.reddit.com/r/reflexfrp/comments/6l5ddn/how_to_structure_a_reflex_application/
- https://www.reddit.com/r/haskell/comments/5z6n4o/reflex_or_blazereact_for_writing_haskell_web_apps/
Community
Both r/reflex-frp and #reflex-frp are welcoming places for newbie questions and existential crisis alike. Reflex-DOM’s author has been helpful every step of the way, with guidance and contributions. The benchmark had significantly worse performance numbers at first, and he saw it as an opportunity to profile and optimize the framework (and tweaking the benchmark itself). It has since been adopted as a benchmark for Reflex-DOM itself.
Documentation
I’d say this is the most lacking area of the Reflex ecosystem right now.
There’s a rather sudden transition from entry-level tutorials to hackage-only guidance. IMHO it makes it harder to grok FRP and/or Reflex.This is exacerbated by there being many ways to skin a cat. I find myself frequently unsure of whether I’m plumbing things the “best” way, of even if there is one.
- https://github.com/hansroland/reflex-dom-inbits/blob/master/tutorial.md
- https://reflex-frp.readthedocs.io/en/latest/
Comments by Saurabh: I’ve played around with Reflex about 6 months ago and felt lost due to the lack of a “UI architecture.” Most people (including me) have never worked with an FRP framework, and could use guiding principles while working with the library.
Thoughts about PureScript, in general (by Thomas)
This discussion is about three PureScript frameworks. But beneath each is the language itself. The code you write benefits from the power and flexibility of PureScript quite apart from the relative merits of each framework.
Language
PureScript is a functional language influenced by Haskell. If you know Haskell, it’s worth spending some time with PureScript by Example. If you haven’t used Haskell or PureScript, then start with The Haskell Book.
PureScript’s design ethos places safety, power, and correctness over raw speed. Neither the language nor the three most popular frameworks optimize much for speed. If you need raw speed, PureScript should not be your first choice. As you review the benchmarks, consider whether the slowdown is enough to hurt your use case.
What if only some of your application is performance sensitive? PureScript has a pleasant FFI to JavaScript, so you can write performance-tuned JS and use it in PureScript.
Tooling
PureScript has lovely tooling for a young language. The build tool, Pulp, provides easy compilation and can even run PureScript projects that rely on Node or projects like PhantomJS. Package management relies on the existing JavaScript ecosystem. Pull in JavaScript libraries with NPM, or PureScript ones with Bower. You can avoid Bower by using psc-package, which provides package sets guaranteed to build together. Finally, editor support comes from the psc-ide package.
I use Spacemacs, and had a streamlined editor in a minute or so by installing the PureScript layer.
Interop
PureScript has an excellent FFI for JavaScript. You can shore up any deficiencies in the PureScript ecosystem by interfacing with your favorite JS library. If you use React, both Thermite and Pux integrate easily with existing React applications.
PureScript is closer to Haskell than any other language, and many developers using it have a Haskell backend. It’s tedious replicating data types and JSON serializers & deserializers between the frontend and backend. PureScript has several resources to limit this as an issue.
- purescript-bridge allows you to generate PureScript types from Haskell ones, which you can then import into your PureScript project and use.
- Both Haskell and PureScript support generic encoding / decoding for JSON. PureScript’s Data.Argonaut.Generic.Aeson provides a flavor of generic decoding and encoding that matches Aeson’s default values.
Personal Verdict
The following is Thomas’ personal opinion. We (Vacation Labs) are still evaluating frontend frameworks and have not decided on anything yet.
I learned PureScript while implementing these benchmarks. I’m also building a medium-sized analytics SPA. I implemented parts of the frontend in Elm, PureScript, and JavaScript, including each of the three frameworks discussed here. In the end, I settled on PureScript’s Halogen as my favorite framework for real-world use.
The Quick & Dirty Framework Comparison (by Thomas)
Common comments on Thermite, Pux, & Halogen
The PureScript frameworks have much in common. Rather than repeat myself ad nauseam, I’ve summarized key differences here.
Implementation
Thermite & Pux rely on React and ReactDOM. Halogen relies on a virtual DOM implementation, making it the only 100% PureScript framework. It’s possible that this virtual DOM will extend to cover rendering via React in the future.
JS Interop
PureScript by design interfaces well with JavaScript; this extends to the three frameworks. If you’re using React, Thermite and Pux are especially easy as they’re based on it. Halogen has the greatest number of examples of wrapping external JS libraries as components in Halogen, including the Ace editor and the ECharts charting library.
Architecture
- Thermite hews closely to React, positioned as a high level wrapper around it. Using Thermite means carrying over lots of your ideas from React. A Thermite app is a React component.
- Pux renders with React. It’s designed to mirror pre-0.17 Elm architecture.
- Halogen doesn’t have a firm design philosophy. You can have a single top-level component and use the same architecture as Pux or Elm, or you can make everything a component and use it like Thermite or React, or move anywhere in between. As always, this flexibility is a blessing and a curse.
Subjective Ease of Use
- Thermite requires the least time spent learning the framework. That said, you’ll want to understand React and lenses. I haven’t built anything non-trivial in Thermite, but it feels quite easy to work with.
- Learning Pux means learning the Elm architecture, too. If you’re coming from Elm, this will be a natural transition. It’s the most opinionated framework of the three.
- Halogen has the scariest types and, on the surface, seems like the most intimidating framework to learn. In practice, though, this is rarely a problem. Once you’re building an app, Halogen’s flexibility and approach to components made it a winner for me.
Purescript: Thoughts on Halogen (by Thomas)
Phil Freeman, PureScript’s creator, described Halogen with “I think of this as more like an “industrial-strength” version of Thermite.”
I felt the same way as I used Halogen. It appears to be the most-used framework in production and the authors are active in maintaining and pushing the project forward. It’s the only framework written 100% in PureScript.
Official Resources
Learning Resources
Community
- Halogen’s developers, including SlamData and the language’s creator, Phil Freeman, are active on the Functional Programming Slack team.
Purescript: Thoughts on Pux (by Thomas)
Pux derives from The Elm Architecture. If you want to build applications in the Elm style, but with access to PureScript’s greater power, Pux is a good choice.
Official Resources:
Learning Resources
- Composing Components with Pux — Andre Van Der Merwe
- Write React Components in PureScript with Pux — Adam Recvlohe
- Let’s Build a Simon Game — Adam Recvlohe
Note: Following the Elm Architecture tutorial in Pux is a good way to learn.
Community
Purescript: Thoughts on Thermite (by Thomas)
Thermite is a high-level wrapper around React. If you ultimately want your PureScript application to become a React component in an existing application, Thermite seems like a good pick. If you’re building your full frontend in PureScript, however, Halogen and Pux may be better choices.
Official Resources
- GitHub Repo
- Examples
- The main guide is in the repo itself. There is also an official interactive tutorial.
Learning Resources
Community
- Thermite is maintained by Phil Freeman, who is active on the Functional Programming Slack team.
Thoughts on GHCJS/Miso (by Saurabh)
Miso is a small Haskell/GHCJS front-end framework that is heavily inspired by Elm (in fact, it implements TEA — The Elm Architecture — pretty faithfully in Haskell, IIUC). I had never written a single line of Elm code before I started writing the Miso benchmark. But, as a testament to Miso’s simplicity, it did not prevent me from picking it up within a couple of hours. I had the first cut of the benchmark ready within 6 hours, or so!
Tooling
Tooling is the biggest problem with the entire GHCJS ecosystem right now (this is not specific to Miso). Editor tooling seems to be non-existent, or flaky, at best. And this is not a fault of Miso, per-se.
This flakiness is one of the main reasons why we might not be adopting any Haskell front-end library, even though having a backend in Haskell makes it the obvious choice for us. (And, surprisingly, the perf-benchmarks are surprisingly better than what we expected them to be!)
If I may offer some unsolicited advice here: anyone who is working on the GHCJS ecosystem should prioritise the GHCJS tooling problem immediately. Contribute in any way you can. I don’t see significant adoption for GHCJS happening if one has to first struggle with which version of the compiler is going to work with which Stack LTS, download it from some obscure URL, and then have separate stack files (one for GHC & one for GHCJS), and then struggle with JS-shims to get your editor (which talks only to GHC) to compile code targeting the browser, but then finally compile with GHCJS to deploy, and then some Closure optimisations don’t work, and on, and on…
JS *Widget* Interop
I think it would be very hard to find a language having a poor inter-op story with its host VM (eg. Purescript-JS, GHCJS-JS, Eta-JVM, Clojure-JVM, F#-CLR, etc). What is more interesting to me is the impedance mismatch when reusing complete libraries/components written in the host language (eg. using Hibernate or Jooq with Eta, jQuery widgets with Purescript, etc)
Given this context, it was very important to check whether existing UI widgets/components written in JS could be reused with Miso. Unfortunately, it seems that being faithful to Elm/TEA would make this task harder for Miso than it should be. Even if it is possible to rewrite basic UI components in Miso natively, frankly, I’m not sure if it is worth the effort. Having said that, I haven’t spent appreciable time on this problem and the Miso team might even have something up-their-sleeves to solve this.
Comment from David (Miso’s author): In regards to pure reusable components. Falco has done good work here (he’s using it in prod as well). This distinction is that these components are pure, not impure like react/reflex. So no I/O. It’s pretty beautiful IMO. Takes advantage of the fact `Effect action` is a monad over `model`. This is something Elm can’t do since it doesn’t have higher kinded types.
Yet another String type?!
By introducing yet another String type (JSString/MisoString), Miso (or is it GHCJS?) makes a bad situation even worse! This is actually the very first mistake that I made in my benchmarking code. I wonder if there is ever going to be a sensible use-case for Text/ByteString/String on the browser. If not, why can’t GHCJS convert all “native Haskell string types” to JSString/MisoString automatically (unless some flag or pragma specifies otherwise)?
Comment from David (Miso’s author): MisoString is just JSString on ghcjs, and Text when compiled with ghc. It’s just a synonym.
What next?
We’re still evaluating our options for using typed-FP in front-end web-development. It seems that we’ll end-up going with the safest option out there (even mentioned by Isaac Shapira in “Selecting a Platform”) — React + TypeScript + Ramda + Linter
Note about React: Since writing the first draft of this blog post, we seem to have hit a wall trying to do typed-FP in the React ecosystem. We’re going to give it some more time, and share our experience in a follow-up blog post.
We’re still figuring out how to configure this stack to get the essential features of typed-FP, namely:
- Immutability by default
- Completely typed
- Using FP instead of OOP (it seems that OOP and mutability are highly correlated)
Once we’re happy with what we have, we’ll benchmark this stack as well, and share our results.
Would you like to contribute?
At Vacation Labs, we’re helping travel companies come online with a suite of SaaS products (website builder, booking engine, CRM, distribution system, etc). If you’re interested in using Haskell in this domain, we’re hiring full-time Haskell engineers. If you’re a student (or a working professional) looking for a internship, we’ve got plenty of full-time internship opportunities. If you can’t commit full-time, there is the Haskell Bounty Programme, as well.
