Haskell-y Ever After: Summer Tales for Every Full-stack Developer (Part One)

A Google Summer of Code’s student story about implementing Haskell on the Hackage Matrix Builder’s frontend application using Functional Reactive Programming approach.

Andika D Riyandi (Rizary)
Jul 12 · 13 min read

TL;DR: This post is about my attempt to rewrite all of Hackage Matrix Builder’s (HMB) frontend code from Purescript to Haskell using one of Functional Reactive Programming library called Reflex/Reflex-Dom as part of Google Summer of Code (GSoC) 2019 project. This post will explain the decision to rewrite the frontend from Purescript to Haskell, and all of my lessons learned from using Reflex/Reflex-Dom (e.g., creating a simple homepage, calling backend API, creating a tag, and adding search box feature) on Hackage Matrix Builder frontend code up until GSoC first evaluation. The reader should be aware that this is not about the Reflex/Reflex-Dom tutorial nor its best practice. I will divide the article into three part and posted it after each of GSoC evaluation phase. Finally, I highly recommend to any full-stack developer out there to try Haskell on their backend or frontend web application. And if you already have backend written in Haskell, then you might want to try Reflex/Reflex-Dom for your frontend application.

Preface

Background

Since this is not a fairy tales, I will not start with “once upon a time…” as an opening sentence. Instead, I will begin by letting you know about Google Summer of Code, Haskell.org, and my project as brief as possible then go to the detail afterward. Start with GSoC; this is an annual program held by Google to give all student in the world a chance to do an open-source software coding project during the summer. From my point of view, open source and student are not a good fit, so Google compensates them for their work. Haskell.org, which are one of the organizations that got accepted to participate in GSoC 2019, is an organization that provides project ideas using Haskell, a purely functional programing language. And the last part is my project, Improving Hackage Matrix Builder as a Real-world Fullstack Haskell Project, an ambitious plan to improve Hackage Matrix Builder as a full-stack Haskell web application which provides a build compatibility test service to the Haskell community for all the packages published on Hackage.

Hackage Matrix Builder in a Nutshell

Hackage Matrix Builder project description comes up to my attention when Haskell.org held their own Haskell Summer of Code in 2017. As I quoted below:

The Hackage matrix builder provides a build compatibility test service to the Haskell community for all the packages published on Hackage. Currently, it answers the question “which versions of GHC does this package build with”, which includes checking the Cabal solver can find a solution and have that solution build successfully. It works by building all versions of all packages with several GHC versions. It relies on clever nix-style caching to make this reasonably efficient. So this is a kind of QA or CI service for everyone. There is also a lot of potentials to use this kind of infrastructure for additional purposes.

We can also say that Hackage Matrix Builder is a massive building bot which repeatedly tries to build packages on Hackage to help detect inaccurate metadata and other quality issues with packages. It is not a substitute for setting up your own CI, it’s instead intended as an additional line of defense to catch problems which weren’t caught by your project CI. Hence, Hackage Matrix Build’s goal is to help to ensure the overall health of the Hackage package collection and aid with curating Hackage (i.e. to help both package maintainers as well as Hackage Trustees).

This is really essential because Cabal/Hackage is built around the constraint solving paradigm and as such, Cabal/Hackage required that the metadata is accurate and the package collection consisting of all versions of packages has sound metadata. Moreover, it is also able to collect the basic runtime metrics of GHC during compilation (memory usage, compile time etc), and can thus help detect suspicious GHC compile time regressions over a big package collection. Interesting isn’t it? Keep on reading!!!

Mentor and Haskell Community

There are two mentors assigned by Haskell.org to guide me for this project in case I got lost while diving into the Haskell Great Sea. They are Herbert Valerio Riedel and Robert Klotzner. Herbert is the lead developer of the Matrix Hackage Builder project. He was also my mentor when I built the project’s user interface using Purescript, a strongly-typed functional programming language that compiles to JavaScript, written in Haskell. Robert, on the other hand, is the developer behind Gonimo, a free baby monitor for smartphone, tablet or PC, also written using Reflex/Reflex-Dom library. Another mentor that is not officially assigned but tremendously helpful is Ben Kolera, FP developer, mentor, and evangelist at Queensland Functional Programming Lab. Ben also authored the RealWorld example app using Reflex to adhere to the Reflex community style guides & best practices (it is more complicated than what I did). Last but not least, all Haskell community, especially reflex’s IRC channel on #reflex-frp (I wish they can have a discord channel, so I don’t have to subscribe to IRCCloud).

From Purescript to Haskell

The idea to move the entire project’s frontend codebase from Purescript into Haskell happened in 2018 when I asked Herbert to mentor me on GSoC 2018. But at that time, he was swamped and also mentoring a couple of Haskell.org project in GSoC. So finally I choose to become a mentor myself too because I was a mentee in the Haskell Summer of Code in 2017. The reason behind the intention to refactor the entire codebase is because I am not too confident to be able to maintain the frontend code due to my lack of Purescript knowledge. And also, it seems that this project will attract more Haskell contributor than the Purescript one if we use Haskell for the frontend. This reason is supported by the fact that Purescript has breaking changes from 0.11 to 0.12, and I become more firm that I need to actualize my full-stack Haskell idea in Hackage Matrix Builder. It turns out Herbert already started refactoring the UI (version 3) using Reflex-Dom as a proof of concept while I was busy doing my final bachelor degree project.


Lesson Learned

Tools and Operating System for Haskell Development

There are many tools that newcomer feel confused about when trying to start their journey into Haskell programming. I have tried several tools like Stack, Cabal, and Nix for doing my pet project, and I like using Nix the most. However, since Nix is another learning curve layer, and this project aims to use a simple, yet powerful tool, so I picked Cabal for doing the development. Cabal, with its Nix-style local builds (aka v2-build or new-build), is excellent nowadays and even better when we can add a non-Hackage package in our project. I remember using Stack when I started a Haskell reading group in my university, and it did not end well for them.

Using Windows 10 as my primary operating system makes developing Haskell projects more challenging. Even though we have the Haskell-Platform and Chocolatey to make the development more comfortable, Windows Subsystem Linux (WSL) is my way to go to help me diving into this project. Moreover, Herbert’s PPA for Ubuntu 18.04 WSL more than enough for Hackage Matrix Builder project because we only need to install GHC and GHCJS, which obtained by using his PPA.

For code editor, many Haskellers prefer Emacs or Vim. While I like both Emacs and VSCode, I prefer VSCode integrations with Remote plugin so I can easily access WSL folder from my current Windows 10 desktop. And also, VSCode is relatively easy to understand for someone who does not have a Computer Science degree.

Hackage, GitHub Source Code, or Community for a Better Reference

Hackage is the only thing that I can rely on when trying to understand some Haskell library. However, some library maintainer (I don’t know what are their reasons) forget to keep their library updated even though we have hackage-trustee. I always search for some function by appending Hackage word in Google search engine like “fromMaybe hackage” for better result. If you are new to Haskell, try to familiarize yourself with Hackage for your survival Haskell kit.

The second best thing for your reference is GitHub where many of Haskell library maintainer put their code. It is harder to read, but you will get used to it eventually. You can search for some functions in each of Haskell repository or use an advanced search to see how such functions are implemented in the code. In addition, since we can add GitHub repository in our cabal.project and build it using Cabal v2-build, it will make Haskell development easier.

The last thing but also important is finding reference through Haskell community. We have our discourse forum server which is really useful for me because having different timezone can make you missed some important conversation between Haskell developer. We also have other Haskell server out there like in Slack or Discord. But for me, I like Haskell discord server the most. The Haskell Discord server is more focused on Haskell than Functional Programming in general and it has an unlimited limitation for chat history while Slack limited its chat history for open source community for only 10,000 messages.

Haskell is a Strange Beast, but Reflex/Reflex-Dom is a Legendary Creature

I am not saying that both Haskell and FRP Reflex/Reflex-Dom are not tameable, but they are just hard to tame, and sometimes bite you very hard until you stop using them. And if you face that kind of situation, the only advice that I can give is to go as quickly as possible into the glorious #reflex-frp tavern (IRC channel on freenode), the place where many Reflex contributors discuss the library.

Haskell is actually for everyone, and I know that because some Haskellers who do not have a Ph.D. can still work with this language. However, if you are like me (a non-English speaker, a lawyer or any other social degree, living in a country when every job posts are about PHP language) and considering to learn Haskell or Reflex-Dom, I hope the lesson I get during GSoC below can aid your journey into programming Haskell, especially using the Reflex/Reflex-Dom Library.

1. Keeping Git PRs more focused

This lesson may be unrelated to Haskell or Reflex/Reflex-Dom directly, but I must confess that keeping your PR from some code refactoring that irrelevant to what you are fixing will help you being thorough on what you are doing with your project. It also can save you later. Also, participating in open source development is not only about what you achieved, but also what benefits other developers can get from your work.

One thing that I always do during my first month of GSoC project is changing some code or fixing it, but this change or fix is not related to my feature git branch. Herbert always reminds me that any unrelated code refactoring should be in separate PR from my feature git branch PR to make it easier for code reviewer.

2. Reflex, Reflex-Dom Quick Reference, and Blog Tutorial are your dictionary

The only thing that matter for you in your first attempt to the Reflex-Dom world is by continually reading its quick reference. I always read it back and forth even when creating a simple static page like in this homepage PR, or when I face a complex HTML structure in my application. However, I felt lost when I read the apostrophe function like el' :: Text -> m a -> m (El, a), until I refer to some GitHub example on how to use it. Reflex quick reference act the same as Reflex-Dom quick reference but more about low-level FRP systems like the function needed to create Event, Behavior, and Dynamic.

When I first learn Reflex and Reflex-Dom, Only quick reference that I have. Then when I am working on my third-year project built with Reflex, there is reflex-dom-inbit which explained a lot about how Reflex-Dom works. After that, Queensland Functional Programming Lab posted a blog post series about Reflex written by Dave Laing. For me, that is the most recent Reflex’s tutorial that very helpful.

3. Function “el” and “elAttr” are a simple HTML tag

Another thing that I noticed is el and elAttr is just an HTML element that can be easily understood like in the below code (see further in the above-linked PR):

Both [2] and [C]are the content of our HTML tag element that can be nested like ul and li In the following code. You do not need to confuse about [3] and [D] because it is only a function return type.

4. PreventDefault works for certain cases

Reflex-Dom has already a built-in function called button :: Text -> m (Event ()) that will become <button> HTML element. However, there are certain conditions that you may want the default action of the event is not triggered. For example, Matrix Hackage Builder has “add tag” button for each package’s page so that Trustee can add specific tag they want. The code for that is:

When I first wrote the code, I use Reflex-Dom’s built-in functionbutton and when I run the application, every time I click the “Add Tag” button, the URL in the address bar change from ng/#/package/a50 into ng/?#/package/a50 . This behavior happened because the default action after clicking the button is calling the API backend to insert package’s tag. So, Ben remind me about this preventDefault function so such default action will not be triggered by clicking the button. We do that in Reflex by creating our button (I called it button_) like below:

Confused? Yeah, that happened to me also when I learned Reflex/Reflex-Dom the first time to make a mini-thesis by building a simple AWS calculator using it and WkWebView. Even after that, I still need to get myself together when reading it. Well, let me explain it as clear as I can.

This button function takes one argument “t” which later this “t” will be inside the HTML element like this <button> t will be here </button>. The cfg variable is a configuration that will be attached to the element function. Thus, we put a preventDefault flag to the element’s config created by the element function which has the type:

element :: Text 
-> ElementConfig er t (DomBuilderSpace m)
-> m a
-> m (Element er (DomBuilderSpace m) t, a)

Unless you want to become Reflex/Reflex-Dom developer, change the (Element er (DomBuiderSpace m) t, a) into (e, a) or more specifically (e, _) when I used in the code.

Once we get the element that already configured by preventDefault, then we can attach that element into click event as we use in this domEvent Click e function in the above “button.hs”.

While preventDefault will saves you from triggering the default action of the event happened, and sometimes we need that default action to occur for a specific activity. I learned this when I am working on Hackage Matrix Builder search box. The behavior of the search box is when a user clicks the search result, it will automatically change the current user page into the package’s page. In this case, using preventDefault will prevent the page changes.

5. Function “foldDyn” and “appEndo” are awesome

Please bear with me as I try to explain it as simple as possible. What I learned with foldDyn, and appEndo are very enlightening. The use case is still around adding tag feature to Hackage Matrix Builder.

If we pay attention to in line 3, we are supposed to fold the list of events and create a final result based on Map.union function. The initial value will be Map.empty, so we apply Map.union to Map.empty along with the rest of the event list. However, I noticed that we have two problems using this technique: a) Delete tag is working, but still need to refresh to make the tag disappear; b) The “Add Tag” button is not working, but if I take out addTag0 from the tagsMapDyn, “Add Tag” button works but still need to refresh to make the new tag appear. Ben told me that I cannot ever delete the tag if I only have Map.union as my reducer

He then showed me the following GHCi about Endo and explained why in His reflex-realworld project, He used Endo for the same case that I have.

It seems my first approach to use foldDyn was wrong, and if you break down how the function that “Delete Tag” button behaves, you can see the step as follows:

  • evMapTags fires with the stuff from the backend on page load. We union those tags to Map.empty. The result is currently the full map (let's say “A”, ”B”, and ”C”)
  • The user clicks on the “Delete Tag” button of “B”. That calls the backend, and that button fires an event out of the list view.
  • delete0 fires with a value of Map.fromList [(“B”,”B”)]
  • This process flows into our foldDyn. We union Map.fromList [(“A”,”A”),(“B”,”B”),(“C”,”C”)] to Map.fromList [(“B”,”B”)]
  • Nothing changes so I have to reload the page

So what Ben explained to me about using Endo is that the events that we push into foldDyn be more like “delete X” (Endo . Map.delete) "add Y" (Endo . Map.insert) "replace the whole thing with these" (Endo . const). So the tagsMapDyn become:

This Endo thing is new to me, and I learned a lot about it. I have to admit that I still do not understand fully about how or when Endo will be my savior while I am struggling to tame the FRP and Reflex beasts.

6. Expanding out your type API declaration for client functions

On Hackage Matrix Builder project, we use a library called servant-reflex to share the API between the server and frontend. We need this library to synchronize our backend which is written in servant combinators, and so the API endpoints are available in reflex’s FRP semantics. You can see my work on this PR. Robert gives us an alternative package to work with Servant and Reflex-Dom by pointing to servant-client-jsaddle. Discussions are going there, but it will be a one big and long post. This package is still under observation by us, though.

The plan on expanding out type API declaration comes from my issues when I have to put ClientFuns{..} = mkClientFuns burl on the where clause for every function which makes an API calls. The burl itself is only a BaseUrl where the API backend located (in this case is in matrix.hackage.haskell.org:443/api). So, Herbert’s primary goal is that all client functions are defined, instantiated, and exported in the API.hs module. Thus, I look into Ben’s reflex-realworld project then start to porting the code into Hackage Matrix Builder repository. While porting the code, Herbert reminds me that Hackage Matrix Builder is not as big as reflex-realworld that I have to make the code more readable and concise. He then gives me some API client example written in the backend.

The following code is the final version of my work on expanding out type API declaration. I use theclientWithOpts function for a basic authentication that will pop up every time Trustee adding a tag or starting some package’s queue.

Conclusion

Up until GSoC first evaluation, I am pleased to participate as a Google Summer of Code student for Haskell.org (although I missed Haskell Summer of Code). Haskell is a beautiful language to learn especially for frontend web development using Reflex/Reflex-Dom or FRP approach.

Furthermore, Reflex-Dom is a tameable creature for every frontend developer who wants to try a new language and learn about FRP. It is very straightforward if you are already familiar with Haskell. It will be more challenging for those who do not know some fundamental concept in Haskell (e.g. Functor, Applicative, and Monad). Also, it is harder for someone who already hates Haskell, GHCJS, or Reflex-Dom itself.

What is Next?

The next post will about my work on routing and fixing the matrix table for each package. So, Stay tuned!

Thanks to Herbert Valerio Riedel

Andika D Riyandi (Rizary)

Written by

I am an ex-Licensed Lawyer who try to be Functional Programmer

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade