I built the first demo of Dark in Python, in about two weeks. A few months later when I started productizing it, I rebuilt it in OCaml. Back in 2017, when I was considering the language and platform to use for Dark, OCaml was extremely compelling:

  • it’s a high-level language with static types, so easy to make large scale changes as we figure out what the language/product was
  • you mostly model data with sum types, which in my mind are the best way to model data
  • it’s very similar to the language I wanted to build (in particular, we could reuse built-in immutable data structures for Dark’s…


tl;dr there’s a Dark v2 roadmap now.

Image for post
Image for post
John Rocque’s 1756 map of Dublin

There are a number of things missing for Dark to be able to have product market fit:

  • Dark lacks SDKs for 3rd party APIs
  • Dark lacks a good user account/authentication story

While these are the major issues, solving these requires some more fundamental changes. As well as simply making the package manager available, we also need to improve on the tools that developers have for building packages, everything from how packages coexist with other functions, the types available to packages, and how to support collaboration on those packages.

Similarly, there is a bunch of tooling needed before we can build a good user authentication story. Dark needs the ability to define HTTP middleware (we need to decompose the existing HTTP framework into middleware), as well as better types. Each of these changes then depends on further changes — you need a good language to have good APIs. …


Image for post
Image for post

I’ve been working recently on a benchmark, to try and see how to get the most performance from the Dark backend. I’ve reimplemented the core of Dark in various languages and web frameworks, notably in OCaml, F# and Rust.

As a reminder, Dark is a language and platform for building backend web services and APIs. Implementation-wise, it’s basically an interpreter hooked to a webserver and a DB. The language is a statically-typed functional, immutable language which is garbage collected.

Dark users can write arbitrary code that runs on our server, including making HTTP calls to slow and badly-behaving 3rd-party web servers. This means we need to efficiently support both computation and IO on the server. …


Image for post
Image for post
Dali’s The Persistance of Memory (melting clocks)

I was very recently the holder of three opinions:

  • Rust is magic fairy dust that can fix all my problems
  • Async is magic fairy dust that can fix all my problems
  • GraphQL is magic fairy dust that can fix all my problems

Unsurprisingly, all of these appear to be wrong. This post is about the first two.

In the journey to get to product/market fit with Dark, I’m taking a slightly different strategy than we took before, which I’m calling “Hard Things First”. Repeatedly with Dark we spent significant time coming up with hacks to work around previous hacks which themselves were working around previous hacks. That made for a codebase where people can only contribute is well-defined ways, as going outside the box was to invite madness. …


On Saturday, we had our third contributor meetup, this one was organized by Shahriyar Nasir (thanks Shah!). This time, we did mob coding to add a Tuple type to Dark, and we recorded the whole thing.

Image for post
Image for post

A tuple looks like this:

let x = (somevar, "a string", 567)

A tuple allows you to combine multiple values of different types. In the video, we added Tuples to the editor and frontend, including in particular the keystrokes to add, update, and delete tuples. Shah followed up later to add backend support. We hope to get this merged soon.

The video is about two hours, which is about how long it takes to add a language feature to the frontend (it probably takes a similar amount of time to add a feature to the backend). This was straightforward as Tuples have a very similar interaction model to Lists — hopefully other language improvements won’t take too much longer. …


Image for post
Image for post

I’m about two weeks into Rust now, so this feels like a good time to write a critique, before I get Stockholm Syndrome’d.

My main motivation in learning Rust is that I have to maintain some of Dark’s Rust code. There was a recent outage related to that code, and I had to learn on the fly, so better to actually know what I’m looking at.

I’ve also been dreaming of rewriting Dark in Rust for quite some time, largely due to frustrations with OCaml as well as some excellent marketing by the Rust community. …


Image for post
Image for post

There was a wonderful post just now that I saw on HN as I was procrastinating writing this post. It was by Tim Bray, and it covered my favorite topic, which is just how batshit cloud services have gotten, this time Traffic Director on GCP. I don’t have anything more to say about this that you haven’t heard before, blah blah blah, complexity bad, blah blah, simplicity good.

Which leads me quite nicely into some clarity I’ve gotten over the last few weeks. After reducing the team down to just me, I spent the last month doing a lot of thinking, really trying to get my head around where Dark is, what I should be doing, priorities, etc.

[Read more on blog.darklang.com]


Image for post
Image for post

I’m always interested in stories about Accidental Complexity — when you waste time dealing with the endless layers of tooling that’s required to make a web application today. I’m also super into using Observability (which I’ll call to ability to introspect live systems, though that’s not exactly right). Fortunately for these two hobbies, I caused a partial outage of one of our systems due to terrible Accidental Complexity, and solved it using some delightful Observability. It’s Story Time, gather round!

[continued on blog.darklang.com]


Image for post
Image for post

One of the things that makes Dark truly unique is what we call “Trace-driven development”. The best way to write a HTTP handler in Dark is to start by making a request to the non-existent handler, then:

  • using the 404s list to create the handler
  • using the actual trace value to see the output of your code as you type it.

[Continued on blog.darklang.com]


Image for post
Image for post

On Friday we had our first contributor meetup. Thirteen folks showed up to a zoom call to discuss how to contribute to Dark.

Obviously, the point of the conversation was to get to know each other. We’re all hanging out in the #contributors channel, but the room isn’t super chatty so talking in person (or whatever counts for in-person on Zoom) is helpful to get to know one another.

Big projects

One of the big topics was how to do bigger projects.

[Continued on blog.darklang.com]

About

Paul Biggar

Tech entrepreneur, software engineer. Founder of Dark: https://darklang.com. Founder of CircleCI. Lover of chocolate and pastries.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store