Functional core, imperative shell in Javascript
In this article I’m going to explore the concept of “functional core, imperative shell”.
Warning: I’ve by no means mastered this technique, so if you have any corrections or suggestions, I would be more than happy to hear it.
In essence, the goal with this approach is to be able to write as much functional code as possible, while moving the side-effecty imperative code to a “shell” around the functional code. By doing this, we enable better unit tests and more predictable code.
Before we look at how a program can be structured into a functional core and an imperative shell, let’s try to understand some of the differences between the two.
Functional programming
First of all, by writing as much functional code as possible we’re getting all the benefits of functional programming itself. I’ll assume you mostly know what functional programming is, but to recap: It’s all about pure functions.
A function is pure on two conditions:
1. The function always returns the same result given the same arguments
2. The function does not have any side-effects
For example, the following code is pure because it follows both rules:
const sum = (a, b) => {
return a + b
}This next bit of code is not pure, because it breaks rule number 1. Sometimes it returns another result (an exception in this case).
const sum = (a, b) => {
if(letTheUniverseDecide()){
throw new Error('The universe is sad today')
}
return a + b
}The following code is also not pure, because it breaks rule number 2. It always returns the same answer, but it’s not side effect free.
const sum = (a, b) => {
launchNukes()
return a + b
}An interesting observation about pure functions is that they can only call other pure functions. A function that calls an impure function must itself also be impure. In other words, impurities spread.
Predictability
The main benefit to functional programming in my opinion is predictability. A pure function always does the same thing. There are no errors to handle.
Unit testing
Another huge benefit of functional programming is unit testing.
When you test a pure function, all you have to care about are the inputs and outputs. There is no need to make mocks or stubs, and thus we avoid the need to know about internals in the unit under test. Actually, one of the goals of “functional core, imperative shell” is to avoid mocking all together.
Problems with imperative code
I should start by saying that imperative code is by no means bad. In fact, unless you go for another language like haskell or purescript, you will always need some imperative code, and even in those languages I dare say you are using imperative code, just in a more abstract way. So the goal here is not to completely remove the imperative code, but to keep it as light and simple as possible, and thereby minimize the problems it can cause.
Also, I should point out that the type of code I’m talking about here is not in the details (like for-loops, which are impure), but more on a high level. Think calls to file systems, API’s, databases, etc.
Let’s look at a few of the issues with imperative code.
Testing
One thing we already covered is testing. When unit testing an api call (typically with some logic around it), we need to use mocks or stubs.
This forces us to consider the internals of the unit under test, and makes the test brittle and hard to write. It would be nice if we could test all the program logic separate from the api call.
Error handling
Calls to external services always require error handling. It may be that this error handling is as simple as adding a log entry, but it’s still error handling.
Proper error handling also considers the difference between known failures and unexpected errors. For example, an API returning a HTTP 404 is probably a known error, and has to be dealt with accordingly.
A network failure on the other hand, is probably not expected, and usually has to be handled in a different way, for example by retrying the request.
All this error handling creates logic, and that logic is hard to test in isolation. Furthermore, errors often propagate through the system, making us consider error handling in more than just one place.
Dependency injection
I’ve written a few articles related to dependency injection, and why I think it’s a good thing, and I still do. However, dependency injection also makes code harder to understand by abstracting away the dependencies. This is a problem that is hard to eliminate completely, but which can be minimized by using an imperative shell.
The solution: Functional core, imperative shell
To reiterate, the idea of this approach is to use as much functional programming as possible, and to keep the imperative parts as a kind of coordination layer on top (in the “shell”).
You may wonder why the imperative layer has to be on top exactly, and not any other place. The reason is that pure functions cannot call impure functions. Therefore, if we place imperative code inside the “core”, the whole core can quickly become impure. So right off the bat, that rules out dependency injection in the core.
It’s still useful to use dependency injection in the shell though, and I definitely recommend that. The difference is that the shell will be much simpler. When each part of the shell becomes simpler, we reduce the need for unit testing as well. When there is hardly any logic to test, it becomes viable to run integration tests instead of unit tests. Another thing that happens by removing logic from the shell is that we no longer have to split it into smaller parts to make it testable and understandable. This, in turn, reduces the need for dependency injection and dependencies in general. Suddenly, what used to be 5 classes can now be just 1 class that’s just coordinating the functional core. That class will have fewer dependencies and will be much easier to read.
So this is all well and good, but how do we actually write code in this way? This can be quite the challenge, and in fact it is a good idea to treat it as a challenge. How much code can you squeeze out of the shell and into the core?
Here are a couple of ideas:
Extract functions
Simply begin by taking a bit of imperative code, and start extracting pure functions.
Example:
async function sendPayment(payment){
const [firstName, lastName] = payment.receiver.name.split(' ')
const apiPayment = {
amount: payment.amount,
receiver: {
firstName,
lastName
}
// ...
}
return sendToPaymentApi(apiPayment)
}By just extracting the pure part into a function, we get this easily testable code:
function paymentToApiPayment(payment){
const [firstName, lastName] = payment.receiver.name.split(' ')
const apiPayment = {
amount: payment.amount,
receiver: {
firstName,
lastName
}
// ...
}
}
async function sendPayment(payment){
return sendToPaymentApi(paymentToApiPayment(payment))
}Notice that the new version of sendPayment is not really interesting to unit test. It will probably be enough to just integration test it.
We can aslo extract error handling:
async function doJobAndHandleResponse(){
const status = await jobStatus()
if(status){
if(status.done){
return markAsComplete()
}else if(status.notDoable){
return markAsFailure()
}
}
const response = await doJob()
if(response.success){
return markAsComplete()
}else{
if(response.error.permanent){
return markAsFailure()
}else{
return markAsError()
}
}
}With error handling extracted (or flattened):
async function doJobAndHandleResponse(){
const result = await attemptJob()
switch(result){
case: 'success': return markAsComplete()
case: 'alreadyDone': return markAsComplete()
case: 'notDoable': return markAsFailure()
case: 'failure': return markAsFailure()
case: 'error': return markAsError()
}
}
async function attemptJob(){
const status = await jobStatus()
if(status){
return jobStatusResult(status)
}
const response = await doJob()
return jobResult(response)
}
const jobStatusResult = status => {
if(status.done){
return 'done'
}else if(status.notDoable){
return 'notDoable'
}
}
const jobResult = response => {
if(response.success){
return 'success'
}else{
if(response.error.permanent){
return 'failure'
}else{
return 'error'
}
}
}In this example, the benefit is maybe not that clear. We have more lines of code and more abstraction going on. At the same time, the error handling part is more out of the way, and is testable. It will depend heavily on the use case if this approach is worth it or not.
“Lift” imperative code as high up as it can go
This is easiest to show by example. Let’s say we have code that looks like this:
async function processPayment(payment){
// lots of stuff here
const currency = await getCurrencyRate(payment.ccyName, 'USD')
// lots of stuff here
}We can move towards the goal of an imperative shell by moving the impure call out, and taking the result as a parameter instead:
async function processPayment(payment, currency){
// lots of stuff here
// lots of stuff here
}Of course, now some other bit of code has to get the currency, and the function is maybe still not pure, but it’s a step in the right direction.
Organize your data processing into pipelines
This is an extended version of the previous trick, where you separate the different parts of a data processing pipeline into an actual pipeline. This is just good practice in general, but it also helps us out with the shell.
Example:
async function processPayments(){
const payments = await getPayments()
for(let payment of payments){
const ccy = await getCurrencyRate(payment.ccyName, 'USD')
payment.amount *= ccy
await sendPayment(payment)
}
}Now, we could do a simple function extraction here as well, but another approach is to separate the data-gathering part from the action-performing part.
The reason we might do this is because the function will be simpler and easier to test, but it will also be much easier to add error handling (which is completely absent in this example).
For example, what happens if one of the getCurrencyRate-calls fails? Since getCurrencyRate is an easily retryable function (while sendPayment is not), we needlessly complicate the error handling. We may end up with only half the payments sent just because the currency api call failed. If we instead separate the process into two stages, gathering stage and action stage, we can more easily handle errors from the gathering stage.
In the example below, I have separated the gather stage, and also extracted some pure functions for good measure.
async function processPayments(){
const payments = await gatherPaymentData()
for(let payment of payments){
await sendPayment(payment)
}
}
async function gatherPaymentData(){
const payments = await getPayments()
const currencyRates = await getCurrencyRates(payments)
return getPaymentsInOtherCurrency(payments, currencyRates)
}
async function getCurrencyRates(payments){
return Promise.all(payments.map(
payment => getCurrencyRate(payment.ccyName, 'USD')
))
}
const getPaymentsInOtherCurrency = (payments, currencies) => {
return zip(payments, currencies)
.map(([payment, ccy]) => convertToCurrency(payment, ccy))
}
const convertToCurrency = (payment, currencyRate) => {
return {
...payment,
amount: payment.amount * currencyRate
}
}
// Take each element of two arrays and put them together
const zip = (arr1, arr2) =>
arr1.map((element, index) => [element, arr2[index]])Again, this example has more code than we started with. We have to add error handling and more logic for the benefit to show.
Put the “core” in the right place
Sometimes, the functions we extract into the core may be small, reusable pieces of logic, while other times they may only be used by the calling code.
It’s difficult to name things, and even more difficult if those things should make sense on a “global” level. So if I have to come up with great names every time I extract functions, I’m more likely to just drop it. Therefore, I find it easier if I can limit the scope of the extracted functions.
One approach is to just keep the functions in the same file as the calling code. This works to an extent, but it can become a bit messy. Some of the examples in this article are already a bit messy, and kind of works against the idea of a simple shell.
Another approach is to create a secondary file that has the same name, but with a “.core.js” postfix. These are just examples though, you’ll probably come up with something better that fits your project.
How do I know if I should use this approach?
So far I’ve mostly covered positive things, but there are obviously negative things to say as well. The most obvious is that by extracting things we get more coupling and more abstraction. So like everything else, the approach has to make up for its own cost. Ask yourself this question: “Do I get something out of unit testing this piece of code, or am I just testing if my mocks work?” If you don’t have any logic that’s worth unit testing, you probably have an imperative shell already, and it’s probably better to leave it alone, and add integration tests instead. Many simple app’s fall into this category.
What about monads? We need monads to write functional code, right?
Well… Maybe (get it?). The thing about monads is that they kind of solve another problem. For example, a Maybe monad solves issues with multiple return types and how to handle those (replacing if/else). That’s to do with the details, and doesn’t do much for the overall architecture. Other monads such as Future may make an api calling function pure, but the call itself still has to be made in the imperative shell, and there’s not much difference between map’ing a monad and just doing the call, while in the shell. At least, I haven’t found that monads makes it easier to create a functional core, though I’m very open to suggestions here!