Nerd For Tech
Published in

Nerd For Tech

Testing filesystem in Node.js: The easy way

When writing applications in Node, you often need to write or read the contents of a file. Node.js provides a fs library dedicated to this purpose. But how do we deal with the filesystem when testing our code?

I’m going to present two approaches to this problem – by mocking individual filesystem methods and by using virtual file in-memory file system. I’m also going to explain why the latter is a much better choice than the former.

Mocking individual methods

The simplest solution is to directly mock individual methods of the fs module. Let’s assume we have a method that reads the first 5 bytes from a file and checks if its contents start with correct header, for example ”hello”:

Tested function — using blocking calls

We could, of course, use fs.readFile() and read whole file contents, but it’s really a bad idea when we deal with large files.

We can pretty easily test the method by mocking fs.readSync():

Testing synchronous filesystem call

The test seems to be pretty simple, but our tested function has one drawback — its implementation uses blocking calls, which blocks the whole JavaScript thread when performing filesystem I/O operations. Let’s rework our method to use asynchronous, non-blocking calls. Node.js comes with fs/promises library for this purpose:

Reworked function to use asynchronous filesystem API

Let’s rewrite tests to use async fs/Promises API:

Testing asynchronous method dealing with filesystem

Ouch, our mock function has grown and became much less readable. We wanted to mock read() function, but to achieve correct results we also had to mock open() and close(). And this is just a simple scenario!

The above method may be sufficient for basic cases, but it becomes very verbose and error-prone for more advanced scenarios. It’s also implementation dependant. To overcome these problems, we should change our testing approach. Instead of mocking individual fs functions, we can mock the whole filesystem.

Using virtual filesystem as mock

In this approach, we replace the real filesystem with an in-memory one. There are a few libraries solving this problem, a popular one is mock-fs, but I’d like to give memfs a try. This is a simple, but powerful and well-documented library for managing virtual volumes. It reimplements fs API one-to-one, so it’s perfect to use it as a mock. Moreover, it’s implementation agnostic — it works well with plain fs, fs/promises as well as other libraries likefs-extra.

Let’s rework our tests to use the virtual filesystem. Firstly, we need to add memfs dependency:

yarn add -D memfs
# or using npm
npm i --save-dev memfs

We also should add manual mocks for fs and fs/promises modules. Create a __mocks__ directory in our project root (or use jest.config.js to configure it as you wish) and create a fs.js file there:

fs module mock

If you’re using fs/promises API, you should also create __mocks__/fs/promises.js file:

fs/promises module mock

Now let’s update our test to use our in-memory filesystem mocks:

Example test using memfs as virtual filesystem

Our test became very simple and straightforward. We write file contents like it was a real file and then pass its path to the tested function. And we don’t have to care about filesystem implementation used in checkHelloAsync.

It’s a good practice to reset the virtual volume before each test — it avoids tests interfering with each other if more than one test manipulates on the same filesystem path.

We can see that bothfs and fs/promises are mocked. That’s because checkHelloAsync() uses fs/promises in its implementation, but for demonstrational purposes I used fs-extra in tests. In real life, you should stick to one of them everywhere (I personally prefer fs-extra).

Creating directory structure from JSON

Mocking files instead of methods is very convenient, but there’s one drawback of the method shown above. Imagine your application operates not on a single file, but on a whole directory structure. Creating it manually using plenty of fs.mkdirp and fs.writeFile may be cumbersome. Fortunately, memfs has another useful feature: directory structure created from a JSON object. Its keys are paths and values refer to file contents. Let’s have a look at the example:

Using JSON to generate directory structure

Now we can create even complicated directory structures using single vol.fromJSON() call. We can even split it into multiple calls and extract to helper functions — it may be useful in more sophisticated test suites. Or even mix vol.fromJSON with fs methods — they will not overwrite each other (unless modifying the exact same path) until vol.reset() is called.

Mixing real and virtual filesystems

There’s one more thing worth mentioning. You may be using the in-memory filesystem, but need to reach some real files — for example some test fixtures, which are not easy to prepare using fs calls. There’s a library called unionfs which lets you join both filesystems. You may find documentation and examples on both libraries sites:

Conclusion

The virtual filesystem is definitely a thing that is worth using when testing any filesystem-touching code. It’s much more flexible and straightforward approach than mocking individual methods.

This article is just a brief introduction to this approach. Please refer to the libraries’ official docs for more details.

All code examples from this article are available on my GitHub.

--

--

--

NFT is an Educational Media House. Our mission is to bring the invaluable knowledge and experiences of experts from all over the world to the novice. To know more about us, visit https://www.nerdfortech.org/.

Recommended from Medium

Day 30Training at Ryaz: Javascript

Javascript: Data Types and Structures

Short cut keys in kali linux.

Exploring Async Javascript

Learn React by Building an Expense Tracker App

Why I chose Javascript as my go to language

Want to get things done and ship? 3 reasons to choose Vue over React in 2018.

NodeAPI: in 10 mins TypeScript+GraphQL

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Bartłomiej Klocek

Bartłomiej Klocek

Enthusiast of electronics and all kinds of software development — from web apps to embedded systems. Expo open-source contributor at Software Mansion.

More from Medium

The Biggest Difference between a Mid-Level and a Senior Engineer

two women working

Deploy NodeJS and ReactJS production code on Windows server

How to Hire Node JS Developers: Under the Microscope

How to Hire Node JS Developers: Under the Microscope

Reactive Programming in frontend — Learn by building — Implement a Free Drawing with Canvas and…