Testing filesystem in Node.js: The easy way

Bartłomiej Klocek
Apr 3 · 5 min read

When writing applications in Node, you often need to write or read contents of a file. Node.js provides a fs library dedicated for this purpose. But how do we deal with filesystem when testing our code?

I’m going to present two approaches to this problem – by mocking individual filesystem methods and by using virtual file in-memory file system. I’m also going to explain why the latter is much better choice than the former.

Mocking individual methods

The simplest solution is to directly mock individual methods of fs module. Let’s assume we have a method that reads first 5 bytes from file and check if its contents start with correct header, for example ”hello”:

We could, of course, use fs.readFile() and read whole file contents, but it’s really a bad idea when we deal with large files.

We can pretty easily test the method by mocking fs.readSync():

The test seems to be pretty simple, but our tested function has one drawback — its implementation uses blocking calls, which block whole JavaScript thread when performing filesystem I/O operations. Let’s rework our method to use asynchronous, non blocking calls. Node.js comes with fs/promises library for this purpose:

Let’s rewrite tests to use async fs/Promises API:

Ouch, our mock function has grown and became much less readable. We wanted to mock read() function, but to achieve correct results we also had to mock open() and close(). And this is just a simple scenario!

Above method may be sufficient for basic cases, but it becomes very verbose and error-prone for more advanced scenarios. It’s also implementation dependant. To overcome these problems, we should change our testing approach. Instead of mocking individual fs functions, we can mock the whole filesystem.

Using virtual filesystem as mock

In this approach we replace real filesystem with in-memory one. There are a few libraries solving this problem, a popular one is mock-fs, but I’d like to give memfs a try. This is a simple, but powerful and well documented library for managing virtual volumes. It reimplements fs API one to one, so it’s perfect to use it as mock. Moreover, it’s implementation agnostic — it works well with plain fs, fs/promises as well as other libraries likefs-extra.

Let’s rework our tests to use virtual filesystem. Firstly, we need to add memfs dependency:

yarn add -D memfs
# or using npm
npm i --save-dev memfs

We also should add manual mocks for fs and fs/promises modules. Create a __mocks__ directory in our project root (or use jest.config.js to configure it as you wish) and create a fs.js file there:

If you’re using fs/promises API, you should also create __mocks__/fs/promises.js file:

Now let’s update our test to use our in-memory filesystem mocks:

Our test became very simple and straightforward. We write file contents like it was a real file and then pass its path to the tested function. And we don’t have to care about filesystem implementation used in checkHelloAsync.

It’s a good practice to reset the virtual volume before each test — it avoids tests interfering each other if more than one test manipulate on the same filesystem path.

We can see that bothfs and fs/promises are mocked. That’s because checkHelloAsync() uses fs/promises in its implementation, but for demonstrational purposes I used fs-extra in tests. In real life you should stick to one of them everywhere (I personally prefer fs-extra).

Creating directory structure from JSON

Mocking files instead of methods is very convenient, but there’s one drawback of the method shown above. Imagine your application operates not on single file, but on a whole directory structure. Creating it manually using plenty of fs.mkdirp and fs.writeFile may be tedious. Fortunately, memfs has another useful feature: directory structure created from JSON object. Its keys are paths and values refer to file contents. Let’s have a look at the example:

Now we can create even complicated directory structures using single vol.fromJSON() call. We can even split it into multiple calls and extract to helper functions — it may be useful in more sophisticated test suites. Or even mix vol.fromJSON with fs methods — they will not overwrite each other (unless modifying the exact same path) until vol.reset() is called.

Mixing real and virtual filesystems

There’s one more thing worth mentioning. You may be in-memory filesystem, but need to reach some real files — for example some test fixtures, which are not easy to prepare using fs calls. There’s a library called unionfs which lets you join both filesystems. You may find documentation and examples on both libraries sites:

Conclusion

Virtual filesystem is definitely a thing which is worth using when testing any filesystem-touching code. It’s much more flexible and straightforward approach than mocking individual methods.

This article is just a brief introduction to this approach. Please refer to the libraries’ official docs for more details.

All code examples from this article are available at my GitHub.

Nerd For Tech

From Confusion to Clarification

Nerd For Tech

NFT is an Educational Media House. Our mission is to bring the invaluable knowledge and experiences of experts from all over the world to the novice. To stay up to date on other topics, follow us on LinkedIn. https://www.linkedin.com/company/nerdfortech

Bartłomiej Klocek

Written by

Enthusiast of electronics and all kinds of software development — from web apps to embedded systems.

Nerd For Tech

NFT is an Educational Media House. Our mission is to bring the invaluable knowledge and experiences of experts from all over the world to the novice. To stay up to date on other topics, follow us on LinkedIn. https://www.linkedin.com/company/nerdfortech

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store