Testing filesystem in Node.js: The easy way

When writing applications in Node, you often need to write or read the contents of a file. Node.js provides a fs library dedicated to this purpose. But how do we deal with the filesystem when testing our code?

I’m going to present two approaches to this problem – by mocking individual filesystem methods and by using virtual file in-memory file system. I’m also going to explain why the latter is a much better choice than the former.

Mocking individual methods

The simplest solution is to directly mock individual methods of the fs module. Let’s assume we have a method that reads the first 5 bytes from a file and checks if its contents start with correct header, for example ”hello”:

Tested function — using blocking calls

We could, of course, use fs.readFile() and read whole file contents, but it’s really a bad idea when we deal with large files.

We can pretty easily test the method by mocking fs.readSync():

Testing synchronous filesystem call

The test seems to be pretty simple, but our tested function has one drawback — its implementation uses blocking calls, which blocks the whole JavaScript thread when performing filesystem I/O operations. Let’s rework our method to use asynchronous, non-blocking calls. Node.js comes with fs/promises library for this purpose:

Reworked function to use asynchronous filesystem API

Let’s rewrite tests to use async fs/Promises API:

Testing asynchronous method dealing with filesystem

Ouch, our mock function has grown and became much less readable. We wanted to mock read() function, but to achieve correct results we also had to mock open() and close(). And this is just a simple scenario!

The above method may be sufficient for basic cases, but it becomes very verbose and error-prone for more advanced scenarios. It’s also implementation dependant. To overcome these problems, we should change our testing approach. Instead of mocking individual fs functions, we can mock the whole filesystem.

Using virtual filesystem as mock

In this approach, we replace the real filesystem with an in-memory one. There are a few libraries solving this problem, a popular one is mock-fs, but I’d like to give memfs a try. This is a simple, but powerful and well-documented library for managing virtual volumes. It reimplements fs API one-to-one, so it’s perfect to use it as a mock. Moreover, it’s implementation agnostic — it works well with plain fs, fs/promises as well as other libraries likefs-extra.

Let’s rework our tests to use the virtual filesystem. Firstly, we need to add memfs dependency:

yarn add -D memfs
# or using npm
npm i --save-dev memfs

We also should add manual mocks for fs and fs/promises modules. Create a __mocks__ directory in our project root (or use jest.config.js to configure it as you wish) and create a fs.js file there:

fs module mock

If you’re using fs/promises API, you should also create __mocks__/fs/promises.js file:

fs/promises module mock

Now let’s update our test to use our in-memory filesystem mocks:

Example test using memfs as virtual filesystem

Our test became very simple and straightforward. We write file contents like it was a real file and then pass its path to the tested function. And we don’t have to care about filesystem implementation used in checkHelloAsync.

It’s a good practice to reset the virtual volume before each test — it avoids tests interfering with each other if more than one test manipulates on the same filesystem path.

We can see that bothfs and fs/promises are mocked. That’s because checkHelloAsync() uses fs/promises in its implementation, but for demonstrational purposes I used fs-extra in tests. In real life, you should stick to one of them everywhere (I personally prefer fs-extra).

Creating directory structure from JSON

Mocking files instead of methods is very convenient, but there’s one drawback of the method shown above. Imagine your application operates not on a single file, but on a whole directory structure. Creating it manually using plenty of fs.mkdirp and fs.writeFile may be cumbersome. Fortunately, memfs has another useful feature: directory structure created from a JSON object. Its keys are paths and values refer to file contents. Let’s have a look at the example:

Using JSON to generate directory structure

Now we can create even complicated directory structures using single vol.fromJSON() call. We can even split it into multiple calls and extract to helper functions — it may be useful in more sophisticated test suites. Or even mix vol.fromJSON with fs methods — they will not overwrite each other (unless modifying the exact same path) until vol.reset() is called.

Mixing real and virtual filesystems

There’s one more thing worth mentioning. You may be using the in-memory filesystem, but need to reach some real files — for example some test fixtures, which are not easy to prepare using fs calls. There’s a library called unionfs which lets you join both filesystems. You may find documentation and examples on both libraries sites:

• memfs github page
• unionfs github page

Conclusion

The virtual filesystem is definitely a thing that is worth using when testing any filesystem-touching code. It’s much more flexible and straightforward approach than mocking individual methods.

This article is just a brief introduction to this approach. Please refer to the libraries’ official docs for more details.

All code examples from this article are available on my GitHub.