Testing filesystem in Node.js: The easy way
When writing applications in Node, you often need to write or read the contents of a file. Node.js provides a
fs library dedicated to this purpose. But how do we deal with the filesystem when testing our code?
I’m going to present two approaches to this problem – by mocking individual filesystem methods and by using virtual file in-memory file system. I’m also going to explain why the latter is a much better choice than the former.
Mocking individual methods
The simplest solution is to directly mock individual methods of the
fs module. Let’s assume we have a method that reads the first 5 bytes from a file and checks if its contents start with correct header, for example ”hello”:
We could, of course, use
fs.readFile()and read whole file contents, but it’s really a bad idea when we deal with large files.
We can pretty easily test the method by mocking
fs/promises library for this purpose:
Let’s rewrite tests to use async
Ouch, our mock function has grown and became much less readable. We wanted to mock
read() function, but to achieve correct results we also had to mock
close(). And this is just a simple scenario!
The above method may be sufficient for basic cases, but it becomes very verbose and error-prone for more advanced scenarios. It’s also implementation dependant. To overcome these problems, we should change our testing approach. Instead of mocking individual fs functions, we can mock the whole filesystem.
Using virtual filesystem as mock
In this approach, we replace the real filesystem with an in-memory one. There are a few libraries solving this problem, a popular one is
mock-fs, but I’d like to give
memfs a try. This is a simple, but powerful and well-documented library for managing virtual volumes. It reimplements
fs API one-to-one, so it’s perfect to use it as a mock. Moreover, it’s implementation agnostic — it works well with plain
fs/promises as well as other libraries like
Let’s rework our tests to use the virtual filesystem. Firstly, we need to add
yarn add -D memfs
# or using npm
npm i --save-dev memfs
We also should add manual mocks for
fs/promises modules. Create a
__mocks__ directory in our project root (or use
jest.config.js to configure it as you wish) and create a
fs.js file there:
If you’re using
fs/promises API, you should also create
Now let’s update our test to use our in-memory filesystem mocks:
Our test became very simple and straightforward. We write file contents like it was a real file and then pass its path to the tested function. And we don’t have to care about filesystem implementation used in
It’s a good practice to reset the virtual volume before each test — it avoids tests interfering with each other if more than one test manipulates on the same filesystem path.
We can see that both
fs/promises are mocked. That’s because
fs/promises in its implementation, but for demonstrational purposes I used
fs-extra in tests. In real life, you should stick to one of them everywhere (I personally prefer
Creating directory structure from JSON
Mocking files instead of methods is very convenient, but there’s one drawback of the method shown above. Imagine your application operates not on a single file, but on a whole directory structure. Creating it manually using plenty of
fs.writeFile may be cumbersome. Fortunately,
memfs has another useful feature: directory structure created from a JSON object. Its keys are paths and values refer to file contents. Let’s have a look at the example:
Now we can create even complicated directory structures using single
vol.fromJSON() call. We can even split it into multiple calls and extract to helper functions — it may be useful in more sophisticated test suites. Or even mix
fs methods — they will not overwrite each other (unless modifying the exact same path) until
vol.reset() is called.
Mixing real and virtual filesystems
There’s one more thing worth mentioning. You may be using the in-memory filesystem, but need to reach some real files — for example some test fixtures, which are not easy to prepare using
fs calls. There’s a library called
unionfs which lets you join both filesystems. You may find documentation and examples on both libraries sites:
The virtual filesystem is definitely a thing that is worth using when testing any filesystem-touching code. It’s much more flexible and straightforward approach than mocking individual methods.
This article is just a brief introduction to this approach. Please refer to the libraries’ official docs for more details.
All code examples from this article are available on my GitHub.