Stepping up Our Test Fixture Game With Fishery
As we gear ourselves up to eliminate security deposits from rental properties across the country, we are always looking for areas of improvement in our developer tooling in anticipation of a growing team.
In a previous blog post, we explained how we significantly improved our test suite runtime. Since then, we tackled another weak point of our test tooling: fixtures.
Fixtures don’t need to be artisanal
The core issue with our fixture setup is that we have to hand-craft them. Our fixtures are JSON files that we use to seed the database for our integration tests. In an effort to keep our fixtures directory somewhat reasonable, we divide it into a subdirectory per database table, where every file within contains only that type of database object. Take for example a books & authors schema. We’d have a subdirectory for the books table, and another for the authors table:
fixtures/
├─ books/
│ ├─ some_special_case.json
│ ├─ default.json
├─ authors/
│ ├─ some_special_case.json
│ ├─ default.jsonEach JSON file contains one or more database objects that are used in a specific test case. As you can imagine, in a real-world scenario this gets unwieldy very quickly as we augment our database schema or even as we add new test cases.
There are many a reason why this setup is less than ideal, but the main two are:
- The fixtures are shared between tests. As you introduce new business logic, you need varying sets of seed data to back your tests. The debate becomes: do you create brand new sets of fixtures to accommodate these variations, or do you seed shared defaults but update the data during test setup? Either choice is unpalatable; one requires a ton of work, the other introduces a single point of failure for multiple tests.
- Object relationship management is tedious. We don’t use an ORM library here at LeaseLock, which means one can only set up test fixtures if they know the dependency tree of the object they’re trying to create. Unfortunately, our schema isn’t as simple as “an author must exist before a book can be created”, but even in a trivial case like that, having to manually set up object relationships with hardcoded foreign keys is less than ideal.
Dynamic fixtures: an oxymoron
If you’re familiar with Ruby on Rails and its testing ecosystem, then chances are you’re also familiar with its plug-and-play Ruby gem for fixture factories, factory_bot.
Fixture factories do exactly what their name implies: you define a factory class or function for your database object, and when called, it returns an instance of that object — typically with sensible, faked default values. This allows you to create object instances on-demand for a test case.
Thanks to the amazing folks at thoughtbot — the same people who built factory_bot for Ruby — we are able to do just that with our Node.js stack, using the fishery library.
Fishery is perfect for our needs. It’s TypeScript-friendly, doesn’t require integrating with an ORM to fully extract its usefulness, and is an overall no-frills factory library, yet contains some sweet features like factory extension and build hooks — more on that below!
Our Setup
Since we don’t use an ORM, we’ve been using TypeScript to type the objects we get from/send to our database interface knex. Continuing with the books and authors example, we’ve got the following:
Using these interfaces, we define our factories with sane defaults powered by the faker.js library:
Great! We’ve got factories set up for our database objects. We now have the ability to provide realistic mock data to our unit tests by simply calling .build(), and if we need to override any default values to suit a specific test case, we can always pass parameters into the build function: BookFactory.build({ name: ‘The Pragmatic Programmer’, genre: 'Software' }). That’s a big win already!
As for integration tests, we leverage the onCreate callback to automatically insert the built object into the database when .create() is called on the factory. .create() has the same API as .build(), with the only difference being that .create() fires the onCreate callback after building the object. This is only the first step in seeding test data, as we have to have a mechanism for, among other needs, seeding complex object trees.
While fishery ships with support for associations, we opted to keep our factories simple and instead implement descriptively named test helper functions to encapsulate seeding data for common scenarios involving deeply-nested object relationships.
In our Mocha-powered test suite, we simply use these helper functions in a before hook, and use references to the seeded data to help make assertions about the function under test:
This pattern has served us well by allowing us to easily seed rudimentary sets of data for our integration tests and end-to-end tests. In practice, our object relationship trees are considerably more complex, so our test setup usually involves composing multiple basic helpers to seed object trees. Nevertheless, the Gists above capture the spirit of how we create and use these helpers.
More often than not, the seeded data has to have specific values, rather than faker-generated defaults, to get the test database in the right conditions for our tests. For example, a function may only process books with genre = 'Fantasy', so it wouldn’t be unreasonable to want to only seed books with that genre.
There are a number of ways to achieve this. The straightforward solution is to update the fixtures after they have been seeded with random values. However, we want to avoid extra round trips to the database where possible — we don’t want to bog down our test suite runtime with unnecessary I/O!
The prevailing pattern in our test suite is the parameterization of our helpers, leveraging Fishery’s ability to easily override factory defaults. Consider the previous helper example createBooksForAuthor but augmented to allow overrides:
By parameterizing the helpers, seeding data to test our hypothetical fantasy-book-function is as simple as: await createBooksForAuthor({ numBooks: 5, bookParams: { genre: ‘Fantasy’ }})!
Another option is to extend the base factories. This is useful in situations where, in order for an object to be in a certain “state”, it has to have specific default values for one or more of its fields. This is not as common a pattern as parameterized helpers, but they don’t have to be mutually exclusive — we keep this in our toolbelt for sure!
The best pull requests are the ones with a lot of deletions
Introducing Fishery to our project was a natural progression of our developer toolset here at LeaseLock. The patterns we have in place are a great start — definitely miles ahead of our legacy static fixture setup — and we’ll certainly leverage more of Fishery’s features as we familiarize ourselves better with the library and identify more recurring patterns in our tests.
Since its addition, we’ve been able to delete multiple sets of static fixtures after migrating a significant number of our test setup procedures to use factory-generated fixtures. Removing the remaining static fixtures is a matter of clean-up — what’s important is that going forward, new tests can leverage the existing factories, and if one doesn’t exist, a factory can be created with minimal effort.
With our team’s emphasis on testing, these fixture factories will surely make our team’s life easier as we engineer our leading insurtech platform to help the world find home.
If you’re interested in tackling problems like this with us, visit our careers page for information about available roles. If you don’t see the role you’re looking for, you can also reach out to us directly at talent@leaselock.com.
