The Curious Case of Circular Dependencies in Node

Reasoning with what happens and how to solve for it

Photo by Alexandre Mertens from Pexels

Problem

Consider the scenario of a simple registration module with two dependent services: one (let’s call it the Document Service) has a generic function to store a copy of a file into our storage backend (say Amazon S3), and the other (let’s call it the Registration service) has a function to process a registration. Registration itself requires photo to be submitted in order to complete, hence it will require the usage of Document service.

During the development, a new feature is introduced, which is an identity card parser. If an identity card image is uploaded, the module will parse the image with OCR technology & automatically register an account in the system.

Assume that we have the project structure looks like this:

+ clients
  - s3-client.js
+ models
  - registration-model.js
+ services
  - document-service.js
  - ocr-service.js
  - registration-service.js

Document service implementation.

// /services/document-service.js
const ocrService = require('./ocr-service');
const registrationService = require('./registration-service');
const s3Client = require('../clients/s3-client');
/**
 * @param {string} identifier
 * @param {string} type
 * @param {string} blob
 * @result {string}
 */
async function store(identifier, type, blob) {
  let localURL;
  try {
    localURL = await s3Client.save(
      identifier,
      type,
      blob,
    );
  } catch (error) {
    throw error; 
  }
  return localURL;
}
/**
 * @param {string} blob
 * @return {boolean}
 */
async function storeIdentityCard(blob) {
  let localURL;
  try {
    const data = ocrService.parseIdentityCard(blob);
    localURL = await store(
      data.icNumber,
      'IDENTITY_CARD',
      blob,
    );
    // [MARK1] PAY ATTENTION TO THIS
    await registrationService.register(
      {
        fullname: data.fullname,
        icNumber: data.icNumber,
      },
      data.photoBLOB,
    );
} catch (error) {
    throw error; 
  }
  return localURL;
}
module.exports = {
  store,
  storeIdentityCard,
};

Registration service implementation.

// /services/registration.service.js
const documentService = require('./document-service);
const registrationModel = require('../models/registration-model');
/**
 * @param {object} data
 * @param {string} data.fullname
 * @param {string} data.icNumber
 * @param {string} photoBLOB
 * @return {boolean}
 */
async function register(data, photoBLOB) {
  try {
    await registrationModel.create(data);
    // [MARK2] PAY ATTENTION TO THIS
    await documentService.store(
      data.icNumber,
      {
        type: 'PHOTO',
        url: photoURL,
      },
    );
  } catch (error) {
    throw error;
  }
  return true;
}
module.exports = {
  register,
};

When the storeIdentityCard() method is called, it fails with the following error:

documentService.store is not a function

Why This Happens

If we closely look at document-service.js and registration-service.js, they’re both referencing each other causing a circular dependency between them.

Circular dependency illustration

We can explain the control flow this way:

1. document-service.js is loaded: During the loading process, several modules are imported. One of them is registration-service.js, which then forces the engine to load this module.
2. registration-service.js is loaded: During the loading process, several modules are imported. One of them is again the document-service.js. At this point, the engine uses the as-yet unfilled exports (think {}) of the document-service.js module here.
3. registration-service.js is fully loaded.
4. document-service.js is fully loaded.
5. storeIdentityCard() of document-service.js is called. By the time registrationService.register() (see [MARK1] tag) is called, it will throw documentService.store is not a function (see [MARK2] tag). This happens because documentService is an empty object.

Solution

A circular dependency is not necessarily a bad thing and may be useful in some cases. However, if possible, it is best to avoid it as it cause bugs that are hard to reason with.

We found two ways to deal this problem:

Code Refactoring

Circular dependency is usually the result of improper code structure/design. Code refactoring is the best way to take care of the circular dependency issue. For our problem above, we can consider refactoring the code by introducing an intermediary service (let’s call it the Document Automation Service) that implements the storeIdentityCard() function.

Imagine this new project structure.

+ clients
  - s3-client.js
+ models
  - registration-model.js
+ services
  - document-automation-service.js
  - document-service.js
  - ocr-service.js
  - registration-service.js

We will move the storeIdentityCard() from Document Service into the Document Automation Service as shown below:

// /services/document-automation-service.js
const documentService = require('./document-service');
const ocrService = require('./ocr-service');
const registrationService = require('./registration-service');
/**
 * @param {string} blob
 * @return {boolean}
 */
async function storeIdentityCard(blob) {
  let localURL;
  try {
    const data = ocrService.parseIdentityCard(blob);
    localURL = await documentService.store(
      data.icNumber,
      'IDENTITY_CARD',
      blob,
    );
    await registrationService.register(
      {
        fullname: data.fullname,
        icNumber: data.icNumber,
      },
      data.photoBLOB,
    );
} catch (error) {
    throw error; 
  }
  return localURL;
}
module.exports = {
  storeIdentityCard,
};

Illustration of services’ dependencies after refactoring.

With the new intermediary, we have successfully managed to get rid of the unwanted circular dependency as illustrated above.

Extend exports instead of re-assigning

If a circular dependency is inevitable and introducing an intermediary is not an option, we can consider changing the way we export methods from modules to accommodate for circular references.

With the same project structure as defined in the problem section, we can make a small change to Document service and Registration service as shown below:

Document Service new implementation:

const ocrService = require('./ocr-service');
const registrationService = require('./registration-service');
const s3Client = require('../clients/s3-client');
...
Object.assign(
  module.exports,
  {
    store,
    storeIdentityCard,
  },
);

Registration Service new implementation:

const documentService = require('./document-service);
const registrationModel = require('../models/registration-model');
...
Object.assign(
  module.exports,
  {
    register,
  },
};

If we set exports through a normal assignment (equals = assignment), its reference, that could have already been made available to dependent modules, gets overridden. This is generally not a problem if there are no circular dependencies between our modules. However, when there are, it is. Retaining the original reference of exports passed to us by the module system allows us to lazily attach methods at a later point — which is what we want to do here. Object.assign permits us to extend the exports object as shown above instead of re-assigning it.

Circling back to our scenario here (see Why This Happens above), the reference to document-service.js that was made available to the registration-service.js while loading remained unchanged earlier even after document-service.js completed loading. This was because document-service.js ended up creating a new reference for its exports that its dependent module did not have access to causing the error we saw. However, by making document-service.js use Object.assign to extend its exports, we force it to retain the reference that its dependent module already has access to instead of creating another one.

So, an inexplicable bug in one of our micro-services forced me to look into how to handle circular CommonJS module dependencies a bit more closely eventually leading me to write this blog post. I found that inspecting the magic module object made available in any module by Node is a great way to reason with the behaviour that I was seeing. Hope you found this post helpful. If you can think of a legitimate need to have circular references between modules, do let us know in the Responses section below.

Credit to my colleague, Simon, for the Object.assign tip.