Automatic fake data with feathers-plus/cli

feathers-plus/cli can generate fake data, including foreign keys, from your JSON-schema models.

Feathers-plus/cli (cli+) generates JavaScript or TypeScript code for your application. However there isn’t very much you can do with that code because the services have no data.

cli+ skirts this issue by providing comprehensive features for generating fake data based on your existing JSON-schema models. This includes generating values for foreign key fields which refer to other generated records.

This article is part 1 in a set of publications dealing with testing in Feathers.

• Automatic fake data with feathers-plus/cli (this article)
• Automatically seeding data with feathers-plus/cli
• Automatic tests with feathers-plus/cli
• Use decision tables to write better tests faster.
• Automatic auditing of tests with feathers-plus/cli (to come)

Generating an app

Medium articles have already been published describing how to generate services with cli+, so we will skim through that process here.

Also generate services users and roles in the same way.

Faking fields

Now let’s fill in the JSON-schema model templates cli+ created for our 3 services.

Models for the services, in JSON-schema.

The fakeRecords, faker and chance props are used by the cli+ data faker. They describe the type of fake values it should generate.

It’s a good idea to describe your fields in as much detail as possible using, for example, the minLength and maxLength JSON-schema props shown above. Not only will cli+ then generate better record validation hooks for you, but the fake data will also respect those props.

In the above schemas, fakeRecords specifies the number of fake records to create for the service. The default, which you can change, is 5.

Most of the faker props are self-explanatory. There are about 150 different types of faker props available. They can be localized to about 35 locales so, for example, your fake data can be in Spanish or German.

The chance prop is also self-explanatory. There are over 100 chance props available.

You can extend the faker and chance props with your own custom fake value generators. The fk in faker: { fk: 'roles:random' } is just such a custom generator added by cli+. It extracts foreign keys from other services.

We can now generate our fake data with

feathers-plus generate fakes

and look at the fake data now stored in seeds/fake-data.js.

{
  "users": [
    {
      "email": "Dortha95@hotmail.com",
      "firstName": "Lafayette",
      "lastName": "Doyle",
      "roleId": "5b71c81d36cb5e0dd2d05db8",
      "password": "9bd9d8d69a674e0f467cc8796ed151a05dfc2ddf....",
      "_id": "5b71c81d36cb5e0dd2d05db2"
    },
    {
      "email": "Peggie.Farrell@yahoo.com",
      "firstName": "Jedediah",
      "lastName": "Baumbach",
      "roleId": "5b71c81d36cb5e0dd2d05dba",
      "password": "cb739205929396fea7596eb10faaa2352c595590....",
      "_id": "5b71c81d36cb5e0dd2d05db3"
    },
    {
      "email": "Sarah.Murazik@yahoo.com",
      "firstName": "Beth",
      "lastName": "Predovic",
      "roleId": "5b71c81d36cb5e0dd2d05dba",
      "password": "aa52c3f5ad019da1ffe78f097b0074f15471b5e6....",
      "_id": "5b71c81d36cb5e0dd2d05db4"
    },
    ...
  ],
  "roles": [
    {
      "name": "Legacy Marketing Analyst",
      "_id": "5b71c81d36cb5e0dd2d05db8"
    },
    {
      "name": "Future Brand Representative",
      "_id": "5b71c81d36cb5e0dd2d05db9"
    },
    {
      "name": "Customer Communications Manager",
      "_id": "5b71c81d36cb5e0dd2d05dba"
    },
    {
      "name": "Legacy Communications Architect",
      "_id": "5b71c81d36cb5e0dd2d05dbb"
    }
  ],
  "teams": [
    {
      "name": "sunt in consequat exercitation proident",
      "memberIds": [
        "5b71c81d36cb5e0dd2d05db2",
        "5b71c81d36cb5e0dd2d05db7",
        "5b71c81d36cb5e0dd2d05db6"
      ],
      "_id": "5b71c81d36cb5e0dd2d05dbc"
    },
    {
      "name": "quis exercitation ullamco tempor",
      "memberIds": [
        "5b71c81d36cb5e0dd2d05db2",
        "5b71c81d36cb5e0dd2d05dba"
      ],
      "_id": "5b71c81d36cb5e0dd2d05dbd"
    }
  ]
}

Our services were created using Mongoose adapters, so the keys you see are ObjectId() bson values. We would see positive increasing integers if the Sequelize, Knex or feathers-memory adapters were used. The proper key format is used for each service if your set of services use multiple adapters.

One more thing, cli+ will regenerate your app if you change a service’s adapter. Afterwards run generate fakes or generate all and the fake data will be regenerated properly.

Foreign keys

You might be happy to know that foreign key support is also built in. The

roleId: { type: 'ID', faker: { fk: 'roles:random' } }

we saw above populates roleId with the key from a randomly selected record from the roles service. The following syntax is supported:

• roles:random:fieldName — A random record is selected from the roles service. The value of its fieldName field is used. fieldName can be in dot notation, e.g. address.city.
• ‘roles:random’ — The value of the id or _id field is used.
• ‘roles’ — The same as ‘roles:random’.

Arrays of distinct foreign keys

The above syntax alone cannot handle several needs we may have. First, we need an array of distinct foreign keys for teams.memberIds.

memberIds: {
  type: 'array',
  maxItems: 10,
  items: [{ type: 'ID', faker: { fk: 'users:next' } }]
}

The memberIds field might contain duplicate foreign keys if we were to use faker: { fk: 'users' }.

The list of keys for the users service is shuffled at the start of generation for each teams’ record. The { fk: 'users:next' } will use the next one of these shuffled keys. Keys will start to be reused only once all existing keys have been used.

This ensures the foreign keys in memberIds are distinct.

Extracting multiple fields from a related record

We may need to copy the values of several fields from the same related record. For example we may need a result like

memberIds: [{ id, firstName }, { id, firstName }, ... ]

We could generate this using

memberIds: {
      type: 'array',
      maxItems: 40,
      items: [{
        type: 'object',
        properties: {
          id: { type: 'ID', faker: { fk: 'users:next' } },
          firstName: {
            type: 'string',
            faker: { fk: 'users:curr:firstName' } },
        },
      }]
    }

'users:next' selects the record containing the next shuffled key. Then 'users:curr:firstName' refers to this current shuffled record, that is, the same one which provided the key.

You can extract any number of fields from a related record using this technique

Expressions

Sometimes an expression has to be calculated in order to populate a field.

dayOfWeek: {
  type: 'integer',
  faker: { exp: 'new Date("December 25, 2018").getDay()'}
}

This exp is another custom prop defined by cli+. It calculates the expression and populates the field with the resulting value.

date: { faker: 'date.soon' },
dayOfWeek: {
  type: 'integer',
  faker: { exp: 'new Date(rec.date).getDay()' }
}

You can reference other fields in the record within the expression. rec is an object passed to the expression containing the record being populated. You can use it in any valid JavaScript expression.

The docs contain more information about exp. They explain, for example, how to pass your own custom objects and functions to the expression.

In summary

cli+ has comprehensive data faking capabilities which are easy to use. They tie in with the record validation hooks which cli+ automatically generates.

You can find more information about generate fakes in the docs. This includes, for example, information on how to customize the configuration for the data faker.

Next …

I’ve written a follow-up article on how cli+ can automatically seed your services with fake data.