Automatically seeding data with feathers-plus/cli

Feathers-plus/cli (cli+) generates JavaScript or TypeScript code for your application. However there isn’t very much you can initially do with that code because the services have no data. We need to seed those services with data.

This article is part 2 in a set of publications dealing with testing in Feathers.


A previous article describes how cli+ can generate fake data based on your existing JSON-schema models. Excitingly, this includes generating values for foreign key fields which refer to other fake records.

In this artcle we’ll look at how we can seed our services’ tables, on application startup, with this fake data.

At that point we’ll have generated code, we’ll have generated data, and we’ll be ready to rock.


Generating an app

Medium articles have already been published describing how to generate apps with cli+, so we will skim over much of that here, while focusing on what’s important for seeding data.

First we generate the app.

Data mutating tests and seeding may run when NODE_ENV is one of (optional)

This is one of the prompts above. A Node program runs in an environment set by the NODE_ENV environment variable. You may decide on using different environments such as the default one (no NODE_ENV specified), development, test, production, staging, and uat (user acceptance testing).

We want to be very careful not to accidentally overwrite our production or other records while seeding our fake data. So we enter here the comma-delimited environment names under which the services’ data may be seeded.

We’ve entered test here. The seeding will only run in this environment, skipping processing in other environments.

On a related topic, cli+ also generates tests. Those tests which may mutate service data also check NODE_ENV. They also skip processing if run in the “wrong” environment.

Seed service records on startup when command line includes — seed?

The second prompt. The code to do the seeding is generated only if yes is selected. Futhermore this seeding code will run only if the Node command line contains a — seed argument. An example of such a command would be

NODE_ENV=test node src/ --seed

The gory details

Configuration: cli+ generates a configuration module for every seeding NODE_ENV we specified. In our case my-app/config/test.json is created containing

{
"mongodb": "mongodb://no-connection-string-config-test",
"mysql": "mysql://root:@no-connection-string-config-test",
"nedb": "nedb://no-connection-string-config-test",
"postgres": "postgres://postgres:@no-connection-string-config-test",
"rethinkdb": "rethinkdb://no-connection-string-config-test",
"sqlite": "sqlite://no-connection-string-config-test",
"mssql": "mssql://root:password@no-connection-string-config-test"
}

When we start our app with NODE_ENV=test, because of the way Feathers configuration works, these properties will override the connection strings cli+ generates in my-app/config/default.json. This override is the mechanism that’s used to saveguard data we don’t want overridden.

So you’ll have to manually copy the connection strings existing in default.json to test.json, changing them to point to the databases we’ll seed.

Scripts: cli+ adds some convenience scripts for us in package.json.

"scripts": {
"test": "npm run eslint && npm run mocha",
"test:all": "npm run eslint && NODE_ENV=test npm run mocha",
"start": "node src/",
"start:seed": "NODE_ENV=test node src/ --seed"
},

Notice test:all and start:seed both have our seeding NODE_ENV embedded in their scripts. cli+ will change these scripts if we rerun generate app and change the NODE_ENV prompt.

start:seed starts our app with data seeding. test:all runs all the tests, including those which mutate service data.


Let’s do it

All that detail is nice, but let’s create more of our app and seed our data! We will generate 3 services.

Let’s create teams, and then create users and roles the same way.

Now we add the JSON-schema for their models, including some decorations on how to fake data for the fields. We also indicate what tables the foreign keys refer to. (The article on fakes explains all this.)

DB-agnostic models for our services, using JSON-schema.

We can now generate our fake data

and look at it in my-app/seeds/fake-data.json.

Fake data generated for NeDB.

The key fields are named _id and have 6 digit strings. This is because the services were generated for the NeDB adapter.

cli+ will default to your previous prompts if you rerun feathers-plus generate service. It will make any needed changes to all the modules if you change any of those prompts, such as changing the adapter to Mongoose or Sequelize. The fake data would then generate the correct key fields for the new adapter.

With our fake data created, we can start our app and seed our services.

npm run start:seed

And see a log as the seeding occurs.


In conclusion

Most of what we did here we’d be doing anyway while generating our app. Seeding data comes down to

  • Answering 2 prompts,
  • Customizing config/test.json,
  • Running feathers-plus generate fakes,
  • And starting the application with npm run start:seed.

In return we get

  • Good services’ data with which we can immediately start exercising our application,
  • Good data we can use in our tests,
  • Good infrastructure to guard our production data from accidents.

You can learn more about @feathers-plus/cli by reading its docs.