Control What you Publish Inside your Npm Packages

It’s hard to decide whether to use the package.json#files field or .npmignore files. Maybe there is an alternative.

NOTE: This post contains information that’s not 100% accurate as a kind reader pointed to me. I’ve published a newer story fixing my mistake. You can read it here.

Photo by George Bonev on Unsplash

onIt’s very important to control what gets published inside a package. You don’t want to publish temporal files and make your package weigh 300 tons of gigabytes or, even worse, put private files in there.

Been there, done that.

With npm is always a good idea to run the npm pack command to see exactly what goes inside your package:

npm pack --dry-run

Newer versions of npm also support the --dry-run on the npm publish command. Also, this newer versions are kind enough to output the package contents while you are publishing. Mark my words: you don’t want to discover that your package is misconfigured at publication time. You should take control of the package contents from the beginning.

package.json#files field and .npmignore files

So, how does npm knows what to pack? According to the docs here:

• You can use the files field to define patterns of files to be included.
• You can skip the files to include everything and use .npmignore to define patterns of files that shouldn’t be included. You can put an .npmignore in the root folder or in any subdirectory.
• If there is no .npmignore, npm will look for a .gitignore file.
• Certain files are always included (package.json, …), and other are always ignored (node_modules, …) no matter what you say with files and .npmignore.

The docs also states something that’s quite baffling:

Files included with the “package.json#files” field cannot be excluded through .npmignore or .gitignore

It’s not easy to grasp this rules, and corner cases are not well documented. Personally, I don’t like this approach. I can’t open a package.json an see at a glance what’s going to be included. If I want to be exact, I have to curate my files or scatter some .npmignore files. Being unable to exclude stuff that was added via files is simply annoying.

Take a look at the following example: a React library with colocated files.

.
├── component
│   ├── component.js
│   └── component.test.js
└── package.json

I want to publish component but exclude the tests, my options are:

• Write an exhaustive files field that list each and every file I want to add (in this case just component/component.js).
• Write an .npmignore that ignores **/*.test.js.

Usually, anyone would go for the latter, but the former is the solution that gives me the guarantee that I’m just packing what I want.

If it were possible to combine files and .npmignore it would be better, but reality is harsh. So, I have two proposals to fix this.

Proposal 1: support includes and excludes through the files field

The files field could support both inclusions and exclusions with clear semantics: define what’s included, if you want to make an exception, use exclusions.

"files": {
  "include": ["component"],
  "exclude": ["**/*.sample.js"]
}

This can break the internet but I guess we can use an alternative property name during the migration phase or another flag to indicate that we are using a newer model.

Proposal 2: support package.js files

Have you ever seen a ruby gem gem spec? If you haven’t, here is one (I’ve omitted some parts for the sake of brevity):

Notice lines 10 to 12. This gem spec defines the files to be included using an executable instruction: get all the source files controlled by git and exclude all kind of tests. Gem specs are just ruby code, so the code will expand that code into an exhaustive list of files. This is really flexible and allows us to define the included file with precision. Beautiful.

So, why can’t we have package.js instead of the boring package.json? Most JavaScript tooling is moving to JavaScript config files instead of JSON ones. JavaScript files are composable, lintable and testable, and that’s very useful.

I understand that npm uses JSON as a way to provide metadata from a package without having to run the package itself. I see the logic behind this reasoning, but it could be possible to use JavaScript in our code and pack the generated package info in JSON format at publish time, having the best of both worlds.

I don’t think is going to happen anytime soon, though 😔.

What can we do today?

Use a workaround: fake the include/exclude support in files with a custom tool.

We are doing this in our projects. We write a custom filesGlob property in the package.json using the fast-glob library supported syntax:

"filesGlob": {
  "include": ["component/*"],
  "exclude": ["**/*.test.js"]
}

And then, a expand-files script converts our expressions into an exhaustive files list using the library. We can see the expanded fields before publishing if we dry run a npm pack:

expand-files npm pack --dry-run

The script modifies the package.json and execute the provided command. Once done, it restores the original file. We use a different property name to avoid problems with other tools that may expect a spec-compliant files property. It’s not perfect but it does the job.

Here is the full script:

Summing up

I don’t think that the npm model is fundamentally flawed. It works for the vast majority of cases and when we have published a package with more or less files than it should, it has been due to our own lack of attention 😅. However, when dealing with colocated files we wish we have something different.

For the time being we will keep using our workaround hoping that one day the people at npm will give us what we need. That’s the thing with community driven software, specifically the JavaScript community, sometimes it moves at the speed of light, sometimes is stuck on years old conventions and practices. You’ll never know what will happen tomorrow.