.npmignore is a serious hazard in Node.js projects you should immediately quit using (except in one situation as outlined below). There is a better alternative that’s been built into npm forever that’s much easier and secure!
What is npmignore?
So I’ve got this npm package called cli-ux. It’s a set of common CLI UX utilities. It has a directory in the root of the project /test that contains the test scripts. To make the npm package a bit smaller, we could do without these files. I can create an npmignore file with the text /test and this will prevent npm from packing that directory into the project when I run npm publish.
Meanwhile…
Inside of this project I locally use direnv, which is a tool that sets environment variables when I enter a directory that has an .envrc. This is in my global gitignore so I don’t have to worry about it being committed into my project. In this project, I have some AWS credentials in there I use for some tests that connect to S3.
I also use nyc in this project for code coverage. This places some local files into a directory .nyc_output in the root of the project. This directory is in my project’s .gitignore so that neither me nor collaborators accidentally commit this directory.
Now, the last thing I want is for either .envrc (for security reasons) or .nyc_output (for tidiness reasons) to find their way into my published package. Thankfully, npm has thought ahead and it will not publish files that are gitignored.
The Hidden Gotcha
However, what you probably don’t know is that my little action of adding the npmignore file actually causes npm to now consult that file instead of the gitignore files. This is a major issue—I’ve now leaked all my AWS credentials out to the public just by adding this .npmignore to hide my test directory.
What’s worse is I probably have no idea this happened. npm publish doesn’t show the files that were packed. I don’t see the files on the npm registry. The only real way to see the files is by adding the package to a project and manually looking inside node_modules. I might do that someday out of curiosity and discover my AWS credentials have been sitting out in the open for months.
EDIT: After writing this article: in npm@6 they’ve begin displaying which files will be packed making this somewhat less problematic.
Anytime you’re working on a project and publish it to npm if that pesky npmignore file exists you’re in dangerous territory. You’ll have no idea if it ships your local dotfiles. Now, npm is smart enough not to ship the .npmrc file specifically, but any other tool you’re using would have to be manually blacklisted in .npmignore!
BTW, if you want to find out what files npm will publish into the tarball without actually publishing, I like to use this little one-liner:
npm pack && tar -xvzf *.tgz && rm -rf package *.tgzWhitelisting
Blacklisting in general is the wrong direction. Almost all projects I’ve seen that rely on gitignore or npmignore ship files that really aren’t necessary (like tests, log files, sometimes entire sqlite databases). You’re playing whack-a-mole to try to add an exclusion every time this happens. (If you even look at what files are in the package).
npm supports whitelisting though, just add a files attribute to package.json with everything you intend to add to the project. Now only the files that are specified in files will be included in the project and your dotfiles will be ignored. If you want to add a dotfile or a test directory (which would be odd), you’ll need to be explicit about it. npm will include the README, package.json, and a couple other files always so you don’t need to specify them.
I like to keep all my JavaScript files all in /lib and use the main file /lib/index.js in my projects. My package.json looks like this:
{
"name": "cli-ux",
"main": "./lib/index.js",
"files": [
"/lib"
]
}Note that you want to prefix all the elements of files with “/”. Otherwise, if you had a directory “test/lib” that would be included as well. Not really a security concern generally, but it helps keep things clean.
This works particularly well for TypeScript projects (as well as babel) where I follow the pattern of /src containing the .ts files, and /lib containing the compiled .js files.
/lib goes into the npm package only and /src stays in source control only.
The one time npmignore is ok
There really isn’t an issue using.npmignore so long as you’re already whitelisting with files. For example, Jest encourages authors to have __test__ directories in their source tree (or .test.js but this solution works with either). It makes sense to define /lib in the files config, but to also add __test__ in npmignore so /lib/__test__ is not included but /lib/index.js is included.
npm and yarn: patch this please
npm has an article on their blog written a few months ago describing pretty much exactly what I’ve just described. I’m far from the first person that’s dealt with this issue. This behavior is confusing and there is no reason for it. I feel the next major version of npm and yarn should put some safeguards in place here.
I think the right solution is for npm and yarn to fail (or at least warn and emit the exact files being packed) if the user attempts to use a .npmignore file without also specifying files.
I have some other ideas, like warning if files like .env or .aws_credentials are committed, but I really feel that blacklisting is an awful way to go about this. Especially with the surprising behavior of how .npmignore relates to .gitignore. At the very least, npm init and yarn init should default to including files.
