How I got to grips with authoring npm packages
Dell Boomi Flow embraces the open source ‘spirit’
Many of our projects here at Dell Boomi are open source, and we like to embrace the open source spirit as much as we can. Our code is available publicly on GitHub, which is a great start, but we are looking to actively drive our communication and collaboration with the open source community, and also start being fully open on some of the things we produce.
So, to make some of our JavaScript projects more accessible to others and potentially more useful outside of any Dell Boomi products, we decided a good starting point was to break the large single repositories down into smaller reusable component parts, and send them out into the wider, open source world!
Breaking out components and publishing to npm
As we use React the most obvious first step was to break out some of our React components and get them published to npm, a tool used by JavaScript developers around the world. (www.npmjs.com).
We initially took two components, a login form and a file upload to begin our journey. While other members of the team worked on getting them into separate, documented repositories on GitHub, my focus was to package and publish them to npm.
You can check out the packages below:
Now, onto my experience of getting to grips with authoring npm packages…
Step 1. Get an npm account
The first thing I realized was that although I’d been using npm packages for years, I had never set up an account! To publish a package to npm I’m going to need an account. A quick bit of form filling and ten minutes later, I was ready to go.
Step 2. Create an ‘org’
Another thing I had never really thought about, but is actually quite obvious, is that all npm packages have unique names. File uploads and login forms are already all over npm so rather than trying to come up with yet another name for effectively the same thing we could do with a way of grouping and namespacing our components — luckily, npm has just the thing: scopes.
However, to use a scope you first need to create an org. Orgs allow teams of contributors to read, write, and publish public or private scoped packages. Seems like a good idea, as all our packages can then be grouped under the same scope. We’ll see how to use the scope later.
Step 3. Login
To enable publishing to npm you first need to be logged in. To login with the terminal use npm login and follow the prompts.
Step 4. Publish
All a package needs to be published to npm is a package.json containing name and version properties. If using a scope make sure your package name includes it, for example:
@scope/package-nameOtherwise you’ll end up publishing the package directly under your username and as I found out, removing a published package from npm is not as easy as you’d expect. You will also want to add a “main” property which should be a path pointing to the package entry point.
Okay, we’re ready.
Oh no, hang on…
Packages published with a scope will be private by default. This is not what we want, the whole point was to embrace the open source community and make it public, so we need:
npm publish -access=publicOr alternatively, you could add this to the package.json:
“publishConfig”: {
“access”: “public”
}Go…
…wow! There it is, we have a published npm package.
Sweet!
What’s in the box?
So, npm created a tarball with the contents of our package folder and sent it up to npm. That raises a question:
What exactly did it package up and send?
Looking back at the terminal it tells me that my package size is around 300kb.
Woah! Why so big?
Well, helpfully it also lists the files it included.
Just. About. Everything.
This is clearly not what we want, but what should we be packaging? Our component code should be compiled to ES5 so people don’t have to process it themselves. Most people will ignore node_modules in their babel config.
The only thing consumers of the package will need is what is output from the webpack build. Now, there are two ways we can filter this; a blacklist of files in an .npmignore file, or more suited to us, a file’s whitelist property in our package.json.
“files”: [
“/dist”
],Much better!
Ok, what’s the size now? My component JavaScript file is still around 140kb!
What’s going on? Let’s take a look inside…
As it turns out, webpack has included react and react-dom libraries into my component. We don’t want this — anyone using the component should be packaging react themselves.
We should instead add them to the externals in the webpack config:
externals: {
// Don’t bundle react or react-dom
react: {
commonjs: ‘react’,
commonjs2: ‘react’,
amd: ‘React’,
root: ‘React’,
},
‘react-dom’: {
commonjs: ‘react-dom’,
commonjs2: ‘react-dom’,
amd: ‘ReactDOM’,
root: ‘ReactDOM’,
},
},Side note: To make sure our component is declared as a module and therefore importable we’ll need to add two entries to the output section:
library: ‘react-file-upload’,
libraryTarget: ‘umd’,We’re now down to around 10kb — phew! Much better!
We should also move react and react-dom libraries out of dependencies and into devDependencies and peerDependencies in our package.json. This will mean multiple copies of those packages won’t end up installed in consuming app’s node_modules.
Note: Having multiple copies of react loaded can cause some strange bugs:
To ensure we always publish a freshly built component file we’ll need to add a script to package.json that will run whenever we publish, or in fact just before:
“prepublishOnly”: “npm run dist”Further reading: npm have some useful notes on Prepublish and Prepare: https://docs.npmjs.com/misc/scripts#prepublish-and-prepare
Local development
While developing the component, it’s helpful to run an application that actually uses it to get a real feel for how it behaves. Although it’s possible to directly install a local tarball file created with “npm pack”, or even run npm install with a directory e.g. npm install ‘./path/to/some/directory’, there is a command that makes things easier: npm link.
Further reading: https://docs.npmjs.com/cli/link.html
- In the package directory, run npm link with no arguments.
- Then in the consuming application directory run npm link ‘your-package-name’.
The app now has a symlink to the package. A shortcut for this is to just run npm link ‘./path/to/some/directory’ in the app’s directory. Any changes made to the code in the package directory will automatically be reflected in the linked package. So, running webpack — watch in the package directory will auto build into our dist folder, this will then trigger a reload of the package in the containing app.
Yay, hot reloading!
We still don’t have debugging with sourcemaps. That would be helpful.
After some trial and error and hair pulling, I found that some webpack sourcemap options didn’t work with linked packages. One I found that does work well is ‘cheap-module-eval-source-map’. Sourcemaps can be tricky to get working on the entry file too (main property in package.json), this lead me to have my code in another module and simply import and export that module in my entry file.
Versioning
After making a couple of simple non-breaking changes to a package I made the mistake of republishing it without changing the version number.
Do. Not. Do. This.
Ever.
Someone else in the team was uninstalling and reinstalling the package to try to get the update. No matter what they did, old version.
Of course — VERSION!
Heed my warning and always change your version when publishing, otherwise people who already have the package won’t get the update.
Conclusion of the journey
Between package.json, npm CLI and webpack config (if using webpack) there are a lot of settings and options to get working together. npm does have good docs though and there is a lot of other info all over the web. It can be difficult, however, to pull it all together and focus on what applies to your scenario.
Further reading:
- NPM docs: https://docs.npmjs.com/
- And this developer guide is also very useful: https://docs.npmjs.com/misc/developers
And there we have it. That was my journey.
So, it’s over to you guys now to prove the value of the open source community to us here at Dell Boomi…
- Have you done this?
- How was your journey?
- What could I have done to make my life simpler?
You can check out the packages below:
Enjoy!
About Dell Boomi
Dell Boomi (Boomi), one of the Dell group of companies, is the leading provider of cloud integration and workflow automation for building The Connected Business.
Our cloud-native, low-code platform helps more than 7,500 organizations run better, faster and smarter. Our technologies connect applications, assure data quality and automate business processes.
Please visit www.boomi.com for more information.
