NPM + Jam3: Clients, Open Source, and 1000s of Dwarfs
At Jam3 in the past two years we’ve gone from no modules to, AMD Modules with RequireJS, and now onto CommonJS with Browserify. We’d like to talk about what we’ve learned during the past 8 months to help people deciding a path for their development workflow as individuals and as companies.
Something which may not be apparent at first is that if your company begins to work with NPM and CommonJS modules is that the company might fundamentally change.
For years Jam3 has used OSS without contributing much. It’s hard for a small company to justify putting their secret sauce out there for competitors to “steal”. Today Jam3 has realized, as many companies have, the code is not the product but rather the knowledge and skill to use that code is the product.
We may, if we’re generous, have had maybe five public repos on GitHub. Now we have 65 public repos (and counting). Since CommonJS modules are designed around the Unix Philosophy people may place just a small bit of reusable code in one public repo therefore you’re going to have many more public repos. Those individual small bits of code are worth little individually but the sum of those small parts and how you put them together is powerful.
It’s much easier as a company to justify Open Source as “we’ll put the nuts and bolts out there for everyone to use” rather than “we’re going to put an entire car out there for everyone to use”. Now we’re not saying we’re not going to put cars out there for everyone to use (we will) but most of the time it’ll be nuts and bolts. We see “cars” as libraries or tools which will pull together all the nuts and bolts. For instance in our case for one of our projects we developed a library and a set of tools to allow for exporting After Effects 3D Compositions and rendering/modifying them in the Browser. This would be considered a car which is private but built from many public modules (nuts and bolts). The point is as you begin to develop more modularly you’ll have to create rules and boundaries for what gets open sourced and what does not. These are rules your company may not have had before.
Working for our clients while adopting this approach makes contributing to OSS a bit tougher. Some of our clients who have been around the block a few times are savvy to the pitfalls of Open Source. Really it’s not Open Source but using code developed by a third party rather than the company their hiring. The real issue is who is liable and responsible if things go bad. These clients will demand up front before the project starts a list of open source libraries which will be used.
Providing a list of open source code that will be used on a project is much easier with monolithic libraries. In the past that list may have included things like jQuery, Backbone, ThreeJS. Today that exact same list might look like this: dom-select, dom-style, dom-event, dom-tree, xhr, bigwheel, vue, and ThreeJS. This only lists the top most modules we’ll be using and we also need to list the dependencies of these modules to our clients. So we’ve gone from listing three libraries to ten (if we’re being generous it maybe way more than ten).
So the challenge is: What is the method to communicate to the client third party software being used for their project? This issue is also compounded by the fact that developers are more likely to include more modules during the project. We’re currently experimenting with a workflow where we initially list modules and libraries we’ll definitely be using on project and then during the project giving updates on libraries/modules used. This process could potentially be automated by creating a project scraper which analyzes dependencies on a project through package.json files and listing them out in a human readable website.
It’s one thing to use modules on projects it’s also another thing creating modules for a client project. It would be unfair to work on a project for a client and open source code developed on a project however it’s still vital. In order to facilitate this we’ve negotiated with clients what we can and cannot open source. Most clients are ok with the fact that we’re open sourcing nuts and bolts and not cars.
Obviously these decisions may affect the type of clients you want to work with. We’ve had to do projects where all code used would be developed by ourselves, closed source, but having to do that now seems archaic and like you’re constantly reinventing the wheel. We’d like to say it’s better to stand on the shoulders of giants but the reality with modules we’re standing on the shoulders of thousands of dwarfs.