An idea : npm should use DNS for optimization

This would be only partial

First, it’s important to understand that this switch would be only for the READ operations. DNS is, by nature, a client-read-from-server protocol and NPM is a full CRUD system. The idea would be to switch all READ operations like “What is the latest version of this package” or “What are the dependencies of this package”. Basically this means the view operation of the NPM Api.

How it work now (briefly) and the optimization key

Let’s say I have a project which depends on lodash

  1. The hostname we made the request on is registry.npmjs.com ; so yes, it’s a data registry, so is a DNS server :) ;
  2. The server header : “CouchDB/1.5.0” ; so ye’s we’re making a (cached by fastly) HTTP query to NPM database. Basically, registry.npmjs.com is a CouchDB database.

The concept

(For anyone that is not comfortable with DNS, should be good to read a bit about it before).

The advantages

Less traffic

An average HTTP query is 250B + the package name length, a DNS question message is 16B + the domain length, that’s more than 90% less.

Yeah UDP !

Yep, UDP traffic is faster, easier to manage and reduces latency

Less parsing

Avoiding the whole JSON document parsing every time you only need to check a package version is a huge win, you can update, deploy and install faster !

Like a free and bigger CDN

Every DNS record has a TTL, this means that every user’s internet provider would offer npm, Inc. a free and optimized CDN while they can still decide and control how long the answer will be cached. Google’s public DNS would become an opportunity of a damned fast mirror !

Easier implementation

Third party project that monitor your package versions (like david) would be able to perform a version check without the overhead of the whole npm dependency and parsing.

Less firewall hassle

Allowing your production server to query registry.npmjs.com is often a nightmare in the enterprise world, you have to negociate hardly with your sysadmin or use a proxy. Well, DNS is quite never blocked because it’s done via a local server. No more configuration !

Implementation

Another good news is that it doesn’t seem very hard to implement, it would need two steps :

A custom DNS server

This part is the easier, with the help of node-dns, it’s just about parsing the queried domain, to parse the JSON from CouchDB (and probably cache it to redis or so) and to reply with the asked property.

Update npm/npm

The goal here is to detect these READ operations to migrate it to a DNS query and to keep the classic CouchDB query where needed.

 by the author.

--

--

CTO, Senior Full-Stack Web Developer, DevOps, hacker. Father. #NodeJS, #Typescript, #Golang

Love podcasts or audiobooks? Learn on the go with our new app.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Julien CROUZET

Julien CROUZET

CTO, Senior Full-Stack Web Developer, DevOps, hacker. Father. #NodeJS, #Typescript, #Golang