The hapi pal roadmap: build a healthy community around building healthy nodejs web services

First, we wanted to say a big thanks to all of you who supported us in the initial launch of hapi pal. The response from the hapijs community was incredibly positive, and that felt really encouraging to both the team and our sponsor, Big Room Studios. Since then we’ve kicked some plans in motion as part of our mission to build a healthy community around cutting-edge hapijs tools and methodologies for building nodejs web services. We’re excited to share some pal news and important bullet points on our short/medium-term roadmap.

Below you’ll find,

  • We spoke at the CBBHacks hackathon. The students were very smart.
  • We launched a “getting started” tutorial. Finally, right? Also, a simple pal deploy via Glitch.
  • We’re releasing a tool that generates knex migrations for you.
  • In progress: three novel CLI debugging tools for hapijs. One of them just might be a server- and route-aware cURL command.
  • We’re designing “overridable” hapi plugins, paving the way for some big plans.
  • A new, concise way to write hapi route pre-handlers.
  • The greater pal community has been busy too! Over the past couple weeks we’ve seen the community write pal-oriented tools for schwifty and Objection ORM, and produce a very curious example project.

We spoke at the CBBHacks hackathon

Devin presents on web services, nodejs, hapijs, and demos pal

CBBHacks was a 36 hour hackathon hosted by Colby College open to university students across Maine (USA), and attended by nearly 90 students. We visited and had an awesome time. It was heartening to be reminded how much awareness has increased in academia surrounding web technologies, even over the past five years. At 11:30pm on the first night, about 50–60 students and organizers gathered to hear @devinivy give his spiel on building web services; how nodejs relates to the web; why he reached for hapijs; the impetus for starting pal; and finally a demo of hapi pal tooling to build an API that serves a random riddle. One team actually utilized pal for their project (so cool!), and we hope that one of them will join Big Room Studios for an internship this summer.

We launched a “getting started” tutorial

Also, a vanilla pal deployment all setup to be remixed on Glitch. Mmmm!

One thing that, in retrospect, we wished we had completed prior to the pal release was a proper “getting started” tutorial. And now, with huge thanks to @zemccartney, anyone should be able to get started in short order! By its nature, hapi pal defers to other packages and ecosystems that are great at what they do — for example, Objection ORM, knex, and hapi itself. Having a high-level overview of how all of these interface with pal (and each other) is really critical context that this tutorial provides. Plus, the tutorial is fun: you help your pal, Paldo, share riddles with their friends by incrementally building a database-backed API. The completed example can be found in the new hapipal/examples repo. Relatedly, we have a new vanilla pal deployment all setup to be remixed on Glitch—just look up and to your left!

We’re releasing a tool that generates knex migrations for you

# Soon!
$ hpal run schwifty:migrate:diff migration-name

As we work on web APIs, we find ourselves spending a fair amount of time combing through new knex migration files to ensure they match-up with our model definitions. It’s tedious, often repetitive, and prone to error. In short, it sounded like a nice thing to automate for ourselves, so we did! (With tons of effort especially from @wswoodruff!) This plays nicely with the hpal CLI’s ability to run commands defined directly on your hapi server: we’re adding a command to schwifty, our model layer for hapi, that performs a diff between your model definitions and the current state of the database, then outputs a base migration file that you can use as a safe starting point to add indexes, foreign keys and other constraints, specifics about column nullability, etc. We’re not trying to cover every possible case or make any assumptions about your application—we just want want to remove the most repetitive tasks associated with writing migration files. We’re aiming to release schwifty:migrate:diff this week!

In progress: three novel CLI debugging tools for hapijs

We have begun development on some new debugging tools for hapijs development, provided as an extension to the hpal CLI, which already allows you to search documentation, create new projects, and scaffold existing projects. We’re working on a REPL for interacting with your server, a panel describing your deployment (routes, services, models, etc.), and a cURL-like command that exposes your routes and their validated parameters over the command line. We’re especially excited about this last one because it potentially removes the common development flow of restarting the server then making requests to it repetitively. Instead, requests can be made easily as one-offs, even when a server isn’t running. Looks for these debug tools released as hpal-debug some time in the next few weeks! Below is a demo of what we currently do have working.

# Just a demo for now, but planning to release soon!
$ hpal run debug:curl post /users --name Paldo -v
post /users (41ms)
payload
───────────────────────────────────────
name Paldo
request headers
───────────────────────────────────────
user-agent shot
host my-computer.local:0
content-type application/json
content-length 16
response headers
───────────────────────────────────────
content-type application/json; charset=utf-8
cache-control no-cache
content-length 24
result
───────────────────────────────────────
{ id: 10, name: 'Paldo' }

We’re designing “overridable” hapi plugins

We’ve dreamt of highly reusable application plugins for almost as long as we’ve been deploying hapi servers. Imagine the ability to pull-in a community- (or internally-) maintained “users” plugin into your project as a simple dependency (users CRUD, login and logout, password reset, etc.), without losing the ability to customize all of its internals for the purpose of your application. It’s a tricky problem that we’ve been chewing on for a while, and over time we’ve gradually built-up the machinery that we think is necessary to make this a reality. In particular,

  • We now have a class-based service layer via schmervice—including the concept of “plugin ownership” of services—that can be used to implement and extend the business logic used in route handlers or elsewhere.
  • A nice result of composing hapi plugins from files via haute-couture is that we have a natural identifier for each call your plugin makes to hapi’s API—say, to add a route with server.route() or a model with server.schwifty(). The identifier is just the filename itself, e.g. "routes/user-create.js” or "models/comment.js”! So if we want to allow skipping or altering any calls like this (which we do!), we have a natural way to reference them.

@devinivy threw together a proof of concept of one plugin overriding another plugin’s routes with relatively minimal changes to haute-couture, and @mattboutet is refining it to more gracefully handle other hard problems that come with this new ability. We don’t know precisely when we’ll be releasing this haute-couture feature or if it necessitates any breaking changes, but be on the lookout!

// Just an experiment... for now
const HauteCouture = require('haute-couture');
// my-user-plugin overrides a base-user-plugin
module.exports = {
name: 'my-user-plugin',
register: HauteCouture.using({
overrides: require('base-user-plugin')
})
};

A new, concise way to write hapi route pre-handlers

We love route pre-handlers. They’re a great place to write route-level authorization rules, fetch data to use in the handler, or any routines that are otherwise incidental to the “guts” of your handler. We love them, but have always wished that they were just a little pithier—mostly because writing an object that looks like { assign: 'user', method: getUser } just feels like it should be able to be written { user: getUser }. And that’s all we’ve done! The new Toys.pre() method transforms concise pre-handler configs into hapi’s longer configs. The toys package contains all sorts of other goodies like this, such as concise request extensions, and helpers to work with events and streams inside async handlers.

The greater pal community has been busy too!

Over the past couple weeks we’ve seen some really sweet work by community members.

Avocat is a much-needed tool by @optii to help transform errors from Objection ORM into relevant boom HTTP errors. Its API is inspired by hapi’s own bounce module, which is pretty neat.

The schwifty-i18n module, also by @optii, makes schwifty models translatable. It’s labeled as a work in progress, but seems to be coming along really nicely. We can see that it’s designed to work especially well with haute-couture, which warms our little developer hearts.

Finally, @jeff-kilbride shared a really interesting example of a minimal haute-couture project organized using multiple plugins. Jeff decided to drop some dependencies of the pal boilerplate so that it could run lighter on AWS Lambda. Slick stuff!

That’s all!

Well, that’s all for now. We clearly have a lot of gratifying work ahead of us, and better yet, many new conveniences on the horizon. Again, big thanks to those mentioned in this update for their contributions. If you’re looking to get involved, feel free to pick up any issue labeled help wanted or simply join us in the #hapipal channel of the official hapi hour Slack.

Your pals,

Devin (@devinivy) and the pal team