Warthog: a TypeScript + GraphQL framework for quickly building APIs.

GraphQL API Framework with strong conventions and auto-generated schema

Introduction

Warthog is a Node.js GraphQL API framework for quickly building consistent GraphQL APIs that have sorting, filtering and pagination out of the box. It is written in TypeScript and makes heavy use of decorators for concise, declarative code.

Version 1.0 was released July 2018, and has just hit version 2.9.2. Warthog is open source, has many contributors and key contributions have been made by several Indigo engineers.

In this article I will explain what Warthog is and hopefully give some context as to why it is special. I will explain where Warthog fits into the GraphQL landscape and outline some of the problems it intends to solve. I will also show how easy it is to get up and running.

What is Warthog?

Warthog is a TypeScript-based GraphQL API framework for Node.js. Warthog incorporates TypeORM for data modeling, uses TypeGraphQL for code generation and follows OpenCRUD conventions.

Features

• Written in TypeScript
• Code generation for GraphQL schema + Database Models
• ORM database models via TypeORM
• Integrates with PostgreSQL
• Free CRUD
• Intentionally opinionated

Why Warthog?

There are numerous API providers based on the GraphQL specification. Express GraphQL is a modified Express server that can handle GraphQL requests.

Apollo Server is a community-maintained open-source GraphQL server that works with all Node.js HTTP server frameworks. Apollo provides a superset of convenient functionality like caching and easy schema generation, but does not include native database functionality.

Several frameworks exist to present your data access layer as a GraphQL graph. Prisma exposes CRUD operations, it glues together your GraphQL server and your database. Hasuru is an open-source engine that connects to your databases and micro-services and gives you a production-ready GraphQL API.

There are managed service providers like AWS AppSync and Prisma Cloud, useful if you’ve invested heavily in a serverless architecture but there are some drawbacks to consider… you are locked into vendors and customization will be limited. This may not be viable if your code has to run in house or on the premises. It may be very challenging if not impossible to integrate into your existing systems.

… these frameworks may be suitable for your needs, but you may need more control over your database. Imagine you have special requirements, or if you just need custom queries for performance reasons.

You could consider writing your own “bare metal” SQL/NoSQL solution (but that would be costly), or you could abstract your database functionality via an ORM… the most popular Node.js ORM’s are Sequelize and TypeORM. These tools abstract away database code and allow you to manipulate complex data [relatively] easily.

TypeScript is popular in the enterprise space because it can prevent bugs and adds intelligence to your IDE with static type checking. It can be more complex but adds safety to your code and enhances your developer experience… for more, please see my article Why You Should Be Using TypeScript + GraphQL.

TypeScript + GraphQL + ORMs go well together, but things can get hairy when it comes to schema generation. Consider that your schema now has multiple potential “sources of truth”: GraphQL types, TypeScript types, and database models… developers complain about duplicate boilerplate code…and don’t forget about unit testing! It’s important to realize each abstraction is valid, but many users resort to code generation to keep everything in check.

Warthog is a TypeScript-based framework that generates your schema from a single code-first source of truth. You get the power of TypeORM built-in and it uses TypeScript class decorators to reduce cruft and allow you to update your API + data models in the same place.

What do you mean by “Source of Truth”?

…otherwise known as “Single Source of Truth”, it’s a term that comes from information systems theory that you hear often when discussing GraphQL.

“Single Source of Truth” is a concept that an organization can apply as part of its information architecture to ensure that everyone in the organization uses the same data when making business decisions. When the data is input into these various systems from different entry points, you will end up with a lot of data flowing around within your organization, between different silos and systems that are difficult to track, manage and consolidate.

There are some special challenges choosing schema-first API development if you’re using TypeScript. You have to create your schema types in SDL (Schema Definition Language) as well as data models for your ORM (i.e. classes in TypeORM) and then write resolvers for your queries/mutations/subscriptions. You’re forced to create your TypeScript interfaces before you can implement resolvers because otherwise, you get type errors and it’s not trivial to just stub out your methods because you will need type information.

There is also a problem with redundancy and management. Just adding a new field, you’ll find yourself updating many files and modifying the model, the schema, the resolvers, the type definitions and your unit test interfaces before you can update your client interface. This is difficult to manage and prone to error. If you could establish a single code-first source of truth, you could potentially eliminate this issue.

Warthog uses a code-first strategy, where the schema is generated programmatically.

Warthog utilizes TypeGraphQL which is a popular code generation tool. It allows you to decorate your TypeORM data models and have it generate the rest of your code.

CRUD? OpenCRUD?

CRUD is an acronym for “Create, Read, Update, Delete.” It is a very common development pattern that describes four different types of functionality. Most data models need to be able to perform these actions. Warthog offers CRUD for free.

Another notable feature of Warthog is that it only does “soft” deletes, i.e. it marks a record in the database for deletion and temporarily prevents it from being selected as opposed to a “hard” delete which permanently deletes a record.

Warthog uses the OpenCRUD standard developed by the Prisma team. It’s a standardized pattern so that tooling can be shared among packages. For example, if you were to write a GraphQL client for OpenCRUD, then it should, in theory, be compatible with all OpenCRUD GraphQL APIs.

End-to-End Typing

In a modern web development stack, the power of typing can be experienced at multiple levels. GraphQL itself provides types and with the right tooling, this can enhance your developer experience tremendously Types allow your IDE to be smarter, suggesting the correct types as you develop. Consider that an intrinsic feature of any GraphQL API is introspection and all GraphQL APIs can be inspected. You can ask any GraphQL server about its fields, mutations, subscriptions, etc. all through a standardized interface.

Introspection is significant because you’re effectively getting [some] documentation and runtime type checking for free, it’s far harder to make a mistake, often you can autocomplete with just a ctrl-space key sequence and things like field deprecation are presented in real-time. Most servers provide integrated API explorers like GraphQL Playground or GraphiQL. Introspection makes it possible to write common tooling.

TypeScript is a language that provides a (strict) superset of features on top of regular ECMAScript. If you’re using a modern IDE like VS Code, TypeScript has strong guidelines and typed guardrails as you develop. Type errors are a huge source of bugs in JavaScript, and typing properly tends to improve your code quality. It also makes you a better developer as you learn about type safety. TypeScript at the language level combined with GraphQL at the API boundaries offers a high level of code introspection and safety.

TypeORM provides TypeScript types for your database models, any changes at the database model level can bubble all the way up to the API interface. There are different packages available for code generation but most offer type generation for your client, which means you can get type definitions generated from your GraphQL API for your client code (e.g. React type definitions).

Once you have everything established and you’re using a TypeScript-based client, a change to a database model will permeate all the way up. If you change something, you’ll immediately see errors and warnings and anything that is broken will be highlighted. You get contract validation across features and this produces extremely clean code because silly errors are much harder to make and breaking changes are immediately apparent.

Warthog generates fully typed code. It has some strong opinions and follows industry standard conventions.

What do we mean by “Opinionated”?

As a developer it’s good to have the freedom to be creative, but freedom comes at a cost. If you’ve got multiple developers working on the same project and they’re all following their own conventions, your code could be messy and incoherent.

To some degree, all frameworks make assumptions about best practices for application architecture but when assumptions are strong, the framework is considered “opinionated.” Conventions can make your application more organized, more readable and more predictable. Consider community support — you want things to be uniform so that people can help each other with less friction. It effectively guides you into a way of doing things.

Assumptions and conventions are intended to speed up development. By design, the software is limiting.

Apple is a great example of a company that has strong opinions about its products. Prettier is one of my favorite open-source projects. It takes the concept of opinionated software and aims to stop all ongoing debates about code formatting by accepting a common style guide.

Warthog @ Indigo

Warthog is used in production on several projects at Indigo. Indigo Marketplace uses Warthog as the framework for the Marketplace Orders API, which is used to manage, filter, search and fill orders. Marketplace is a complex system involving products, accounts, authentication and price modules for various markets and order state management.

Indigo employs the most advanced tools and techniques for technology-driven agronomy. Indigo agronomists are equipped with advanced weather data and satellite imagery, along with aggregated research across millions of acres using Indigo Atlas, to provide year-round support for recommendations specific to a farm’s unique needs.

Warthog is used to power Indigo’s Agronomic Data Collection API, which involves agronomist management, reports and models for farms/growers and geographic zones/boundaries.

Indigo Transport™ is a technology-enabled platform that creates choice by connecting carriers with growers to move grain more efficiently. Warthog is used on internal utilities for transport functionality.

Prerequisites

Warthog works with any operating system that supports Node.js. Node 12 is the LTS version as of June 2020, you can find official instructions here.

NPM and Yarn are needed for package management. You can find the official Yarn installation instructions here.

You’ll need to install Postgres, I recommend using Docker because it is self-contained, the config lives in git, so it’s easy to recreate, we’ll have to set up our project before setting up a docker-compose.yml file… alternatively you can install Postgres locally first.

Here is the docker-compose.yml file I will be using, this creates a db called warthog-starter-development. note: you won’t need to run the db:create command later if you’ve already created a db.

Install Notes

OK, so now that I’ve given you some context and rationale, let’s get started with Warthog!

I’m going to be installing on Ubuntu LTS 18.04. You should be able to use these instructions on all major operating systems.

… I have an old TypeORM project, I thought it’d be a good idea if I could get my simple TypeORM entities to work with Warthog, I will elaborate as I go along.

1. create a new Node.js project and initialize it (it’s also a good idea to init your git repository with git init), replace warthog-demo with your desired folder name, npm will ask you some questions about things like project name, author, license etc:

mkdir warthog-demo && cd warthog-demo
git init
npm init

(Note: you have to enable experimental support for decorators, you must enable the experimentalDecorators compiler option either on the command line or in your tsconfig.json)

2. Now we can add Warthog to our Node project.

# Add warthog so that we can use the CLI
yarn add warthog
# Bootstrap a new application using Warthog CLI
yarn warthog new
# Install dependencies from generated package.json
yarn

yarn warthog new

It will ask what you want to call your warthog project. I’m calling mine warthog-demo . Note after running warthog new we can see our first generated files:

Folder structure

3. Let’s add some entities.

Here’s where things get interesting, I have some models that I’d developed previously using TypeORM, I’ve shown them below, I want to figure out how to get the same functionality with Warthog, my models don’t match the Warthog standard exactly, Warthog has a base model, so I’m going to have to adjust them slightly.

These models are simple but demonstrate some powerful features of TypeORM, I’m generating many-to-many relationships for categories and the categories themselves have children implemented via a closure-table, this is not a trivial thing to do with raw SQL, generally it requires a decent amount of knowledge. Notice how Warthog and TypeORM incorporate decorator declarations “@expression”.

BrainStrike Card Entity / Model

BrainStrike Category Entity / Model

Lets initialize our Warthog API with a Category entity first, note I’m starting with just a name variable and Warthog should add the additional fields from the base model, I will add the many to many relationship later after creating the Card entity, we will make those changes by editing the model files:

# Generate a resource (model, resolver and service)
yarn warthog generate category name

If we look at our demo folder we can see a new modules/category folder containing our generated model, resolver and service.

category entity

Now lets generate our schema and TypeScript classes:

# Generate typescript classes and GraphQL schema
yarn warthog codegen

Note a new generated folder:

codegen generated folder

Make sure you’ve got a Postgres server running, you need update your .env file (in the root of the project) with the correct server details:

.env configuration

Now we can add our database and the initial migration, note if you’ve already got a database called warthog-starter-development you won’t need to do the db:create command.

# Create your DB
yarn warthog db:create
# Generate the DB migration for your newly generated model
yarn warthog db:migrate:generate --name=create-category-table
# Run the DB migration
yarn warthog db:migrate

after db:migrate

Next we can start the server:

# Start the server
yarn start:dev

If all goes well you will see:

‘🚀 Server ready at http://localhost:4100/graphql'

Opening your browser to that location should present the GraphQL Playground:

warthog running GraphQL Playground

That’s great, our entity has been created!… let me try the same commands to create the Card entity, then I’ll try to extend those and regenerate:

# Generate a resource (model, resolver and service)
yarn warthog generate card number:int label:string# Generate typescript classes and GraphQL schema
yarn warthog codegen# Generate the DB migration for your newly generated model
yarn warthog db:migrate:generate --name=create-card-table
# Run the DB migration
yarn warthog db:migrate

7. Now I want to extend my entities to have the same many-to-main and closure-table tree functionality that I had in BrainStrike, it took me a minute of poking around to figure out the changes, but the formats are very similar, I had to use non null assertions and some small tweaks:

card.model.ts (for Warthog)

category.model.ts (for Warthog)

Now all I have to do is save the files, codegen again and re-start my server:

# Generate typescript classes and GraphQL schema
yarn warthog codegen# Start the server
yarn start:dev

If you look at the startup log, you can see the schema being synced:

startup after changes

…and loading up playground in my browser again — it’s all there!

final folder structure

Wow!, that’s pretty remarkable because that’s a lot of functionality and would’ve be a lot of work. I did all this in less than 2 hours.

I’ve made the code I created available on github…but you should see the Warthog starter for a more complete/supported example. For more detailed information about TypeORM migrations, please read the TypeORM documentation.

Conclusion

GraphQL is rapidly becoming a new internet standard. TypeScript has been around since 2013 and interest has only grown over time.

Warthog is great for getting a GraphQL API started, it’s also very capable for complex use cases…. if you have a living code-base, Warthog can be very powerful. It has very impressive code generation, can reduce cost, it’s also battle tested, used in production on several large applications at Indigo. If you’re a GraphQL API developer, you should check it out!

If you’re interested in learning more about Indigo and our engineering efforts/opportunities, please check out our blog.

Resources

Warthog

Node.js / TypeScript / TypeGraphQL / OpenCRUD / GraphQL specification

Apollo / Prisma / AWS AppSync / Prisma Cloud / Express GraphQL / Hasuru

Sequelize / TypeORM

Warthog Code Example / Warthog starter

Indigo AG Engineer Blog