Writing a GitHub webhook with Rust! Part 1: Rocket

This is part 1 in a series of posts on writing a web service with Rust.

I’ve been very interested in Rust since it made its 1.0 release in 2015, but one of the things I always found lacking in the ecosystem was support for writing web services. I’m a web developer by trade, so a lot of my ideas end up revolving around stuff that can respond to HTTP requests and communicate with a database. So it was with great excitement that I kept close tabs on Rust as libraries like Diesel and Rocket began to emerge and mature.

In this series, I’ll chronicle my experience writing a simple GitHub webhook service in Rust. My team uses GitHub extensively, and one of the things we love about it is how easy it is to automate workflows with their webhooks and REST API. The service I’ll demo in this post will just do a few simple things:

  1. Receive webhook events at a URL
  2. Store information about a Pull Request’s state in a database
  3. Also expose a simple REST API for querying Pull Requests the service knows about

If you want to take a look at the source, check out the repository on GitHub. Disclaimer: the code will be a little bit ahead of these posts, so some things may be different!

Starting a new project

To start off, I’ll generate a new project with Cargo. I’m going to use the Nightly compiler since one of the libraries we’ll use later, Rocket, requires access to Nightly features.

$ rustup update nightly
$ rustup override set nightly
$ cargo new railgun

There’s been some discussion about whether it’s safe to use Nightly for “production”. Personally I don’t have a problem with it, and I think the Rust team’s commitment to stability should mean that in general Nightly shouldn’t have huge bugs in it. The bigger concern in my opinion is that Nightly changes so quickly and not all libraries update at the same rate. It’s possible that you could get stuck on a version of Nightly for a long time because one of your dependencies isn’t able to update quickly enough.

I’m generating a library crate here instead of a binary crate for a few reasons. The main one is that it’ll make writing integration tests easier in the future, but the main drawback is that it requires more configuration upfront.

Here’s the top of the Cargo.toml:

[package]
name = "railgun"
version = "0.1.0"
authors = ["Chris Fung <aergonaut@gmail.com>"]
[lib]
name = "railgun"
path = "src/railgun/lib.rs"
[[bin]]
name = "server"
test = false
doc = false

The interesting part here is the [lib] and [[bin]] sections. This basically lets me tell Cargo that in this crate I want to build a library but I also want to build a binary.

The [lib] section names the library crate “railgun” and specifies its path. The pattern “src/<name>/lib.rs” is that pattern Cargo itself follows, and one I think is pretty intuitive.

The [[bin]] section specifies the binary crate. The brackets are doubled because technically “bin” is an array key in Cargo.toml, which allows you to specify multiple binary crates. We only need one though. Extra binary crates go in “src/bin/<name>.rs” by convention, which I also think is pretty intuitive.

Now the starting directory structure of the project looks something like this:

├── Cargo.lock
├── Cargo.toml
└── src
├── bin
| └── server.rs
└── railgun
└── lib.rs

Choosing dependencies

To write the web service, I’m going to use several key crates:

  • Rocket: the “web framework”, provides facilities for routing HTTP requests to specific functions
  • Diesel: a SQL query builder that we’ll use to access the database
  • Serde: a serialization framework that we’ll use to parse and dump JSON

There are several other crates that we’ll use to provide other functionality, but these three form the core of the service.

Structuring the server binary

The server.rs will be the entrypoint for starting the service. It has just a very short main function that calls into the library to construct the Rocket instance and then immediately launches it.

extern crate railgun;
extern crate dotenv;
fn main() {
dotenv::dotenv().ok();
railgun::app().launch();
}

That’s it!

Cargo treats binaries as separate crates, so we have to import the service’s crate into the binary in order to use it. All the main function does is call the service’s app() function and then call launch() on the result. We’ll see what the app() function does later.

We’re also using Dotenv here to manage configuration via environment variables. Calling dotenv() before launching the app makes sure that the environment is correctly configured.

Building the app()

Now let’s look at the app() function in src/railgun/lib.rs:

pub fn app() -> rocket::Rocket {
rocket::ignite()
.manage(db::establish_connection_pool())
.mount("/", routes![handlers::webhooks::receive])
}

This constructs a Rocket instance in the same way Rocket mentions in its docs. The main difference the call to manage() in the middle.

manage() is used to introduce managed state to a Rocket app. Basically managed state is like a kind of global value that your handlers can request access to as a parameter. Here I’m using manage() to manage a global connection pool to my database.

The db::establish_connection_pool() function is defined in src/railgun/db.rs. It looks like this:

pub type ConnectionPool = r2d2::Pool<r2d2_diesel::ConnectionManager<diesel::pg::PgConnection>>;
pub fn establish_connection_pool() -> ConnectionPool {
let config = r2d2::Config::default();
let connection_manager = r2d2_diesel::ConnectionManager::<diesel::pg::PgConnection>::new(std::env::var("DATABASE_URL").unwrap());
r2d2::Pool::new(config, connection_manager).unwrap()
}

Pardon the unwrap()s.

I’m using r2d2-diesel to create and manage the database connection pool. Basically this function just follows the documentation and returns a new connection pool using the default settings.

This would be fine, but the first problem I encountered had to do with integrating the connection pool into Rocket’s managed state. In order to request access to the managed state, your handler function needs to have an argument of type State<T>, where T is the type of the state value you want. If you’ve programmed with Rust for a bit, you’ve probably realized that the types can sometimes become very long-winded. To make naming the type easier, I created a type alias: ConnectionPool. This lets me write State<ConnectionPool> in my handlers instead of having to write the full name of the type. It also saves me from having to import the references to all of the intermediate types in all my handler modules.

Going back to the app() function, the next part mounts the route handlers. As you can see, I’m storing the handlers in an umbrella handlers module, with submodules for each of the service’s concerns. Inside each of those submodules are the handler functions proper. In terms of a framework like Rails, the submodules would be like controller classes, while the handler functions would be like the methods on the controllers. The handlers umbrella module would be like the app/controllers directory.

Writing a handler

Let’s look at the webhook handler in src/railgun/handlers/webhooks.rs:

#[post("/webhook", data = "<payload>")]
pub fn receive(event: Option<GitHubEvent>, payload: SignedPayload, db_conn: State<db::ConnectionPool>) -> Result<()> {
// ignored for now
}

Just in the function signature we’re already using a lot of cool features from Rocket!

First off, there’s the annotation above the function. This tells Rocket that the function is going to be handler for POST requests to the “/webhook” path. We also tell Rocket that the handler will need to process the request data in some way, and it should use the type of the parameter “payload” to do it (this is the data = “<payload>” bit).

The last parameter requests access to the database connection pool via State<ConnectionPool>. As I explained earlier, this signals to Rocket that it needs to pass the global connection pool into this function. Later, we’ll see how to use the state to acquire an actual connection from the pool.

Request guards

The first parameter to the function, event, has type Option<GitHubEvent>. Basically all parameters to Rocket handlers that aren’t named parts of the route are either Request or Data Guards (or State). In this case, event is a Request Guard. You can tell because it wasn’t named in the “data” part of the annotation.

Request Guards are based on types and provide a way to preprocess the request headers to extract some kind of information. With GitHubEvent, I want to extract the type of webhook event that happened from the X-GitHub-Event header.

GitHubEvent is defined in src/railgun/request.rs:

#[derive(Clone, Debug, PartialEq)]
pub enum GitHubEvent {
PullRequest,
IssueComment,
Status
}

It’s just a simple enum with 3 variants. These are the 3 events that I want to handle with my service.

Now I need to implement Rocket’s FromRequest trait:

impl<'r, 'a> FromRequest<'r, 'a> for GitHubEvent {
type Error = ();
    fn from_request(request: &'r Request<'a>) -> request::Outcome<GitHubEvent, ()> {
let keys = request.headers().get(X_GITHUB_EVENT).collect::<Vec<_>>();
if keys.len() != 1 {
return Outcome::Failure((Status::BadRequest, ()));
}
        let event = match keys[0] {
PULL_REQUEST_EVENT => GitHubEvent::PullRequest,
ISSUE_COMMENT_EVENT => GitHubEvent::IssueComment,
STATUS_EVENT => GitHubEvent::Status,
_ => { return Outcome::Failure((Status::BadRequest, ())); }
};
        Outcome::Success(event)
}
}

Again, a lot going on in here. FromRequest is how Rocket understands that GitHubEvent is a Request Guard. Rocket calls from_request() and passes the request object in.

I only want to handle the 3 event types I specified, so first I extract the contents of the X-GitHub-Event header. If it’s not there or if there’s more than one of them, I just return a Failure right away. This causes Rocket to forward the request to my failure handler and generate a 400 response.

Next I need to match on the header’s contents. If it matches any of the 3 types, I convert the header into the appropriate type and move on. If it’s not, that’s also a failure.

The last line finally returns a Success wrapping the type of the event.

In the handler’s signature, I wrapped GitHubEvent with Option. This basically means that although GitHubEvent might fail to process the request, that’s OK for this handler, and Rocket should just give us None instead. Inside the handler, I’ll deal with mapping over the Option.

I think Rocket’s guard mechanism is a very neat way to leverage Rust’s type system to implement logic like this. Rather than giving the handlers direct access to the raw request, making guards pre-process the request keeps with the philosophy of type safety and explicitness.

Data guards

Request Guards are cool but Data Guards are cooler. The second parameter to the handler is payload which has type SignedPayload. This is a Data Guard, which is like a Request Guard, but also gets access to the body.

With SignedPayload I want to make sure that the webhook payload I received includes a valid signature. GitHub lets you sign your webhooks with a secret to improve security. If the signature is not valid, I want to halt processing the request right away.

SignedPayload is also implemented in src/railgun/request.rs:

#[derive(Debug, PartialEq)]
pub struct SignedPayload(pub String);
impl FromData for SignedPayload {
type Error = ();
    fn from_data(request: &Request, data: Data) -> data::Outcome<SignedPayload, ()> {
let keys = request.headers().get(X_HUB_SIGNATURE).collect::<Vec<_>>();
if keys.len() != 1 {
return Outcome::Failure((Status::BadRequest, ()));
}
        let signature = keys[0];
        let mut body = String::new();
if let Err(_) = data.open().read_to_string(&mut body) {
return Outcome::Failure((Status::InternalServerError, ()));
}
        let secret = match std::env::var("GITHUB_WEBHOOK_SECRET") {
Ok(s) => s,
Err(_) => { return Outcome::Failure((Status::InternalServerError, ())); }
};
        if !is_valid_signature(&signature, &body, &secret) {
return Outcome::Failure((Status::BadRequest, ()));
}
        Outcome::Success(SignedPayload(body))
}
}

A lot going on!

First off is the definition of SignedPayload. It’s a simple tuple struct with one String member. This will hold the request body if the signature is correct.

Now the FromData implementation. Like FromRequest, this tells Rocket that SignedPayload is a Data guard. Rocket calls from_data() and passes the request and the body.

Like before I start by extracting the signature from the X-Hub-Signature header. If there isn’t one, then the webhook isn’t secure so I return Failure immediately.

Next I read the body and grab the webhook secret from the environment variable. is_valid_signature() is a helper function I wrote that computes a HMAC signature of the body and uses a constant-time compare function to compare it with the signature in the header. If the 2 don’t match, a Failure is returned, otherwise, a Success is returned wrapping the SignedPayload that itself wraps the body.

Like Request guards, I think this is a really cool feature! The SignedPayload type lets me extract the concern of validating the signature out of my handler functions and run it before the functions are even called.

The return type

The final type in the handler’s signature is the return type, which here is written Result<()>. Since the handler is responding to POST requests from GitHub, and GitHub isn’t expecting any response back, it makes sense that the handler would return an empty body. With Rocket, returning () (pronounced “unit”) from a handler automatically makes the body empty.

How about the Result type? That’s not the standard Result but is actually a custom type defined with error_chain. Rocket also understands how to handle Results from handlers. By default, if the Result is Err, Rocket translates that into a 500 response; if the Result is Err and the Err type also implements Rocket’s Responder trait, the response is delegated to that type instead.

This is another case where I think Rocket’s type-driven design really shows! Results are pretty ubiquitous in Rust and it’s great that Rocket has incorporated them into the built-in error-handling flow. The Responder trait is also great and lets you abstract how to handle the different errors in your app away from the main code in your handlers.

In my case, I’m not implementing Responder yet, but I think I will in the future. For now, it’s fine that my handlers just return 500, but it would be great to be able to specify different response codes for different kinds of errors.

To be continued

That’s all the types in the function signature!

It’s really cool to me how type-driven Rocket is, and how advanced features are all encoded with traits like FromRequest and FromData. Even cooler was how easy it was to implement these traits for my own types and provide my own functionality!

I didn’t expect to write a whole blog post just about the signature of one function, but it turned out there was a lot to unpack in just the signature! In the next post, I’ll start looking at the body of the handler function, accessing the database, and writing tests!

Like what you read? Give Chris a round of applause.

From a quick cheer to a standing ovation, clap to show how much you enjoyed this story.