APIs and Laravel: Part 2— JWT Tymon

In last post, we discussed what a JSON Web Token was and why you might want to implement it within your application. In this post, we’ll talk about the tymondesigns/jwt-auth package. We’ll discuss some of its use cases, how to use its API, and break down how the package actually integrates with Laravel.

If you feel lost reading this post, don’t get discouraged. I am attempting to present you with sample authentication flow for our use cases while also trying to reveal how the tymon package actually works. If all the extra explanation and code examples muddy the waters too much, consider reviewing documentation as an alternative.

Tymon

The Tymon implementation is a well known package that is great for constructing and managing JWT tokens in Laravel. It ships with a comprehensive API that, among other things can parse tokens from the HTTP request, authenticate users, authorize users, and refresh tokens.

While the Tymon package does have a development branch and will most likely experience a significant upgrade in the future, we’ll be installing the stable (0.5.*) branch as the development branch is not as well documented and, from my experience, occasionally ships with composer conflicts depending on your Laravel installation.

Installation

I’ll briefly cover package installation. However, this post primarily focuses on the package itself, and, if you want, you can either install just the package or review it on github as a long as you understand where the config and service provider are located.

After creating your laravel 5.* project, install tymon’s jwt-auth.

$ composer require tymon/jwt-auth

Next, add the service provider to the bottom of the providers array in our config/app.php.

Tymon\JWTAuth\Providers\JWTAuthServiceProvider::class

As you become more familiar with the package, it will be important to review the service provider so you understand how Laravel’s dependencies/services are loaded into the package’s components.

With the service provider installed, we can now publish the package assets.

$ php artisan vendor:publish --provider="Tymon\JWTAuth\Providers\JWTAuthServiceProvider"

This will add a jwt.php file to your config folder.

Finally, we’ll use tymon’s custom command to generate an api key.

php artisan jwt:generate

Make sure to check your config/jwt.php file to ensure that your secret has been added.

You also have the option to add two Facades to your config

'JWTAuth' => 'Tymon\JWTAuth\Facades\JWTAuth',
'JWTFactory' => 'Tymon\JWTAuth\Facades\JWTFactory'

The JWTAuth component will be our primary interface with the package. The JWTFactory is a facade for the PayloadFactory which is useful for manually creating tokens.

What do we want to do?

First, we want to be able to have a user submit credentials, and have the capability to provide them with a token which we can later use to validate their identity.

Second, we want to be able to validate token claims, verify the token’s authenticity, and issue new tokens if necessary.

Setting the Token

When a user submits their username and password (credentials), we can use the package to create a token for that user. If their credentials are correct, we return them an encoded token either through JSON (recommended), a HTTP Header (recommended) or as a cookie. We’ll discuss how to do this later as JSON when we review the API.

If passing the token back to the client as a cookie, you need to add the cookie name to the $except array on the EncryptCookies middleware.

Note: Even if you pass the token back as a cookie, you do not have to submit the token as a cookie. On the client, you can grab the token from the cookie and use any of the three methods below to submit the token.

Passing The Token on Each Request

After validating the client’s credentials and returning them a token, the client must attach at token in some form to every request if we want to be able to identify and evaluate the user.

HTTP Header: Even if you saved the jwt token as a cookie, in memory or localStorage on the client side, you can still pass it to the server on each request as HTTP Header (recommended). By using the JWT protocol’s standard “Bearer” header, the package will be able to identify and parse the token by default.

Authorization: Bearer aaa.bbb.yyy

Many front end frameworks such as Vuejs, Angular, and React, allow you to apply middleware/interceptors to outgoing api calls. This is a great place to attach the token.

Laravel 5.3 actually ships with a Vuejs interceptor by default for csrf in its bootstrap.js script which we can amend to suit our purpose.

Vue.http.interceptors.push((request, next) => {
  request.headers.set('X-CSRF-TOKEN', Laravel.csrfToken);  
  
  // we've stored the token locally in localStorage
  let token = localStorage.getItem('jwt-token');
  request.headers.set('Authorization', 'Bearer ' + token);
  next();
});

Query String: You can also pass the token in the query string (not recommended). If you do so, it is best to use “token” as the key name as this is the query string key name the package expects.

application.dev?token=aaa.bbb.ccc

Although query parameters are encrypted over https, they are still saved in your web history, and could potentially be stored in logs on the server.

Cookie: If you are passing the token back to the server as a cookie (not recommended), you need to ensure that the cookie has a secure flag and that http_only is set to false. If using this method, you will have to extract the token value from the cookie manually on the server as the packages’ parsing helper (getToken()) will not work.

So now that we understand the use cases, let’s look at the package.

The Config

It is important to review the config to understand how the package will work. With the exception of the user value, most of these will probably work by default.

user and identifer

The user configuration (not the same as in the user value in the providers array) will determine which user object will be resolved. If you do not use the default User namespace (such as if you moved App\User to App\Models\User\User), make sure to update the value to reflect this. It is similar to updating the provider value in auth.php.

In addition, the package will set the user’s identifier as the sub value on the token, as well as use the identifier to find the user. If you have a custom identifier (such as a uuid or specific api_id column on your table), you can change the identifier value to match the attribute’s name. Otherwise just leave it as default to use the database id.

blacklist_enabled

By definition, JWT Tokens are stateless and are valid for as long as its claims are valid. In order to invalidate tokens, we must store them on the server and check each token against this blacklist.

Set blacklist_enabled option to true and ensure that you are implementing a valid cache driver in Laravel, which will store the blacklisted tokens. By default, Laravel has a file driver so this should work without any additional configuration.

ttl and refresh_ttl

The ttl, or time to live, value determines the number of minutes for which the token will be valid. By default it is set at 60 minutes and should be kept short. The exp (expiration) claim on the token will be calculated from the this value.

If you look at the service provider, you can see where we set register the ttl on the PayloadFactory (Tymon\JWTAuth\PayloadFactory).

function registerPayloadFactory() {
    ...
    return $factory->setTTL($this->config('ttl'));
    ...
}

Then, when the factory generates a new token, the exp value will be set using the $ttl

public function exp(){
 return Utils::now()->addMinutes($this->ttl)->timestamp;
}

When a token is refreshed, the current token will be added to the blacklist and a new token is created with the same sub and iat (issued at) value. The refresh_ttl will determine if a token can be refreshed by looking at the iat, which will always be the same as when the token was first issued.

required_claims

When a token is decoded from the request, it will verify that all the required_claims are present. Otherwise, it will throw an exception. The default required claims will be set automatically on the payload so you do not need to worry about setting them. However, if you want to ensure that a custom claim is always set and available, you can expand this list. Be aware that this will be applied to all tokens, and without a clear case, you can leave this as the default.

algo

The algo configuration will determine which algorithm is used to hash our token. The default value works well for us, and you can refer to the previous article for more information.

I’ll touch on some of the more in depth configuration values such as the providers at the end.

The API

The Authenticator (Tymon\JWTAuth\JWTAuth)

The authenticator is the your primary interface with the package throughout your application. Nearly all of it’s public functions are helpers for interacting with the specific components of the package. A few of it’s primary methods are:

• attempt($credentials) // returns encoded token | false | exception
• authenticate($token) // returns user | false | exception
• refresh($token) // blacklists old $token and returns new JWTToken
• toUser($token) // returns user | false
• fromUser($user) // returns $token
• getToken() // sets and returns the parsed $token
• parseToken($method = ‘bearer’, $header = ‘authorization’, $query = ‘token’) // sets token on JWTA

The package has many components that power the main JWTAuth authenticator such as the Token (Tymon\JWTAuth\Token) class. It is important to understand when to expect to pass or receive an encoded token (string) or a token object (Token). In the above API, all $token arguments are encoded strings.

The authenticator allows for further specificity including passing custom claims as well as some additional methods, but these will be your most used functions.

Authentication

Let’s look at a request for a token where a client submits credentials.

$credentials = $request->only(’email’, 'password’);
$token = JWTAuth::attempt($credentials); 
// handle if false
// return a response with either json, http header, or a cookie

Under the hood, attempt() method looks like:

function attempt($credentials, $customClaims){
  //validate credentials
  if (! $this->auth->byCredentials($credentials)) {
    return false;
  }
  // generates token from User
  ...
}

The attempt function does two things:

1. Validates credentials
2. Generates token from User

In step 1, the authenticator uses an adapter ($auth attribute) for the Laravel AuthManager to call byCredentials. If the credentials fail, it returns false.

In step 2, the authenticator retrieves the user from the same AuthManager adapter ($auth) to retrieve the user, and generates a token for that user.

You can review the byCredentials() method in the IlluminateAuthAdapter.

public function byCredentials($credentials) {
  return $this->auth->once($credentials);
}

Here, the IlluminateAuthAdapter’s $auth attribute is the actual Laravel AuthManager. The package is calling the once() method on the AuthManger. which will set the user on the manager.

Therefore, the JWTAuth authenticator can now generate a token by calling the fromUser() method and passing through the user on the $auth adapter.

public function attempt($credentials, $customClaims){
  //validate credentials
  ...
  // generate token from User
  return $this->fromUser($this->auth->user(), $customClaims);
}

The auth adapter, as well as the other adapters listed in the providers array on the jwt.php config, are good sources of intel for how the package actually interfaces with the Laravel framework.

After getting the token through the attempt() method, we can then return it to the user through our preferred method (JSON, HTTP Header, cookie).

Authentication

Here’s a simple flow to parse the the token and validate a user.

// parse token
// authenticate
// refresh if necessary
// additional validation
// continue with request

Parsing The Token

Unfortunately, many of authenticator’s API methods require a $token as a argument. By default, the package will not retrieve the token from the HTTP header, cookie, or query string when you call it’s methods. Instead, you need to retrieve it yourself and pass it in to the JWTAuth’s methods.

Luckily, if you use the HTTP Header or query string options (described above), the JWTAuth component gives us a helper parseToken() method which we can use to extract the token and set it on the authenticator. (I’ve listed the default values for the parameters above the function for readability)

// $method = 'bearer'
// $header = 'authorization'
// $query = 'token'
public function parseToken($method, $header, $query) {
  if (! $token = $this->parseAuthHeader($header, $method)) {
    if (! $token = $this->request->query($query, false)) {
      throw new JWTException(
        'The token could not be parsed from the request', 
         400
      );
    }
  }
  return $this->setToken($token); 
  // returns $this
}

The getToken() method will either call parse token and set it on the JWTAuth authenticator or will return the token we’ve already set.

Let’s update our flow.

$token = JWTAuth::getToken();
// authenticate
// refresh if necessary
// additional validation
// continue with request

Authenticate

After parsing the token, we can authenticate the token which will perform some validations. On the actual token, it will ensure it is in the right format (aaa.bbb.ccc).

On the actual payload (when the data is decoded from the token), it will ensure that the token signature matches the payload, the payload contains all the required claims as dictated in the jwt.php, and the timestamps of exp (expiration), iat (issued at), and nbf (not valid before) are valid. These three timestamps are handled automatically by the package.

$token JWTAuth::getToken();
$user = JWTAuth::authenticate($token);
// refresh if necessary
// additional validation
// continue with request

Refresh

Refreshing a token is an important part of our authentication flow as expiration (exp) values are relatively short. Fortunately, the package throws specific exceptions to handle this case. When a token’s expiration has passed, we can catch a TokenExpiredException, and perform a refresh.

The downside to refreshing tokens is that we have to deliver the new token to the client.

If you wish to refresh a token, you have a few options.

1. You could stop performing the request, return the new token, and require the client to resubmit the request.

$token JWTAuth::getToken();
try {
  $user = JWTAuth::authenticate($token);
} catch (TokenExpiredException $e) {
  $newToken = JWTAuth::refresh($token);
  throw new HttpResponseException(
    Response::json(['token' => $newToken, 'msg' => "Have this new token"])
  );
}
// additional validation
// continue with request

2. You can refresh the token, continue with the request, and attach the new token to the response.

$token JWTAuth::getToken();
try {
  $user = JWTAuth::authenticate($token);
 } catch (TokenExpiredException $e) {
  $token = JWTAuth::refresh($token);
  JWTAuth::setToken($token);
  $user = JWTAuth::authenticate($token);
}
// additional validation
// continue with request
// add new token to response

As you can see, we reset $token to equal the new token’s value, set the new $token on the JWTAuth authenticator, and reauthenticated with the new token. In addition, we would need to pass the new $token back to the client as the current token would be invalid.

There are two obvious ways to pass back the token. In the controller, you can just add it to the response as JSON, HTTP Header, or cookie. If you are performing this logic in middleware, you can attach it to the response at the end of the request as “after” middleware.

This is an okay approach, but in reality, the token may not be able to be refreshed because the refresh_ttl may have passed so we would have to handle a second TokenExpiredException inside the catch block.

$token = JWTAuth::getToken();
try {
  $user = JWTAuth::authenticate($token);
} catch (TokenExpiredException $e) {
  try {
    $token = JWTAuth::refresh($token); // might fail
    JWTAuth::setToken($token);    
    $user = JWTAuth::authenticate($token);
  } catch(TokenExpiredException $e) {
    //token cannot be refreshed, user needs to login again
    throw new HttpResponseException(
      Response::json(['msg' => "Need to Login Again"])
    );
  }
}
// additional validation
// continue with request
// add new token to response

Custom Validation

It is important to understand that besides checking for required claims as stated in our config file, the authenticate method will not perform any validation on custom claims. Any additional claim information will need to be validated separately.

Sometimes, we may want to require additional validation. For example, we could ensure that a custom claim, such as ip, could match the current request.

$token = JWTAuth::getToken();
try {
  $user = JWTAuth::authenticate($token);
} catch (TokenExpiredException $e) {
  try {
    $token = JWTAuth::refresh($token);
    JWTAuth::setToken($token);    
    $user = JWTAuth::authenticate($token);
  } catch(TokenExpiredException $e) {
    //token cannot be refreshed, user needs to login again
    throw new HttpResponseException(
      Response::json(['msg' => "Need to Login Again"])
    );
  }
}
$payload = JWTAuth::getPayload($token);
if ($payload->get('ip') === $request->ip()) {
  // we have our $user
  // we have validated the token
  // continue with request
}
return 403 / 401 / 422

We know have some boilerplate for our primary two user cases. There is so much more I can go into with the Tymon package. I’ll talk some more about the package’s architecture below, but if you want some great boilerplate, I would recommend checking out the documentation.

Events

There are a few events to know about.

// fired when the token could not be found in the request
Event::listen('tymon.jwt.absent');

// fired when the token has expired
Event::listen('tymon.jwt.expired');

// fired when the token is found to be invalid
Event::listen('tymon.jwt.invalid');

// fired if the user could not be found (shouldn't really happen)
Event::listen('tymon.jwt.user_not_found');

// fired when the token is valid (User is passed along with event)
Event::listen('tymon.jwt.valid');

The package does not ship with any default listeners or subscribers so it is up to you to handle these events if you choose to.

Middeware

Tymon offers two out of the box middlewares for our applciation.

tymon.refresh

The refresh middleware will refresh the token on each request and attach a new Bearer header to the http response. This middleware only affects the response as it is “after” middleware. Therefore, it will not refresh the token before the request is processed. You won’t be able to access this refreshed token within the application. In addition, this means that each token will only be valid for one request.

jwt.auth

The auth middleware calls the authentcate() method on the JWTAuth authenticator and sets it on a $user. It then fires a ‘tymon.jwt.valid’ event with the $user as an argument. If desired, you could catch this event in a Listener and save it for later use. How you choose to save it is up to you. For example, you could have a listener with the injected AuthManager and save it there.

But be careful saving the $user to the AuthManager. The AuthManager has a magic method which allows you to actually call on the default/active guard (which is usually is the SessionGuard). When the JWTAuth authenticator calls byCredentials() on the IllumateAuthAdapter, it is actually storing $user on the SessionGuard. This is fine, but it also makes it very easy to overwrite the $user.

Documentation

Check out the documentation.

Digging Deeper

Adapters

In the config, you will notice that the providers array contains a list of adapters. An adapter is an object oriented design pattern that allows two classes to interact with each other. Each adapter on the provider array injects the target object (for example the AuthManager or CacheManger), so that the package can interact with it using the package’s desired API.

Storage Adapter

If you look at the IlluminateCacheAdapter, you can see that it implements tymon’s StorageInterface. This Interface dictates all the methods that the tymon package needs to be able to call. The adapter itself, to be compatible with this interface, injects laravel’s cache manager within the service provider, and uses it to implement all the required methods.

I mention the storage adapter for three reasons.

First, it is important as it will be used to store blacklisted tokens.

Second, it provides the best use case for explaining how these adapters work and can be customized. For example, if you wanted to store blacklisted tokens in a database, you could write an illuminate database adapter that injected the database manager and implemented the StoreInterface. If you do decide to do something similar, remember to alter the ServiceProvider (for example the registerStorageProvider method) in order to provide the correct implementation.

Third, you can see that it injects the CacheManager. Not the specific FileStore, DatabaseStore, or RedisStore. The actual manager. Therefore, whatever cache implementation you are using by default with laravel, will be used in the package without any additional configuration.

JWT Adapter

The Tymon package provides many objects for us to interact with such as Token, Payload, PayloadFactory, and the JWTAuth authenticator. However, the actual cryptographic checks and mutations performed on the token are handled by the NamshiAdapter. The adapter allows the package to interface with the namshi JWS package (Namshi\Jose\JWS). So while we may call decode() on the JWTAuth authenticator, at it’s core, the package is reading the token using the Namshi adapter.

Auth Adapter

We discussed the Auth adapter sporadically throughout the post. I’ll reiterate that the Auth adapter injects the AuthManager which is uses to authenticate users by either credentials or by an id using the once() and onceById() methods respectively. In addition, it also provides a user() method to retrieve the user. However, the user() method is only effective if the user has been authenticated either through the byCredentials/once or byId/onceById methods, and only as long as the $user attribute on the SessionGuard has not been overwritten.

User Adapter

I won’t say too much about the user adapter or provider. In short, it is an adapter used to create the eloquent user is fairly small.

Validation

Validation of the current token’s claims actually occurs when the JWTManager decodes the token and get’s the payload. This method is called in the JWTAuth authenticator toUser() or authenticate() methods. The JWTManager’s decode method actually implements the NamshiAdapter ( the $jwt attribute) to get an array of the values.

public function decode(Token $token) {
  // the namshi decode() method verifies signature
  $payloadArray = $this->jwt->decode($token->get());
  // uses payload factory to create Payload object
  
  //checks blacklist
  return $payload;
}

The factory then creates a Tymon Payload object through the PayloadFactory.

public function decode(Token $token) {
  // the namshi decode() method verifies signature
  ...
  // uses payload factory to create Payload object
  $payload = $this->payloadFactory>setRefreshFlow($this->refreshFlow)->make($payloadArray);
  // checks blacklist
  ...
  return $payload;
}

When the factory makes a Payload it passes in a validator (Tymon\JWTAuth\Validators\PayloadValidator).

public function make(customClaims = []) {
  $claims = $this->buildClaims($customClaims)->resolveClaims();
  return new Payload($claims, $this->validator, $this->refreshFlow);
}

On construction, the validator will then perform the actual validation (timestamps, required_claims) as well as verifies the token’s signature.

Structure:

protected function validateStructure($payload) {
  if (count(array_diff_key($this->requiredClaims, array_keys($payload))) !== 0) {
    throw new TokenInvalidException(
      'JWT payload does not contain the required claims'
    );
  }
  return true;
}

Timestamps:

protected function validateTimestamps($payload){
  if (isset($payload['nbf']) && Utils::timestamp($payload['nbf'])->isFuture()) {
    throw new TokenInvalidException(
      'Not Before (nbf) timestamp cannot be in the future',
      400
    );
  }
  if (isset($payload['iat']) && Utils::timestamp($payload['iat'])->isFuture()) {
    throw new TokenInvalidException(
      'Issued At (iat) timestamp cannot be in the future', 
      400
    );
  }
  if (Utils::timestamp($payload['exp'])->isPast()) {
    throw new TokenExpiredException('Token has expired');
  }
  return true;
}

Refresh:

protected function validateRefresh($payload){
  if (isset($payload['iat']) && Utils::timestamp($payload['iat'])->addMinutes($this->refreshTTL)->isPast()) {
     throw new TokenExpiredException(
       'Token has expired and can no longer be refreshed', 
       400
     );
  }
  return true;
}

Like Ogres, the Tymon package has layers.

What we didn’t cover

I omitted much of the code referencing custom claims. If you look at the source code for the JWTAuth authenticator, you’ll find that passing in custom claims greatly affects the package’s authentication parameters.

Alright

For better or worse, I’ve thrown a lot at you. I did my best to expose some of the moving parts within the Tymon jwt-auth package while also giving a you a simple authentication flow. In the next section, we’ll build a custom JWT Guard.

Part One: JWT

Follow me on twitter? Maybe? @3lpsy

Feel free to comment down below or reach out to me on twitter if you have any comments, corrections, or suggestions for how I can improve this post.