More Open Banking Message Signing

Published in

Syntaxa Tech Blog

5 min readDec 18, 2018

Tripping over Base64 encoding and implications for signing.

Context

This is a follow up post to Open Banking Message Signing where I covered the specification requirements for non-repudiation on the Payment Initiation APIs. It purposely ignored the difficulties we faced implementing the mandated Base64 header claim, in favour of covering the gaps in the existing tooling for critical header parameters. This post is focused on the Base64URL encoding and the potential pitfalls.

JSON Web Signatures

As we covered in the previous post, the non-repudiation requirements are met through the use of JSON Web Signatures (JWS), standardised in RFC 7515. This standard has a number of extensions to increase its utility; the one I focus on here is RFC 7797 — Unencoded Payload Option.

Why not encode?

A couple of years ago, it was deemed that for a number of potential use cases, the vanilla JSON Web Signature standard would not suit if (amongst others):

The signing payload was rather large and Base64URL encoding would add substantial computational overhead
The signing payload would not be transmitted with the signature (the detached model (as Open Banking has adopted)) thereby negating the need to Base64URL encode

RFC 7515 explicitly states the algorithm for encoding the input for signing in Section 5.1 as:

Compute the JWS Signature in the manner defined for the particular algorithm being used over the JWS Signing InputASCII(BASE64URL(UTF8(JWS Protected Header)) || '.' || BASE64URL(JWS Payload)).  The "alg" (algorithm) Header Parameter MUST be present in the JOSE Header, with the algorithm value accurately representing the algorithm used to construct the JWS Signature.

When it comes to standards you cannot just do something different (that would be anarchy!) so Michael B. Jones created a new standard that extended the initial JSON Web Signature RFC to modify this approach through the use of a new JOSE Header parameter, b64.

Open Banking Specifics

The Open Banking standards make use of this new JOSE Header parameter and enforce it is set to false (i.e. signing input must not contain a Base64URL encoded payload element).

This is achieved by including the b64 header with a false value — Open Banking additionally marks the b64 parameter as critical.

The JOSE Header portion of the signature input looks akin to:

{
  "b64": false,
  "http://openbanking.org.uk/iat": 1543587262,
  "crit": [
    "b64",
    "http://openbanking.org.uk/iat",
    "http://openbanking.org.uk/iss"
  ],
  "kid": "nWNjoBVmFEhkEI-YPmgOXlTniTU",
  "typ": "JOSE",
  "http://openbanking.org.uk/iss": "C=GB, O=OpenBanking, OU=0015800001ZEZ3WAAX, CN=4tHCFYzhmRTp5ed7Tr5IN6",
  "alg": "RS256"
}

Implementing the parameter

Having been unable to make use of tooling such as https://jwt.io (covered in the previous post) and now being overly cautious given our experience of critical header parameters we have built tests covering:

Nimbus based JWS signing
Nimbus based JWS verification
Handcrafted JWS signing (to enable output comparison with Nimbus)
Handcrafted JWS verification (to enable output comparison with Nimbus)

We found our tests repeatedly failing.

We took for granted that the library we are using for JSON Web Token and JSON Web Signatures (Nimbus JOSE+JWT) supported the extension standards. We had naively expected that the library would inspect the JOSE Header component of the signing input and skip Base64URL encoding of the input payload.

Digging Deeper

Running the debugger and stepping through the internals of Nimbus, we found that it was hard coded to Base64URL encode the payload into the signing routine. We then found an interesting ticket on their bug tracker from August 2018 (https://bitbucket.org/connect2id/nimbus-jose-jwt/issues/273/support-rfc-7797-b64-header). Coincidentally this was raised by a member of the Open Banking Slack community, Nicholas Irving.

The statement in question is line 120 in JWSObject.java

signingInputString = composeSigningInput(header.toBase64URL(), payload.toBase64URL());

Whenever you instantiate a new JWSObject, the library by default will store the payload as a Base64URL encoded value.

Can I work around it?

The short answer is, “yes”.

After initially despairing, the team sat down and discussed options; broadly speaking we settled on either:

Raising a PR against Nicholas’s issue to implement RFC 7797 — risky as we had no appreciation for the full code base of Nimbus neither their build, test and release cycle
Subclass the necessary classes to work around the hard coding in the class’s constructor — horribly brittle to a version bump but also the requisite fields are private (so no subclass access)
Attempt to integrate at a lower level to bypass the constructor hard coding — worth exploring
Find a library that supports the RFC natively — Nimbus is used throughout the organisation so we would prefer to maintain some consistency and whilst the switching of a library should be straight forward, there are always implications

We therefore decided to push ahead with (3).

Just go direct

It is possible with Nimbus to use the Signer and Verifier classes directly if you know which to instantiate. Open Banking currently only supports the RS256 signing algorithm (RSA with SHA-256) until market adoption of PS256 improves. In Nimbus, both are implemented in the RSASSA* class pair (RSASSASigner and RSASSAVerifier).

Both the signing and verification methods take a JWSHeader and the signing input (as a byte array). Consulting RFC 7515 Section 5.1 and augmenting with RFC 7797 Section 3, the signing input is as follows:

ASCII(BASE64URL(UTF8(JWS Protected Header)) || '.' ||       BASE64URL(JWS Payload))

If we modify the code from my previous post we end up with the following functions (I have again removed the validation and utility methods for brevity).

Updated code to create a detached JWS

Updated code to verify a detached JWS

Conclusions

Whilst this post shows that it is possible to create RFC 7797 compliant JSON Web Signatures with Nimbus JOSE+JWT, I would not recommend it. The implementation I describe exposes the signing algorithm to the application developer. Libraries should be used to abstract the complexities in implementation from the average developer. Whilst this approach aids with the asymmetric cryptography, anyone maintaining the approach above must understand how the signing input is generated (it is documented but is deep in the requisite RFCs).

Developers would be far better to consider alternative libraries. Having spoken with the team over at RAIDIAM, they recommend investigating Jose4J by Brian Campbell at Ping Identity. Reviewing the BitBucket README shows explicit support for RFC 7797.

My experience building Message Signing leaves me with reservations as to how sensitive it is to the various JSON (de)serialisation routines on both sides of the Open Banking interface — additional white space and newline characters will invalidate the signature during verification. I believe this is exacerbated by the methodology outlined above but still exists by the very nature of detaching the payload from the signature.

Open Banking uses JSON Web Tokens (which are signed using the JWS standard) as part of the Authorisation Server redirect (in the form of an Open ID Connect Request Object). The Request Object is an attached JWS, therefore the payload is encoded exactly as it was during signing so the chances of it failing due to white space and newline differences is almost zero. In production, I am not aware of failures to verify the signature due to serialisation — only to the “standard” mistakes; key mismatches, expiring tokens and issued times in the future.

Using attached JWS is not the answer either — generating a signature for file uploads (which can be multi-megabyte) will cause concern amongst all participants in Open Banking. I believe a standard (even draft) such as HTTP Signatures would be worth considering and debating within the community.

I would like to hear from others in the Open Banking eco-system on how they are approaching message signing. I posit that many will generate the signature as part of their application rather than, as originally envisaged, externally in an API Gateway or similar componentry. I believe this will lead to differences in the textual representation of the JWS payload and the HTTP body, resulting in signature verification failures and concerns as to how strong non-repudiation really is for participants.