Shopify webhooks: HMAC validation on NodeJS Express
Shopify apps do HMAC validation to ensure that the request received by them has actually originated from the Shopify platform. Shopify uses a secret key to generate an HMAC of the request body and sends it along with the request. The app that receives the callback should extract the request body and use the same secret key to generate an HMAC hash on its side. If the HMAC received along with the request matches the HMAC generated by the app, the request is deemed an authentic request from Shopify. The security here relies on the assumption that the secret key is known only to the app and the Shopify platform. Hence the advice — Do not share your API secret key with anyone.
Nothing prevents a shopify developer from creating an app that skips this validation before responding to requests. But such an app might end up divulging information to an unauthorized third party. Therefore, embracing the dark side and choosing not to validate an incoming request is a serious security threat.
Shopify apps should do HMAC validation on 2 types of incoming requests
- Install callback before doing access token exchange
- Webhook callbacks
I will be discussing only the webhook callback verification with a focus on NodeJS. I struggled quite a lot before I got it set up. I came across the following hurdles on the way.
1. Getting hold of the shared secret
Shopify’s OAuth documentation on HMAC verification makes clear reference to a secret key that is to be used for HMAC validation. This refers to the API Secret Key visible upfront in the app’s partner portal in the same name. On the other hand, the webhook verify documentation makes reference to a mystical shared secret in multiple places.
This term — shared secret — added an hour or so to my ordeal of setting up webhooks. I scoured through a lot of documentation, the entire partner portal, and community posts before I found in one of the posts that the elusive shared key is none other than the same old API secret key.
The shared secret key mentioned in the webhook verify documentation is one and the same as your API secret key in the partner portal of the app.
2. Calculating HMAC
The first mistake I made here was in assuming that the HMAC evaluation for a webhook callback is the exact same as the HMAC evaluation for the OAuth callback. The below note in the OAuth documentation did not catch my eye until long after I had figured it out
My assumption was correct, except for a very subtle detail that took an excruciatingly long time to figure out. The install callback, sends HMAC in a query parameter, encoded as a hex value whereas webhook callbacks send HMAC in a response header, encoded as a base64 value. This point is casually mentioned in the Shopify document about verifying webhooks but the code examples given there do not include NodeJS.
In the install callback, shopify sends a hex encoded HMAC where as in a webhook callback, it sends a base64 encoded HMAC.
The NodeJS code for evaluating HMAC for install callback needs to be tweaked as shown below to make it work for verifying webhook callbacks
3. Input message for HMAC
The final detail that took me the most time to figure out was the message upon which HMAC needs to be evaluated. From the code examples in the documentation, I understood that it is the request body that needs to be passed as the message. The issue here was that it was not working in my NodeJS/Express server
Attempt 1 — I read a couple of community posts claiming that the message to be hashed on NodeJS was the stringified JSON response body. Armed with confidence from these posts, I used the Express body-parser to parse the request body as a JSON and then stringified this JSON and generated its HMAC hash. Oh, the naivete.
Attempt 2 — I then came across posts suggesting that the raw request body straight off the network(without any intermediate parsing by middlewares like body-parser) is what gets the job done. This made sense to me as I knew that the generated hash can vary drastically even if a single character is off. That is when I set off on a quest, or rather a sub-quest, to retrieve the coveted raw request body. Soon enough, my sub-quest proved to be a daunting task to execute in Express. Koa framework has a body parser that does this out of the box. But in the Express body-parser, I could only get either JSON or raw body. I needed both here — JSON for my underlying middlewares and raw for HMAC verification. Solutions to getting both of them together seemed non-existent. Later, I found posts by some StackOverflow veterans, who had solved the problem some years back, by manually compiling the raw request from incoming chunks. But this solution(as shown below) did not work for me
Attempt 3 — Finally as I was on the verge of giving up on my sub-quest, my quest, and my career as a web developer, deliverance came from the Express body-parser itself where I found a benign verify option in the JSON parsing section that calmly advertised its willingness to provide me access to the raw request body.
The attempt I made with the verify option in body-parser bore fruition and the below code found residence in my codebase.
After the raw body was extracted and stored in req.textBody, the final code for evaluating HMAC for incoming webhook callbacks looked like this
HMAC validation for shopify webhook callbacks should be done on the raw request body straight off the network before passing through any body parsers
And so, with a panache of irony, the problems that were kicked off by a lacking verify documentation got resolved by the verify option of a package that resided in my app from its very beginning.
4. Responding to mandatory webhooks
The mandatory webhooks are for GDPR compliance. The shop/redact webhook is applicable to every Shopify app. Upon receiving this callback, the app has to respond back with a 200 immediately and eventually purge all info about the store from its servers.
The customers/redact and customers/data_request are mandatory webhooks too but they may or may not be invoked for an app depending on the scopes that the app asks for. customers/redact and customers/data_request webhooks will only be invoked for apps that ask for any of the order or customer-related scopes. But irrespective of whether your app asks for these scopes or not, a callback URL must be registered for these. These webhook callback URLs need not do anything but respond with a status 200.
customers/redact and customers/data_request webhooks will only be invoked for apps that ask for any of the order or customer related scopes. But irrespective of whether your app asks for these scopes or not, a callback URL must be registered for these.
Originally published at https://kubric.io/blog on March 21, 2020.
