Azure Service Bus, Azure Functions and Urban Airship at work together for an e-commerce website
The article shows how to implement an architecture where Azure Service Bus receives messages to signal a product going out of stock or coming back into stock, which would trigger an Azure Function to send push notifications to clients that subscribed to ‘back-in-stock’ events for the affected products.
A modern e-commerce website based on the microservices architecture is probably powered by a variety of different services and APIs: Promotions, Customers, Product Catalog, Search, Checkout, Recommendations, Order Processing and potentially many more pieces. These services can communicate between each other by direct API calls, if they need an immediate feedback, or they can use a messaging-based system implementing the Publisher-Subscriber pattern, where they basically send a message and then zero, one or multiple subscribers can be notified and react to the message in different ways. In the Microsoft Azure cloud, the messaging systems are Azure Service Bus, Event Hub and Event Grid (see here for a comparison), and listeners can be WebJobs, Console apps, Azure Functions or other types of applications.
The diagram below offers a simplified and high-level overview of how some of these pieces can work together to implement a feature that allows customers to receive push notifications when an out-of-stock product comes back in stock:
This article focuses on showing how to set up and implement the bits around sending messages to Service Bus and the Azure Function that is triggered by those messages, which then sends a push notification through Urban Airship.
1 — Setting up Azure Service Bus with a Topic and a Subscription
I won’t cover this step by step, because this article on MSDN does a great job already. Suffice to say, I created a new Azure Service Bus resource, created a Topic named ‘productstockchanged’ and a subscription for it named ‘BackInStockNotifications’. (Since I wanted to test how things work with multiple subscriptions, I also created a second, named ‘UpdateCatalog’. This is an extra step and not needed for the scenario being implemented here). Here’s what the final state looks like:
Reminder: Topics are for when you might need multiple listeners to receive and process an event independently, while Queues are for when there will be only one processor. As shown by the diagram at the beginning, our ‘stock changed’ messages would be processed by at least a couple of services (one that sends out push notifications and one that updates the ‘in stock’ information on the Product Catalog database), so a Topic was required.
In reality, it would probably be more accurate to say that a Queue is for commands that someone sends to a recipient, and the recipient is the owner of the message. A Topic instead implements the actual Pub-Sub pattern, where the publisher is the owner that broadcasts a message without knowing who are the listeners that will pick it up.
2 — The Console app that sends messages to the Topic
In the real implementation it would be the Stock microservice that sends messages to the Service Bus Topic once a product goes out of stock or is back in stock. For this demo however, I created a simple Console app that sends messages according to comments entered on Terminal.
Microsoft.Azure.ServiceBus is the NuGet package that must be added as a new dependency:
The code is quite simple: a TopicClient object is instantiated with the Service Bus connection string and the Topic name in input, and its SendAsync method is used to send a new Message object, which is created with the json produced by serialising a ProductStockChanged object. ProductStockChanged contains the ID of product affected by the change, and a boolean indicating whether the product is now in stock or out of stock. An additional attribute that I could have added is the product title, but I decided to keep things simple.
New messages are sent by sending commands like ‘instock 123’ or ‘outofstock 123’ (where ‘123’ is the ProductID) from the command line. Here’s the complete code:
If you now run this, send a few sample commands, and then reload the Topic page on the Azure Portal, you’ll see that the ‘Message Count’ for the subscriptions created previously was increased because there are now messages there waiting to be picked up and processed.
3 — Setting up Urban Airship for web push notifications
There are plenty of good services to send push notifications for iOS, Android and Web, but I chose Urban Airship (UA) because it’s well documented, has a nice UI for creating messages (I used the UI during some initial testing, before switching to the API) and a good client-side SDKs and RESTful API. However, other services work just as well (Microsoft itself offers Notification Hubs, and I have also successfully used OneSignal previously), and the overall solution would stay exactly the same.
For this demo, I chose to send web push notifications rather than notifications to an Android or iOS app, because I integrated push notifications on many native mobile apps in the past but never tried it on web and I felt it was easier to set up for a demo project build from scratch. I was right in thinking this — while it takes a few steps for web, it takes longer to create a dummy native app and deal with push certificates. However, the beauty is that since we’re using UA we’re abstracted away from the platform-specific implementation — we just ask UA to send a push notification and it does it for all registered/configured platforms. This means that if we wanted to add support for iOS or Android later, there wouldn’t be code changes.
So, I created a free account, and set up a web-based project: you choose a title, default action url (this is where the user would be redirected to if they click the notification) and icon url, and you’re ready to go: Urban Airship will now let you download a zip file, which contains a JS file and a snipped to add to your front-end website. More on this later…
As we are going to interact with UA via its API, I accessed Settings / APIs & Integrations and grabbed the AppKey and App Master Secret, which are required to authenticate the calls as you’ll soon see in the Function implementation.
4 — The Azure Function that sends the notifications
It’s time to create the ‘core’ of the solution, which is the Azure Function that actually sends the push notification when it gets triggered by new messages for the productstockchanged Topic on the BackInStockNotifications subscription. I created a new Azure Functions resource and created a new function from the ‘Service Bus Topic trigger’ template, as shown below:
Should you need to modify the trigger’s settings after the function has been created, visit the Integrate page:
When a new message is posted to Service Bus, the Azure Function is instantiated and its Run method is invoked. Run has a string input parameter, which is the json of the Message created previously, that contains the ProductId and IsInStock attributes. The json body is deserialised into a ProductStockChanged object, and, if its IsInStock property is true, the UA API is used to send a push notification to all clients that have expressed an interest in knowing when that particular product is back in stock.
How do we target only those customers/clients and not everyone though? A quick solution is using ‘tags’: a client registers for a tag named notify-instock-{product id here}, and the push request has a parameter indicating that we’re only targeting clients with that tag associated to them. A client could of course also unregister for that tag later on, and therefore stop receiving notifications for some products.
The actual UA API call is done by doing a POST request through a HttpClient object, which has an Authorization header with the UA credentials, and takes in input a json body with the text of the notification and the audience’s tag. (Targeting can be much more advanced, for example, you can combine multiple tags using AND or OR; can target only specific device types, and can schedule a notification for a specific time. Read more here.)
The following code should be self-explanatory:
The Urban Airship credentials needed to call the UA API should never be hardcoded in the script, but be in app settings and read through the environment variables, or in Azure Key Vault.
Note the #load statement in the first line, which loads an external file with the ProductStockChanged class definition. This was created from the ‘View files’ tab displayed below:
Please note: I initially created and tested the Function locally with Visual Studio, but then I manually copied/pasted the code into the online editor because the ‘Publish to Azure’ wizard was not available on my installation for some reasons. I’m using macOS, but I guess you’d have a better experience with Visual Studio on Windows though.
Refer to this page about Azure Functions Core Tools to find out more about the local development experience. Developing the Function locally and then deploying to Azure means that you’ll have a Function based on a precompiled library rather than on a C# script, and also means you’ll be able to more easily reference other libraries/namespaces as you would do normally. In order to use this option however you can’t create the Function from the portal.
Refer to this page to see how a new Function can be created through the Azure CLI rather than the portal.
Once the Function is complete, go back to your console app developed previously, and send a ‘instock 123’ command. The log window on the Azure Function should display some lines confirming the Function was invoked by the Service Bus Topic trigger and some custom logs. If you then switch to the Urban Airship’s Activity log, you should see a new push notification sent for the notify-instock-123 tag.
The above log is the only way to verify that the UA call is successful so far, because the client web page hasn’t been implemented yet.
This article by Zhongming Chen, published on the ASOS Tech Blog, has some very good notes about Azure Functions, so take a look if you’re planning to use them.
5 — The web front-end
The last step involves creating the client app, which is a website or, rather, a single web page for this demo. In a real-world implementation there would be product listing pages and product details pages getting data from something like a Product Catalog API, and, as part of the product attributes returned by the API, there would be something like an ‘isInStock’ attribute. When that’s false, the customer might click a button to register their interest for the product, and the notify-instock-{productId} described before would be associated to them.
For the purpose of this article however, the simplest, and ugliest page — with a static list of three product names and a ‘Notify’ checkbox on their right-hand side will do just fine. The UA client-side SDK, as well as the initial configuration, is fully explained on this page, but here’s the gist:
- Download the zip with the JS files from the UA configuration page and upload the push-worked.js file on the root folder of the website.
- Make sure SSL is enabled on the website/domain (you can use CloudFlare to do this for free and with a couple of clicks, if you don’t have this already). If this and the previous step are not done correctly, you risk wasting a lot of time figuring out why the implementation doesn’t work…
- Create the HTML page, which:
1) imports the JS snippet that loads the Urban Airship’s JS SDK.
2) when the SDK is loaded, checks the isSupported and canRegister properties, and, if they’re true, calls the register function to have your browser ask the customer for the permission to send push notifications.
3) handles the click on the checkboxes to add/remove the notify-instock-{productId} tag.
Here’s the full HTML/js code:
When the page loads for the first time (or even subsequently, if you just cancel the prompt for the notifications) this is what you should get:
Click ‘Allow’, and then tick the ‘Notify’ checkbox for one or more products:
Finally, use the Console app to send a ‘instock 123’ command. If everything is set up correctly, you should get a notification like this on Chrome (even if you’re on another page of course):
On a Mac, the notification will also be shown on the system-level Notifications panel on the right-hand side of your desktop:
Please note: don’t despair if you don’t immediately get a notification. It sometimes takes a few minutes to show up, after you see the log on the Urban Airship console.
Urban Airship and the other third-party providers abstract a lot of details. If you want to better understand how the Web Push Protocol works behind the scenes, start from this page.
By using third-parties, it spares you the need to create all the infrastructure to record the IDs of the devices that register to get the notifications (ie: a DB with a table of device IDs, their tags etc.) and all the work to send the notifications to all targeted devices. After you make a single call to Urban Airship targeting a tag, it might mean 1M calls that UA does on your behalf, one per device, if there are 1M users/devices registered for that tag. You can easily see how not having to worry about that is good.
EXTRA — Adding a filter for the BackInStockNotifications subscription
When you create a new Azure Service Bus Topic subscription through the Azure Portal, you won’t have the option to set up a ‘filter’, which means that all new messages for that Topic will be copied into the subscription. However, you might only want to process a sub-set of messages, not all. In the scenario described in this article, only messages with ‘IsInStock = true’ need to be processed, assuming that customers only want to be notified when something they liked is back in stock…and they don’t need to be notified about something going out of stock (that might be a reasonable option as well in reality, but not for the purpose of this demo).
The way it works so far is that there’s an if statement when the Function starts and the json data is decoded into a ProductStockChanged message. While this works, it’s not optimal, because it means the function is going to be executed plenty of times for messages it doesn’t need to process — this means wasted compute time, which in turns means additional instances being necessary and additional cost. A better solution is to create a subscription with a rule/filter, so that only messages matching certain conditions are copied into it for further processing. It’s not possible to create the filter directly from the Portal unfortunately, but luckily, it’s simple enough to do it programmatically (in a method that you could execute from a management console app or something similar). Here’s the method that modifies the existing subscription by adding a SqlFilter for ‘IsInStock = true’:
The SqlFilter can refer to attributes defined in the message’s UserProperties array, not on attributes defined directly in the json (because the message’s body could be plain text, json, xml or anything else)…so now line #67 of the Console app’s code displayed in Section 2 of the article should be clearer.
After executing this code, use the Console app to send some ‘outofstock {id}’ commands, and you’ll see that they’ll end up in the UpdateCatalog subscription (the second subscription I had created at the beginning, with no filter), but not in the BackInStockNotifications subscription. A much better result. (I still left the check in the Function itself as well though, just in case the filter doesn’t work or is removed by mistake.)
An even better solution that doesn’t require executing custom managed code as part of your deployment/configuration pipeline is using the Azure CLI as described here.
Possible further enhancements
This was meant to be a POC, but in a real implementation you will very likely want to consider the following aspects when sending out push notifications:
- Support multiple languages: this might be as easy as targeting the notifications with a language tag (eg: lang_fr) in addition to the notify_instock_{productId} tag. The Azure Function would call UA once for every language (so five times if you support five languages), but a customer would only get one notification because they would only be registered for a single language tag. Some third-parties also allow the use of ‘templates’, where the body/content of the notification is dynamically injected by the provider itself according to the language or other conditions. The technique above, albeit more primitive, should work with all though.
- Enrich the notification with the name of the product, price or other information. As mentioned previously, this info could be part of the Service Bus Message itself (which would be super simple to add), or otherwise could be retrieved dynamically by calling another API, such as the Product Catalog API. Though there would need to be an additional caching layer, the Product Catalog API would potentially need to be scaled up to support the additional load.
- Currently, the notification always opens the homepage of the website when clicked because the default action url defined at the project configuration time wasn’t overridden on the specific notification. It would be better instead to add a url/deeplink to the notification, so that the product details page is loaded when the notification is clicked. All third-party providers allow this as it’s a basic requirement, but of course the syntax/attributes on the input json vary by provider (each has its own proprietary json, since they abstract the real content of the lower-level platform-specific notifications).
Conclusions
That’s all folks. Albeit quick and simple to set up, the proposed implementation is quite powerful and flexible, as it’s based on loosely coupled components (the checkout and stock microservices know nothing about the Function that handles the stock-related messages to send push notifications) and it offers great scalability (the Azure Function will be scaled up automatically according to how many messages it has to process). Last but definitely not least, the use of Service Bus offers a better overall reliability of our system, because if the delivery of the notification fails for any reason (eg: Urban Airship is down), the Function fails, the message goes back into the queue and it will be processed again later. Hooray Azure!
Who am I / what do I do? I proudly work as a Solutions Architect in the Mobile Team @ ASOS.com (iOS app | Android app), and we’re always looking for strong, friendly and talented developers that want to have an impact on how customers shop online. ASOS is the biggest online-only retailer in the UK and, let’s be real, the best tech and fashion company in the world. Some of the technologies we use are Swift for iOS, Kotlin for Android, React and Node on the web front-end, .NET and Azure on the back-end. If that sounds interesting to you, and you happen to live in beautiful London (or are willing to move here — after all, it’s the best city in Europe, except for some in Italy!), do get in touch!
