Sending Real-Time Push Notifications with AWS IoT
In modern-day software systems, real-time communication can be observed often. Software systems have become the backbone for businesses and enables them to thrive.
Real-time notification systems bridge the gap between businesses and customer facing applications. This article focuses on how these software systems can leverage AWS IoT to send real-time notifications to web applications.
A Brief History of Push Notifications
Push notification technology is not novel to modern-day software developers. It has been around for years and there have been a variety of methods and techniques engineered to implement push notifications.
- Polling
Before push technology was introduced, polling used to get frequent updates on web and mobile applications. In web applications, this was achieved by initiating an AJAX request to a web server and fetch updates periodically. There would be an inevitable delay for the updates to reach end users, hence this method was not real-time. Since this method operates on HTTP there is a considerable amount of protocol overhead for each request we make and subsequently, high bandwidth usages were introduced as the application scaled. These reasons make polling inefficient for real-time communication and a slight variation called long polling was introduced as an alternative.
2. Long Polling
Long polling works almost exactly as polling except for the fact that the web server keeps the connection open for sometime until it has data to respond with. With this technique the client is not required to initiate frequent requests, even when there are no updates from the web server, making it slightly more efficient than polling. Still, this cannot be considered as “true push” since the client periodically requests for data.
3. Server Sent Events
This is an asynchronous method of sending data from a web server to a web client continuously. The client bears the responsibility of initiating the connection with necessary headers to indicate to the web server that the data should be sent as an event stream. The server keeps the connection open as long as the client is alive and the protocol itself supports re-connections. Being a low latency protocol, Server Send Events gained traction in software development environments and consequently almost all browsers now support Server Sent Events.
4. Web Push
Web push is nearly identical to how push notifications work on mobile devices. Users that receive notifications over web push do not need to be on the web page in the browser from which they are expecting notifications. The notifications can be received by just keeping the browser window open. Since this is a browser-based feature, the notification UI features depend on the browser and the underlying operating system, resulting in less flexibility with the customization of notifications.
5. Web Sockets
Being a full duplex and bi-directional communication protocol, web sockets are ideal for real-time communication in applications such as massively multiplayer online gaming, real-time weather updates, char applications and many more. Since the protocol itself does not define how to handle disconnections and re-connections, a slight effort goes to the client side implementations to handle disconnections and re-connections. However, modern web socket libraries have implemented the above functionality and have built it into the libraries so that developers can use web sockets out of the box.
Message Broker Architecture
A Message Broker is a software system which enables communication between your applications and services. A software system can be decoupled from its independent components using a message broker, since the message broker translates the messages into multiple protocols and delivers messages to it’s clients, even if the clients are of different languages and supports different communication protocols. Message routing in message brokers can be either content based or topic based and it is done using the Publish-Subscribe pattern. Hence your system components need not have any knowledge of the internals of the other components in the system.
Why Managed Services
Although, we can take the decision to design and develop real-time notification platforms using Server Sent Events, Web Sockets or any other technology, most organizations tend to utilize an already available implementation (unless the business requirements rule against using third party software systems). Among these third party services, the most commonly used services seem to be Managed Services. Managed services has advantages for the developers such as the delegation of maintenance, security updates and patches and other housekeeping activities, leading to improved productivity.
AWS IoT Core as a Message Broker
One of the reasons to select AWS IoT is because it is a fully managed service. This means we could integrate AWS IoT into our application without much effort and AWS itself would manage all the housekeeping activities around this service. What makes AWS IoT ideal for real-time communication is its’ internal message broker. You could even say that the entire AWS IoT Core is driven by the message broker. A variety of devices including web clients and mobile devices can send and receive messages to and from AWS IoT Core using protocols such as MQTT over WebSockets, MQTT and HTTPS. IoT Core provides multiple security options for its clients. The Client devices can use X.509 certificates for connections while the web applications have the ability to use either AWS Cognito identities or custom authentication. In addition to that IAM users, groups and roles can be used to initiate an authentication and connect to AWS IoT Core.
An Example Use Case
In this article we will be taking a look at a simple web application with authentication based on AWS Cognito. Let’s assume the users on this web application receive notifications from a back-end server, although considering the simplicity of this article, you should consider this back-end server as a black-box.
The Architecture
As the first step, the users should be able to login to the web application where they will be authenticated against the AWS Cognito user pool. Assuming that the authentication is successful, the AWS Cognito identity pool bound to the respective user pool issues a identity id for the authenticated user. This identity id holds an AWS IAM role which consists of policies we define during the configuration. With this role, the web application is granted permission to communicate with the AWS services. However, AWS IoT Core requires more granular permissions to allow the web clients to communicate with the IoT Core. This can be achieved by defining an IoT security policy in the AWS IoT Core. The web application then makes a request to it’s back end server along with the identity id of the user to attach the IoT policy to the principal identity of the user. With this, the web application can connect to the AWS IoT Core and subscribe to receive messages on behalf of the authenticated user.
Provisioning the Resources
AWS Cognito user pool
The first step of provisioning the resources required for the above architecture is creating a user pool in the Cognito dashboard. Here, we are creating a user pool with the default configurations.
Once we create a user pool, Cognito provides an id for the created user pool. This user pool id can be retrieved from the General Settings under user pools and will be used in the steps that follow.
In the App Clients section, we can create a client for the web app with the following configurations. Since we are using this client for our front-end web app, we are not generating a client secret. As authentication flows, we are using username and password based authentication and SRP protocol based authentication. Refresh token based authentication will be used by default.
Once we create the app client, Cognito generates an app client id for the web app.
Afterwards, the settings for the app client should be configured properly. Since we are using Cognito to maintain our users we should use Cognito as our identity provider in the app client settings. If there are other identity providers defined in the user pool, we have the flexibility of using those providers.
As callback and signing URLs we can use the web app home URL. In this exercise, since we have hosted the front-end web app on the local machine we have provided the localhost URL and the application port.
Allowed OAuth flows must be according to the OAuth flows we require. In this application, we require authorization code grant type and implicit grant type. Also we have the flexibility to select OAuth scopes as per the need.
Since we are going to be signing into our front-end web application with Cognito, we must have an auth domain. We can specify a prefix for the domain which is provided by Cognito and check if that subdomain is available.
As the final steps in configuring the user pools, we need to create the users in the users and groups section under general settings. By providing a username, temporary password, and an email, we can create a user.
After the users have been created, the account status will be FORCE_CHANGE_PASSWORD. When the user signs in using AWS Amplify Auth, the user will be prompted to change the password and the account status will be updated to CONFIRMED.
AWS Cognito Identity pool
Identity pool can be configured after the user pool has been configured properly. The option to create an identity pool is available in the federated identities section of the Cognito dashboard.
The ids of the user pool and the app client created in prior steps can be given as the authentication provider configurations while creating the identity pool. This is only when Cognito is being used as the identity provider. Other identity providers must be configured accordingly.
The configuration wizard will redirect to IAM section to create roles for unauthenticated users and authenticated users after creating the identity pool. Since we are using federated identities only for AWS IoT connections and requests, the following policy can be attached to the authenticated user role while keeping the unauthenticated role as is.
When the identity pool has been created, the identity pool id issued by AWS can be retrieved by going into edit mode. Identity pool id will be used when the front end web app is configured with Amplify auth.
AWS IoT Security Policy
The only configuration that needs to be setup in AWS IoT Core is the security policy configuration. This policy must be attached to the federated identities so as to allow the user to connect to IoT Core.
Using the advanced mode, the policy can be defined as follows:.
Apart from the IoT security policy there are no other resources required to provision in AWS IoT. IoT Core message broker is already available to be used for communication.
Configuring the Front-End Web App for Authentication
The front-end application can be configured with AWS Amplify to use Cognito authentication and sign in users. AWS Amplify provides a set of tools which can be used to develop front-end web and mobile applications utilizing AWS services.
Amplify provides a ReactJS library which we can use in React web applications to configure authentication and it can be installed using either NPM or Yarn package managers.
The required configurations can be injected to the application environment variables using a file matching ‘.env.development’. Please refer React documentation regarding the use of environment variables inside React web applications.
To create the sign in page another two packages are required to be imported to the React code.
Inside a useEffect hook we can define the logic which determines what happens after a successful authentication by checking the value of ‘newAuthState’.
Given that the configurations are correct, the sign in page will prompt the user to sign in to the application.
Connecting and Subscribing to AWS IoT Core message
AWS Amplify PubSub module can be used to configure the IoT Core connection in the React web application. Before connecting to IoT Core, the ‘AWSIoTProvider’ plugin must be added to Amplify. This can be done in the index.js file before the code segment where ReactDOM renders the components in the React project.
‘AWSIoTProvider’ can be found in the Amplify PubSub module.
When adding AWS IoT Provider as a pluggable resource, other than the region, the IoT device data endpoint is required. The device data endpoint can be retrieved from the settings section in AWS IoT Core.
After adding the AWS IoT Provider plugin the React web application can subscribe to topics and receive messages. Before subscribing to topics it is important to attach IoT policy to the identity of the signed in user so that the web application can connect and receive messages from IoT Core on behalf of the users.
The identity id of the signed in user can be retrieved by using the Amplify Auth module.
A back-end web server is required to attach the IoT policy to the given identity. In this example we will consider a simple NodeJs web server implemented with expressJS. This back end web server will contain the code required to call AWS services to attach the IoT policy to the provided identity id.
In the back-end web server, we define the route POST /api/attach-policy which accepts the identity id in the request body as below
{
identityId: '***'
}Inside the route handler the service function to attach policy is called.
Inside the attachPolicy function in service file, an instance of the AWS IoT module is created using the AWS SDK, and policy is attached using the IoT instance.
It is important to have the AWS credentials with required permissions injected as environment variables or included in the AWS credentials file in the developer machines when we start the back-end web server. If the web server is hosted on AWS cloud, attaching a role with required permissions would be the best practice.
Having this endpoint exposed in the back end web server, the front-end web application can call this endpoint to attach IoT policy to a users’ identity.
After attaching the IoT policy, the web application can subscribe to topics. The Amplify PubSub module can be used to subscribe to topics and receive messages. When subscribing to topics, a topic name, next callback for incoming messages, error callback for handling errors and close callback for handling connection terminations must be provided.
After adding all code segments together, final code will be as follows:
Finally, it is best to unsubscribe from topics when the user leaves the web page.
To test the implementation we can use MQTT Test Client provided by AWS IoT Core.
With a bit of spice added to the front-end web application with ‘notistack’ npm library, the result would look like the example below
Notistack can be acquired from npm public repo.
Conclusion
In this article we discussed using AWS IoT Core message broker to implement a real-time notification system. Although there is a variety of technologies and services, the selection of a suitable service must be done carefully along with an assessment that determines which service serves our requirements the best. Having said that, it is important to be cautious of limits imposed by AWS before we proceed with using AWS IoT.
More on AWS limits can be read using the following link
References
- https://aws.amazon.com/messaging/
- https://aws.amazon.com/blogs/iot/configuring-cognito-user-pools-to-communicate-with-aws-iot-core/
- https://docs.amplify.aws/lib/auth/getting-started/q/platform/js/
- https://docs.amplify.aws/lib/pubsub/getting-started/q/platform/js/
- https://docs.amplify.aws/ui/auth/authenticator/q/framework/react/
- https://www.npmjs.com/package/notistack
