Architect your solution with queues, grids, and hubs

Consider a scenario, we are building a wind energy billing application. First, let’s walk through every piece in this application. We are simulating a full billing pipeline that is using all 4 main messaging services. Here a container instance is used to basically simulate a wind turbine, this container pumps continuous stream values like kWh. This stream of data is being ingested by an Event Hub. It uses the capture feature to write this in batches to storage. Simultaneously the Event Hub emits an event to the Event Grid communicating to create the capture file. The Azure Event Grid uses the Azure Relay to send this event to the on-premise system which picks up the batches in Storage.

This calculates the energy consumption for that batch of energy usage. Then this bill is dropped into the Service Bus Queue to bill the customer.

What is a Message?

Messages can be classified into two broad categories:

1. Command — For any command message in Azure we use Service Bus
2. Fact — The difference between command and fact is that fact is more loosely coupled from the event publishing upstream system and downstream system. There are 2 types of facts -Time Series and Discrete.
3. Time Series — Event Hubs can be used for things like logs. Event Hubs is the solution for streaming data. It is a data ingestion pipeline, in the above scenario, Event Hubs ingests data that are streaming out of the wind turbine. It is collecting how much energy each customer on the energy grid is using at a given time. Event Hubs has the capture feature, using which you can set time or data boxes.
4. Discrete — Within the Azure portal, you can use Event Grid for dealing with discrete events. It enables event-based programming within the Azure ecosystem as well as beyond pub/sub semantics. Event Grid notifies when a state change occurs in one system to another system.
5. Azure Relays are not quite the same as other messaging services. You can use Relays for scenarios involving on-premise. Azure Relays allows you to create a relay endpoint and then you can connect to any messaging system to the relay endpoint.

Serverless360 manage and monitor your Azure Serverless components

Serverless360 is one tool that you can depend to manage and monitor all your distributed Azure serverless components from one place. This is what we call “Composite Applications”. With the concept of Composite Applications, you can easily group your Azure Services that make up your integration solution and manage them effectively. Even if your Azure services are from different Subscriptions, you can manage and monitor your services.

Dashboard

Serverless360 dashboard makes it easier to get information about your integration solutions with simple Composite Application Cards. At a glance, you’ll be able to see the Composite Application title, type, and the number of Azure resources associated with that application, and the status of the alarms configured in that application. Click the Manage button on the card to operate and monitor the resources associated with the Composite Application. In this scenario, you can see the resources like Azure Service Bus, Relay, Event Hub, and Event Grid all mapped to one composite application. By this way, you will get the complete picture of one wind turbine.

The entity dashboard presents all the entities mapped to the composite application at one glance. You can also view the current monitoring status and instant access to the monitoring issues in detail.

Monitoring

Serverless360 offers the following 4 types of monitors, each one to address a specific business purpose. All the 4 types of monitors can now accommodate multiple entities mapping.

• Data Monitor to get notified on efficiency or reliability or consumption of the Azure entities by monitoring their extensive list of metrics.

• Status Monitor to generate a report at specific times in a day representing the state of entities against the desired values.

• Threshold Monitor to monitor your entities when certain properties violate desired values for a specified period say few seconds/minutes.
• Watch Monitor to get notified in near real-time on the failure of associated Logic App or Azure Functions.

Value Additions

Alone with dashboard and monitoring capabilities, Serverless360 offers other features like message processing, automated activities, back up Service Bus message to Storage Account Blob, upload multiple messages from excel to Service Bus Queue or Topics, custom role-based user management, auditing etc. You can read more about Serverless360 here.

Conclusion

If you are an organization investing heavily in Azure Serverless components, I hope this article shows how you really should be thinking When to use which Azure resource and for what. This will give you a great platform to develop a very good range of solutions. Within the Azure portal, when you have operations and monitoring challenges, Serverless360 can help you have a massive shift from solving those challenges to focusing your project team on solving the applications requirements and business problems. The productivity increase from this can be significant.

Note: The scenario explained in this blog is extracted from the session “Architect your solution with queues, grids, and hubs: When to use which and for what” by Bahram Banisadr at Microsoft Ignite 2018.

Serverless360 is a one platform tool to operate, manage and monitor Azure Serverless components. It provides efficient tooling that is not and likely to be not available in Azure Portal. Try Serverless360 free for 15 days!

Originally published at www.serverless360.com on November 28, 2018.