Decision-making with Linked Data Event Streams and PowerBI
How to monitor, analyse and make informed decisions with Linked Data Event Streams
In recent years, the use of Linked Data Event Stream (LDES) has grown rapidly to represent and share large amounts of real-time data. However, visualizing and monitoring fast-moving real-time data can be a challenge. This is where PowerBI comes in. It can be used to create powerful visualizations and dashboards of LDES data, making it easy to monitor the content and performance and use it as a decision support system.
We explore how PowerBI can visualize LDES data and how it can be used as a decision support system. We will also discuss the benefits of using PowerBI for LDES data visualization and the key features that make it an ideal tool for this purpose.
Synced dashboard in PowerBI
A Linked Data Event Stream is a constant flow of immutable objects (such as version objects, sensor observations or archived representation) containing information about an event or state change originating from a system that continuously creates data.
Since Linked Data Event Streams are always in sync with the source (e.g. sensors, financial databases, etc.), every dashboard that consumes an LDES can be kept in sync and display real-time data.
PowerBI is an intelligence tool that enables users to connect to multiple data sources and create and share interactive and intuitive dashboards. One of its key features is the ability to visualize geospatial data in real time through a link with PostgreSQL. In this way, it allows users to create real-time custom visualizations from LDES streams that are consumed in sync in a PostgreSQL database.
The Linked Data Event Streams specification provides a standardized way of representing real-time geospatial data, which can be easily integrated into PowerBI.
This allows organizations to create dynamic and up-to-date dashboards that display real-time information about the events occurring in a specific geographic location. The characteristics of the Linked Data Event Streams specification ensure that the data is always in sync, providing organizations with an accurate and current representation of their data. PowerBI leverages this data to create interactive maps, heat maps, and other visualizations that help organizations better understand and analyse their geospatial data.
The Generic Information Public Domain (GIPOD) data stream is used in the example underneath. This data stream brings together all information about (earth) works, events and nuisances in the public domain. This data stream is published as an LDES, and is consumed into PostgreSQL. Afterwards, this LDES is visualized in PowerBI.
Underneath you can see a part of a fragment of the published LDES:
{
"@context": [
"https://private-api.gipod.vlaanderen.be/api/v1/context/gipod.jsonld"
],
"@id": "https://private-api.gipod.vlaanderen.be/api/v1/ldes/mobility-hindrances?generatedAtTime=2021-01-06T18:04:12.173Z",
"@type": "Node",
"viewOf": "https://private-api.gipod.vlaanderen.be/api/v1/ldes/mobility-hindrances",
"collectionInfo": {
"@id": "https://private-api.gipod.vlaanderen.be/api/v1/ldes/mobility-hindrances",
"@type": "EventStream",
"shape": "https://private-api.gipod.vlaanderen.be/api/v1/ldes/mobility-hindrances/shape",
"timestampPath": "prov:generatedAtTime",
"versionOfPath": "dct:isVersionOf"
},
"tree:relation": [
{
"tree:node": "https://private-api.gipod.vlaanderen.be/api/v1/ldes/mobility-hindrances?generatedAtTime=2021-01-06T18:05:43.477Z",
"@type": "tree:GreaterThanRelation",
"tree:path": "prov:generatedAtTime",
"tree:value": "2021-01-06T18:05:43.477Z"
}
],
"items": [
{
"@id": "https://private-api.gipod.vlaanderen.be/api/v1/mobility-hindrances/10590330/15",
"@type": "MobilityHindrance",
"gipodId": 10590330,
"identifier": [
{
"@type": "Identifier",
"Identifier.identifier": {
"value": "10590330",
"type": "gipodId"
},
"assignedByName": "https://gipod.vlaanderen.be"
}
],
"isConsequenceOf": [
{
"@id": "/api/v1/events/9197361",
"@type": "Event",
"gipodId": 9197361
}
],
"description": "Verhuiswagen",
"owner": {
"isVersionOf": "/api/v1/organisations/14768eaa-6c54-0075-837a-90178a74ce8c",
"@type": "Organisation",
"preferredName": "Gemeente Kapellen"
},
"contactOrganisation": [],
"zone": [
{
"@id": "/api/v1/mobility-hindrances/10590330/zones/019e77f3-e2fd-4624-b6d7-8cfdbe57683f",
"@type": "Zone",
"consequence": [],
"geometry": {
"@type": "Geometry",
"wkt": "<http://www.opengis.net/def/crs/EPSG/9.9.1/31370> POLYGON ((154536.4910038893 221859.32163325883, 154538.4706818894 221854.19150346238, 154540.80870575312 221855.08900541347, 154538.82902577805 221860.21913343482, 154536.4910038893 221859.32163325883))"
},
"zoneType": {
"@id": "/api/v1/taxonomies/zonetypes/0fb72ef7-6ac9-4a70-b295-a30ea215d250",
"prefLabel": "HinderZone"
}
}
],
"period": [
{
"@type": "Period",
"start": "2021-02-26T23:00:00Z",
"end": "2021-02-27T22:30:00Z"
}
],
"timeSchedule": null,
"permittedBy": [],
"status": {
"@id": "/api/v1/taxonomies/statuses/a411c53e-db33-436a-9bb9-d62d535b661d",
"prefLabel": "Onbekend"
},
"generatedAtTime": "2021-01-06T18:04:12.173Z",
"eventName": "MobilityHindranceWasImportedFromLegacy",
"isVersionOf": "/api/v1/mobility-hindrances/10590330",
"memberOf": "https://private-api.gipod.vlaanderen.be/api/v1/ldes/mobility-hindrances",
"lastModifiedOn": "2021-01-06T17:11:43.167Z",
"lastModifiedBy": {
"isVersionOf": "/api/v1/organisations/14768eaa-6c54-0075-837a-90178a74ce8c",
"@type": "Organisation",
"preferredName": "Gemeente Kapellen"
},
"createdOn": "2021-01-06T17:11:43.167Z",
"createdBy": {
"isVersionOf": "/api/v1/organisations/14768eaa-6c54-0075-837a-90178a74ce8c",
"@type": "Organisation",
"preferredName": "Gemeente Kapellen"
}
},...LDES data flow
To store Linked Data Event Streams in PostgreSQL/PostGIS, Apache NiFi is utilized. Apache NiFi is a highly flexible and scalable open-source platform that streamlines data processing and system integration. The platform boasts a user-friendly web interface allowing users to easily create, manage, and monitor data flows. It offers a variety of pre-configured connectors and processors for executing data processing operations.
The structure of this data transformation flow has already been described in the next article (data flow is the same for importing data to PostgreSQL or to TimescaleDB):
Power BI is a widely used business intelligence tool encompassing services, applications, and connectors. It enables you to extract raw data from multiple sources and generate informative reports. To link Power BI with a data source like PostgreSQL, an appropriate ODBC driver can be utilized.
Full code
The full code is available on Github if you want to dive into this use case. Follow the instructions to ingest Linked Data Event Stream into Power BI.
Conclusion
In conclusion, Power BI provides a powerful tool for visualizing Linked Data Event Streams in real-time, enabling effective analysis and interpretation of large and complex data sets. Its interactive dashboards, customizable visualizations, and real-time streaming capabilities make it a crucial tool for data-driven decisions.
If you like what you read, be sure to ❤️ it — as a writer, it means the world. Stay in touch by following me as an author.
Contributors to this article are ddvlanck (Dwight Van Lancker) (github.com), sandervd (Sander Van Dooren) (github.com) at Smart Data Space (Digital Flanders). In a rapidly changing society, governments need to be more agile and resilient than ever. As a strategic partner, we realize and supervise digital transformation projects for Flemish and local governments.
