Heroes develops accurate & reliable Azure Data Platform for STIZON

Challenge: Stichting Informatievoorziening voor Zorg en Onderzoek (STIZON) was looking for a futureproof data-driven solution which would enable them to process data extractions, from different sources, faster, more accurate, more secure and in a transparent way to support their business model.

Provided solution: A data-driven platform, build on Microsoft Azure, which enables STIZON to process data extractions of practitioners much faster and more accurate and secure.

Technologies used: Azure Cloud, .NET, C#

STIZON collects medical data from patients to make it available for scientific research, both internally and externally. They receive these data through various channels, mainly from general practitioners but also from hospitals and pharmacies. They then clean, structure and enrich the data with various medical standards and reference sources. The goal is to support the healthcare suppliers with enriched data which can increase the insights in the treatment of patients. For this business model it is crucial for them that they can count on reliable & accurate data while guaranteeing privacy since they are ISO 27001 and NEN 7510 certified.

For this specific project we have been focusing on the data provided by the general practitioners. Periodically, data extractions are delivered, this concerns about 850 practices. A general practice uses a general practitioner information system (HIS). There are 8 different systems available and each system has its own form of extraction. For a large number of practices, the extraction is done by the HIS supplier. The other practices do the extraction themselves. All extractions are delivered in ZIP files, with or without a password or via a secure FTP connection.

Heroes has been asked to come up with a futureproof data-driven solution as an answer on the situation above.

Next to the possible inaccuracy, the current process is very time-consuming and relies on one person. Currently, the processing of the extractions is done manually. It takes one FTE, about half a month, to collect, unpack, rename, structure, process and archive all extractions.

In short, the current way of working was very time-consuming, error prone, not transparent enough in according to new regulations and certainly not future proof.

The solution we came up with is a completely new developed data platform in Azure. This platform processes the extractions completely automatic and is fully metadata driven.

The most important components are:

- Azure Data Factory (ADF)

- Synaps Analytics (formerly Azure Data Warehouse) (DWH)

- Azure SQL Database

- Blob storage / Data Lake

- Azure DevOps

There was already a database in place to keep record of all the supplying practices. This contains information about which practice uses which HIS, but also about what an extraction of a HIS looks like. This database is the heart of the extraction process and we have further expanded it in order to achieve a good metadata registration. Something we also needed to consider is that every practice can change HIS overtime and the HIS itself is also further developed which can have consequences for the extraction. This means while building the platform we needed to make sure it’s very agile and can adapt easily to unforeseen changes.

The entire process is built in ADF and globally it looks like this:

During the entire process, progress is tracked and any errors are recorded and reported. This makes the entire process from delivery to processing in the DWH transparent, structured, documented and above all automated.

This new platform gives STIZON all the necessary tools they need in order for them to be ready for the future. It speeded up the whole process significantly, it reduced the amount of failures and all failures that do occur are reported. It also standardized the data into a Common Data Model which makes it much more transparent of what type of information is stored and it makes it much easier to add new types of data sources, like pharmacies and hospitals. This all was built on the Microsoft Azure Cloud, making it a cost-effective platform.