Enterprise Data Hub: Your All-in-One Data and Analytics Portal

Iman Cizmic
RUBICON Stories
Published in
6 min readFeb 2, 2024

RUBICON delivers a comprehensive and secure cloud-based solution that serves as a single entry point for data consumers, bringing together all elements of data platform and analytics needs.

Quote

In developing our Data and Analytics portal, RUBICON has crafted a valuable solution for our organization. This solution enhances our data literacy and streamlines data, platform, and analytics integration for our users. Throughout RUBICON’s end-to-end development process, they effectively translated our ideas into software that holds the potential to consistently enhance our user experience and collaboration.

We appreciate RUBICON’s commitment to product ownership, their unwavering focus on delivering top-notch solutions, and their dedication to achieving high-quality results. Furthermore, the team consistently went above and beyond our expectations. The energy and motivation of the RUBICON team has had a profoundly positive impact on our work. This is a testament to their commendable level of agility.

We are grateful for their expertise, guidance and unwavering dedication. They have delivered a valuable solution that will open up new opportunities for us and set us on the path towards becoming a data-driven organization. Thank you for your great work, RUBICON!

Digital Product Manager (Analytics Strategy & Products)

Overview

Data analysts and data developers at the leading global Chemical and Consumer Goods Company often encounter challenges in their data journey. These challenges include limited transparency of available data and a heavy dependence on experts to discover and gain access to essential data. Additionally, many face hardships when trying to navigate applications and utilize analytics tools, leading to an unfriendly user experience.

To address these issues, the Client sought to create a Data and Analytics portal to empower data consumers and data developers to use Data Platform services. The vision was to reinforce users with a self-service experience by bringing together the Client’s Data Platform, which was developed by using a data mesh architectural pattern, and the Data and Analytics Portal. It would allow them to explore data assets, access data products, and engage with their data-driven community. All of this, while upholding the highest governance and security standards.

RUBICON stepped in to create a solution, a central Data and Analytics portal that acts as a single entry point throughout the client’s organization. Users can now seamlessly explore and analyze high-quality data. The Data and Analytics portal boasts an intuitive web interface, granting users access to the central metadata datastore. This, in turn, facilitates access to essential data and a suite of analytics tools and services.

Challenges

During the Data and Analytics portal development, our team needed to overcome the following challenges and find technical solutions:

  • Implementing a cloud-native solution that enables users to discover data products, browse data assets, request access to relevant resources and analytics tools, and engage in various learning courses
  • Enabling usage of a headless open-source content management system (CMS) that empowers content administrators to independently create, manage, and publish content, making it available to users in real-time
  • Designing an intuitive interface that serves end-users with diverse technical backgrounds and varied data needs
  • Creating a robust and secure technical architecture on Microsoft Azure, ensuring it aligns with best practices and the Client’s strategy and standards. This architecture needed to support multiple interfaces and connections to the Client’s Data Platform, data catalog, systems, and databases
  • Enabling end-user authentication using Single Sign-On, all while upholding strict security and privacy standards in line with the client’s policies and regulations
  • Transforming complex and lengthy user journeys into a smooth self-service experience across the Data and Analytics portal, ensuring a user-centric approach that meets the needs of personas ranging from non-technical business users to highly experienced technical experts

Solution

Months before the Client approached RUBICON, they envisioned a central portal designed to assist and guide data users throughout the organization, simplifying data discovery and usage. Before kicking off the project, the RUBICON team organized a three-day Lean Inception workshop with Client stakeholders. This collaborative session aimed to define requirements, focusing our discussions on end-user needs and their daily interactions with data.

The initial Data and Analytics portal launch included the provision of a web interface with functionalities designed for user self-service. We introduced features with a simple and modern UX/UI design, allowing users to access training resources on data and analytics, learn how to utilize analytics tools, and connect with the central data catalog system. The team also delivered a scalable content management system, allowing portal administrators to manage content efficiently and publish up-to-date information visible to end-users.

In the next development phase, our attention shifted to specific stages of the data product lifecycle, including processes like requesting data access and creating new data products. The idea behind the self-serve data infrastructure platform is to adopt platform thinking into data infrastructure. The Client’s Data Platform team provides domain-agnostic functionality, tools, and systems to enable building, executing, and maintaining data products. With its platform, the Data Platform team enables data consumers and data developers to consume and create data products seamlessly. Acknowledging the complex architecture and interdependencies with multiple services, RUBICON organized the Event Storming session to help both, the Client and the RUBICON team, better comprehend the processes. This workshop aimed to build a Big Picture model of the business process involved, deepen our understanding of requirements, and clear any misunderstandings.

We successfully implemented the processes and connected the Data and Analytics portal to several data services, transforming it into a one-stop shop for data users. Now, they can effortlessly discover and access all the data they require through the Data and Analytics portal. To enhance the user experience further, we incorporated a notification management system, ensuring timely notifications for both data requesters and approvers at each stage of the data request process.

Data and Analytics portal solution in Azure cloud architecture

Data and Analytics Platform Architecture

Results

Our dedicated team managed to create the platform with the following results:

  • Data and Analytics portal that includes the overview of various data sets, data use cases, access to services and analytics tools
  • Transparent availability of data sets pulled from the central data catalog
  • Implementing simplified and automated processes in the portal as the single entry point for requesting, accessing, and creating data sets and resources
  • Creating a self-service module through the Data and Analytics portal that provides easy guidance for business and technical users
  • Lowering barriers to data-driven decision-making across the organization by creating learning paths and training courses for data users
  • Developing an easy-to-use content management (CMS) solution for creating relevant content visible to users on the Data and Analytics portal
  • Configuring user authentication using Single Sign-on to provide secure portal login for authorized users

Technology Stack

Back-end: C#, ASP.NET 7, Azure App Service, Azure AD, Azure Container Registry, Azure Functions, Azure SQL, Azure Storage Account, Strapi CMS

Front-end: Angular, Azure Static Web App, TypeScript

DevOps: Azure Pipelines, Terraform, Docker

More about the tech stack

  • C#, ASP.NET 7 - Used to implement backend API
  • Azure App Service - Host API and systems on Azure
  • Strapi CMS - Open-source CMS system, used to manage Portal content
  • Azure Functions - Server-less solution used
  • Azure SQL - The main database used to persist data
  • Azure AD - Used to implement user authentication and Client internal accounts
  • Azure Storage Account - Blob storage used to persist Portal media assets
  • TypeScript, Angular - Used to implement Portal as a single-page application (SPA)
  • Azure Static Web App - Globally distributed web hosting of Portal SPA, with SSL certificates
  • Azure Pipelines - Used for continuous integration and deployment pipelines
  • Terraform - Automation of infrastructure code, and automation of infrastructure provisioning

Quick Facts

  • Region: Western Europe
  • Industry: Consumer Goods
  • Project Duration: Ongoing (January 2023 — )
  • Team: Product Owner, Solution Architect, Backend Engineer, Frontend Engineers, UX/UI Designer

Originally published at: https://www.rubicon-world.com/case-studies/all-in-one-enterprise-data-and-analytics-portal/

Data and Analytics Platform preview

--

--