DATA 360 : from idea to data product
How to develop the right data product for the right need, with the right technology, with the right data, for the right users?
The ideation phase in data product development is a crucial step in the generation of innovative concepts and ideas related to the use of data.
The main objective is to define use cases based on data that can be exploited to improve the user experience, generate value, optimize features and solve specific business problems.
This step involves a process of co-construction with the products owners, allowing us, during the workshops,
- the desirability, to understand the business challenges and user needs,
- the datability, to identify the different sources of data and their qualities,
- the feasibility, to analyze the technical and scientific feasibility of the features
- and assess the viability of the approach.
To implement this funnel from ideation to production, the Data Team has developed our own methodology, which will be described below:
1. Ideation: the Design-Data methodology, a use cases booster
The aim is to identify and map information flows between people, businesses, business processes, tools, etc. during workshops with business users, in order to identify the organization’s hot spots.
An information flow is an exchange of data, knowledge or messages from one point to another, whether within an IT system, between people, or even in natural processes. It can be digital data, verbal conversations, written documents, electrical signals and so on.
The HBDS (Hypergraph Based Data Structure) modeling is used to represent information flows. This method is the work of François Bouillé, who, in his thesis “Un modèle universel de banque de données simultanément portable”, presents a model based on graph theory, hypergraph theory and set theory.
This modeling concerns information structure, characteristics, links and updating. The aim of this approach is to facilitate the deployment of this model on another geographical perimeter, and thus prepare the transition to the scale of the future product, by identifying all the specific characteristics of businesses, sites, people, etc. in these attributes.
Once the modeling has been completed, data use cases can be identified and prioritized. For this, an analysis of information flows will be important to identify hot spots: multiple links with low digital maturity.
Two choices can be made:
- to improve, automate and optimize flows, and thus respond with data use cases to pain points (user, business or organizational)
- to use data use cases to modify the structure of the modeling carried out.
2. Framing: The DS framework, from idea to action plan
This methodology is designed to ensure a rigorous approach to data science, from understanding the problem through data exploration and modeling, to implementing the approach. It ensures that data science use cases are aligned with business objectives, user needs and generate business value.
A 4-step method:
- The business section addresses the vision and value proposition of the use case. It identifies the distinctive advantages and benefits that the product will bring to its users, as well as the strategic statement that clearly and inspiringly describes the direction that product development will take.
- The aim of the product part is to provide context and to break down the product vision defined in the business part into Data Science features, for a defined and identified perimeter and users.
- The Data section allows us to refocus on the real data requirements for the features, scope and users identified and defined in the Product section. We will also address all data management and data processing issues, which can have a major impact on the DS Features that will feed the DS models.
- The build section prepares the best possible way for the development by data scientists, but also for users in the case of change management.
During the scoping phase, we will identify a clear vision of Data Science Ops to automate and rationalize the process of developing, training, deploying and maintaining machine learning models, while guaranteeing their stability, scalability and performance.
The global framework:
In conclusion, the implementation of a methodology within TotalEnergies Digital Factory that promotes the development of customer and data-centric products is an essential element in the current landscape of rising data science powers.
Companies are increasingly recognizing the intrinsic value of their data and seeking to transform it into innovative products and solutions. However, creating effective data products requires much more than technical skills. It requires a methodical approach that integrates an understanding of business needs, user requirements, the cloud ecosystem, data collection, management and governance.
In this article, we propose a methodology, adapted to the specific needs of companies, that guides the process from conception to realization, while ensuring the quality, scalability and relevance of data products. It also encourages collaboration between teams, and promotes transparency and traceability by involving all stakeholders from the ideation phase through to the build phase. It is therefore recommended to carry out this exercise again once the product has been implemented.
Finally, investing in the implementation of a data product methodology is not only a guarantee of success in exploiting data, but also a crucial step in remaining competitive in today’s data-driven marketplace. We are convinced that the companies that adopt this approach will have a strategic advantage in terms of innovation, decision-making and value creation from their data.
