How OSDU® Forum is redefining data management and collaboration across the energy sector
By Tim Hunter Rowe, Principal data manager, bp
The energy landscape is undergoing a significant shift driven by the need to balance affordability, security, and sustainability. For bp, digital innovation is at the heart of this evolution.
However, challenges exist due to lack of integration, extensive and siloed data repositories and data quality essential for powering the industry. This blog dives deeper into these challenges and explores how by integrating information through the OSDU® Forum we can unlock a brighter energy future.
Breaking down barriers for a better future
The energy industry is data-intensive and relies on a consistent flow of high-quality information. Traditionally, this data has been collected using proprietary formats and then stored in specialized databases tailored to specific analytics applications, a method commonly known as an application-centric approach.
The advantage of this method is that it provides a reliable data source through a vertically integrated technology stack. However, this approach has significant drawbacks, primarily because it leads to the creation of data silos. These silos usually consist of copies of datasets in different proprietary formats making it challenging to share data between organizations and technologies. Additionally, multiple copies of the same dataset impair the ability to track lineage and for users to know which dataset is the source of truth.
This limitation affects organizations in the energy sector in several ways. First, no company in this industry operates in isolation; they frequently exchange data with various partners such as suppliers, regulators and other entities. Therefore, it is crucial that we can manage data securely within our own organization but also facilitate sharing of data with our various partners as required.
Second, data silos burden individual organizations by necessitating multiple copies of the same data in various formats so that different applications can access it. This redundancy leads to higher operational costs and data quality issues that erode trust.
Moreover, the presence of siloed data hampers an organization’s ability to capitalize on business transformation opportunities. Inefficient access to comprehensive datasets hinders innovation, as it forces analysis against a compartmentalized data store. This siloed approach also impedes the ability to leverage newer technologies such as AI/ML, which benefit from vast amounts of clean, integrated data to function effectively.
Innovation comes with collaboration
The industry response to these challenges began in 2018 and what we have learnt from conversations with other operators covers several key points:
Data uniformity
Operators gather the same types of data, whether it’s seismic data, well bore data, fluid samples, or rock cores. The collection, storage, and management methods are uniform across the industry and offer no competitive edge.
Cost efficiency
How do we break away from the application centric approach that we have developed over the decades that requires costly, third-party proprietary technologies to manage our data? These systems necessitate expensive software, involve complex data pipelines and ETL (Extract, Transform, Load) processes and dedicated data management teams.
Collaborative approach
Benefits of cooperating across the industry to address issues openly. By doing so, organizations could reduce the time spent on low-value tasks, such as organizing data, and allocating more resources to high-value activities like data interpretation and analysis.
Open innovation
Developing an open-source data platform with universally accepted data standards and open-source APIs, would foster innovation within the industry and enable software vendors and operators to adopt emerging technologies, such as artificial intelligence, more quickly.
These factors all contributed to the creation of the OSDU organization. Originally standing for Open Subsurface Data Universe but soon expanding beyond Subsurface and now just known as the OSDU Forum.
Expanding horizons — the evolution and collaboration of the OSDU Forum
The original goal of the OSDU community was to manage our subsurface data estate effectively. However, the scope of OSDU initiative quickly broadened to encompass various sectors, including wells, engineering, production, carbon capture and storage (CCS), and renewables. Additionally, the Open Footprint® Forum is a parallel activity in which ongoing efforts aim to establish standard data schemas for emissions data. The mining industry has also shown interest in leveraging the foundational work of the OSDU community.
The OSDU Forum represents a collaborative effort of over 200 member companies, encompassing diverse participants. These include operators, national oil companies, cloud service providers, software vendors, service integrators, and academic institutions. This collaboration is facilitated by The Open Group, which provides the essential infrastructure, including a working environment, code repository, and organizational support for meetings plus a legal framework ensuring compliance with anti-trust legislation.
To effectively address the diverse needs of this initiative, several working groups have been formed, focusing on two primary areas:
Standardization
The industry has united to define standards in critical areas, notably data and technical standards. Data standards ensure the OSDU initiative can effectively support the different data types required in the industry using a ‘common language’.
For instance, geophysicists from various companies collaborate to agree on a seismic data standard, using existing standards where possible rather than defining new ones.
The introduction of the next OSDU standard release is due soon and will mark the publication of the first technical standard, outlining all technical details from core services like search, ingest, and data entitlements to domain-specific APIs through the use of a DDMS (Data Domain Management System). DDMS, is a collection of APIs focused on one specific business domain, for example; Well bore, Seismic, Reservoir.
Open-source software
One of the challenges with previous data standards initiatives was the dependency on individual organizations to determine the appropriate technology for implementation. This hampered application integration as there wasn’t a common set of APIs to access the data.
The OSDU community is tackling this issue by developing a data platform using open-source software under the Apache 2 license. This allows for vendor-neutral solutions that can be deployed in various environments, whether cloud-based, on-premise, or even edge computing.
This collaborative development involves data subject matter experts, architects, software engineers, and information security experts, ensuring a robust and adaptable data platform that can be deployed on any cloud or on-premises environment.
Setting new standards
Within the OSDU data platform, there is ongoing development to support real-time data with edge computing capabilities. This enhancement allows for edge analytics and efficient streaming of high-value data to central systems. Another critical advancement is transforming the initiative into a System of Engagement (SOE). While it currently serves as a System of Record (SOR) for managing ‘gold’ records or definitive sources of truth, the transition to an SOE will enable project teams to use the OSDU data platform to collaborate on project data that may eventually become new gold records.
Changing the professional landscape
The OSDU project is enhancing the industry by establishing standards and an open-source software platform for deployment. Beyond this, the initiative promotes numerous other benefits through its collaborative model.
Having spent 15 years in the energy sector, I’ve noticed that Operators were typically reluctant to share experiences broadly. Open, collaborative discussions were rare and usually aimed at specific outcomes. The introduction of the OSDU standard has revolutionized this approach.
The OSDU community hosts upward of 60 weekly meetings, with industry experts from member companies coming together to exchange ideas and discuss their areas of expertise. This setup offers a prime opportunity for newcomers, whether geoscientists, data experts, or technology engineers, to connect with some of the most knowledgeable professionals in the energy and digital technology fields. There are also conferences each year, in Europe and Houston, where members can meet in-person to build relationships and exchange insights.
OSDU Forum at bp
The OSDU data platform is considered a strategic priority for bp with multiple platform versions deployed to our cloud, ongoing testing is underway to understand the impact on our workflows with data for key assets ingested to the new standard. We are actively engaged in the community, from the management committee to platform code development and are collaborating with technology partners to enable the application ecosystem to integrate via the OSDU APIs.
Conclusion
The OSDU Forum unites experts from the energy sector and tech companies to establish data standards and develop open-source technology for their implementation. This initiative will overcome longstanding technical challenges, simplify data sharing, and foster innovation critical for addressing the energy trilemma such as advanced analytics and generative AI.
The OSDU Forum presents a unique chance for professionals within the industry to connect and collaborate across various fields. These connections can provide newcomers with a strong foundation for their careers, opening doors to mentorship opportunities, collaboration on real-world projects, and access to cutting-edge knowledge that can accelerate their professional growth. For seasoned experts, it’s a chance to network, share insights, and contribute to an initiative that will influence the industry for years.
I find involvement in the OSDU initiative to be extremely energizing. What’s encouraging is the continuous improvement approach shared by industry peers, which is uniquely fostering industry collaboration and powering a new, open ‘circular economy’. It is a feeling of being something that’s progressing in the industry with the potential to be a step change in how we manage data in bp too. Interaction with so many peers and experts across the industry is stimulating and there is always something new and interesting to get involved with. We invite you to join the OSDU community and contribute toward a revolutionary change in the way our industry manages data.
Tim has 30 years of experience providing innovative Data Management solutions to a broad range of organizations varying from large multi-national companies to national and local government. This work was spread across a number of industries including oil & gas, environmental, offshore survey, mining and national/regional planning. He has been at bp for over 10 years currently working in the Data & Analytics Platforms group which is part of Digital Technology. He represented bp on the first OSDU Forum Management Committee for three years and now co-chairs the Business Capability Maturation working group. He is also actively involved in a number of other OSDU community groups.
