Data Here, Data There, Data Everywhere

Bridging the Gap Between Manufacturers and Consumers

Melanie Brunache
IBM Data Science in Practice
5 min readApr 5, 2021

--

[co-authored by Samantha Rayl]

An image of a city’s skyscrapers through a window with a neon sign saying “Data has a better idea” below the window
Photo by Franki Chamaki on Unsplash

Data Lakes, Warehouses, Marts, Cloud, Applications, Web Services, Local Files –the increasing list of siloed data sources goes on. Even after we’re able to connect to these sources –how can we make the data truly accessible in a single view for all users to access when needed? The answer should not be using numerous ETL (Extract Transform Load) jobs that may result in unnecessary duplication and data security risks.

It’s simply no longer viable to continue to move and/or copy all of the data we need, whenever we need it. The growth of Big Data brings complexity, resulting in an increase in the gap that already exists between IT and business consumption. In this article, we’ll explore several use cases where IBM can help clients with Data Virtualization.

The Problem

A leading furniture manufacturer wanted to increase operational efficiencies and inventory accuracy to improve cross-sell opportunities. The enterprise needed a single view of their product inventory, manufacturing data, and customer data. In order to do this, they needed to integrate their disparate data sources and make the information available for several varying lines of business. With the growing market of online shopping, they realized how imperative it is to have a single “glass pane” view into all of the product and transaction data across vendors.

The Solution

At IBM, we are constantly working to close the gap between technical and business users. We start with making data available in real-time for business users, technical and non-technical, speeding overall time to value.

We started by introducing these questions to the client’s team: What if you could tap into all of your critical data assets no matter where they are physically located? What if you could query 2 or 2,000 data systems with a single query? If you have a consistent view of your customer, inventory, and sales data, how could that improve marketing recommendations to customers and increase operational efficiency?

They were able to accomplish this by utilizing Data Virtualization.

What is Data Virtualization? The ability to view, access, manipulate, and analyze data without the need to know its physical format or location, and without having to move or copy it.

By using this technology, combined with analytical sales forecasting, clients are able to improve the customer experience as well as sales revenue. The enterprise can create a single view of all available data sources without having to navigate different silos across vendors, risking inaccuracies and outdated information. With Data Virtualization, they are also able to increase data security by encrypting database credentials and keeping them on private local devices, not cached on other devices or in the cloud.

Our Approach

Data Virtualization uses data federation technology to define access to logically mapped remote data sources. Query technology is used to pull against multiple disparate sources and access all of the data we need at once. Contrary to traditional data integration methods, we can leave the source data exactly where it is, versus moving a copy of the data. We can use Data Virtualization on IBM’s Data & AI platform to “shop” for and manipulate the data, without knowing technical details such as its formatting and physical location.

Screenshot from Data Virtualization landing page on Cloud Pak for Data platform.

Take a Closer Look

Once we’ve connected to our sources and virtualized the data, we now have the ability to pull that data into a plethora of third party tools. Most enterprises today are using tools from many different vendors. With Data Virtualization, there is less time spent on integrating these proprietary services and more time spent on creating meaningful insights from the data. We can seamlessly create data connections, find the data we need, and use that data within third-party business analytics tools.

Tune into this brief demo video to see how we can virtualize data from different sources and integrate that data into a third-party service in less than 10 minutes.

Data Virtualization on Cloud Pak for Data

Landing page of Cloud Pak for Data

Another industry where data virtualization has enormous potential for creating change is that of freight manufacturing and maintenance. When we think about the maintenance needed on aircrafts, trucks, trains, and other freight transport vehicles, we know that this data needs to be accessed in real-time to prevent any unforeseen issues. This is imperative for health checks, maintenance planning, and product failure analytics. Business analysts are then able to generate urgent insights using their BI tool of choice to connect to the latest source of truth for the data.

This use case is applicable to many different enterprises across all industries. If there is a large number of disparate data sources sitting on different clouds or systems and the data needs to be accessed for customer service, supply chain optimization, predictive maintenance, and other BI and analytics, then the enterprise can see advantages from Data Virtualization. This is especially beneficial when there is a need for up-to-date, real-time transactional data, which is a growing demand in today’s application ecosystem.

Wrap-Up

By leveraging data virtualization, we are able to have access to unprecedented speed, security, flexibility, and ease of use. Combine this with the power of accessing data, regardless of where it resides, and the potential of the data turns nearly limitless.

Here at IBM, we strive not just to be industry leaders for our products, but to also be the best in the industry for helping customers throughout their journey to AI. To do this, we believe in creating meaningful learning experiences for our clients. This can be done by hosting Data and AI Expert Labs, getting users connected to our Data Science Experts to expedite learning and implementation, and as a result getting our clients a return on investment as quickly as possible.

From the video, we’re able to see how fast and simple connecting Data Virtualization to a third-party tool can be. Take a look at the following links to further explore this technology and how it help can speed time to value.

https://www.ibm.com/analytics/data-virtualization

https://www.ibm.com/analytics/services

--

--