How to avoid the dark data trap

Photo by Federico Beccari on Unsplash

Data rules the world and nowadays every single enterprise, big or small, should know that it needs to make proper use of data to succeed long-term.

But the amount of data available is growing exponentially every year with new connection points created, the number of source systems and IoT devices coming online and supplying new data streams.

Unfortunately access to more data can make it harder to manage, more difficult to understand and use effectively for valuable insights and decision making — or even at all.

There is exponential growth in data.

Dark data — what is it and why should I care?

Gartner defines dark data as the information assets organisations collect, process and store during regular business activities, but generally fail to use for other purposes (for example, analytics, business relationships and direct monetising)

Wikipedia defines dark data as data which is acquired through various computer network operations but not used in any manner to derive insights or for decision making.

In essence you might be falling pray to dark data in your organisation if:

• You do not have overview of what data you are collecting.
• You can’t process data in real time.
• You do not understand your data.
• You can’t control who sees your data.

At best these issues will prevent you from deriving meaningful insights from your data but at worst they might represent a huge security risk in terms of unwanted parties having access to sensitive data.

But what can enterprises do to avoid falling prey to dark data?

Form a data strategy
First form a strategy that addresses which data you want to collect, how you want it transformed and stored, and what interested parties should have access to it, by which means and for how long.

Be wary of data hoarding
Many enterprises simply settle for collecting the ever growing amounts of data they generate for future use without a proper plan for its use, consolidation and transformation to a synthesised form.

The overall strategy of collecting data for future use is usually not as smart as one thinks since the future use case will in most cases involve big investment in data engineering to get the data into a usable and understandable format.

Also relying on advanced algorithms such as unsupervised learning to generate insights from unstructured data can be hopeful at best.

Not to mention that data hoarding can be very expensive in terms of data storage and unnecessary energy use.

Transform the data to a synthesised format when it is received
Transforming the data to an understandable and synthesised format when it is received is a much better strategy than collecting data from multiple sources and then applying minimal transformation that still leave the data in silos and not ready to be used across the enterprise.

Control which interested parties receive data
The same goes for simply routing available streams of data to interested 3rd parties (or internal parties) without properly setting up the required gates and administration interfaces where the flow of data can be managed or stopped.

Employ the proper tools to assist your data strategy
For clear overview of how your data is received, how it is transformed and which parties receive the data you should employ the right tools. Otherwise your data strategy will have no hope of being able to succeed.

Ankeri’s Data Connections Hub to the rescue
Ankeri has built the Data Connections Hub to help enterprises take control of their data and prevent their data from becoming dark and unusable.

This includes being able to:

• Connect to multiple types of source systems using custom built connectors and configuration option.
• Convert the received data to a synthesised format and standard tags.
• Configure specific outcomes that control how and for what period interested party are able to connect to data, what data tags (measurements) they have access to and from what source systems.

With these features your company can rest assured that data is properly managed going forward.

The Data Connections Hub provides a cloud-based administrative overview of all data sources from the various assets (vessels in this case), the data tags associated with each source and then made available to interested parties (internal or external) by way of custom interfaces. Source connections to specific vendor systems can be created and configured as well as output interfaces (outcomes) which can have an optional valid period for managing how long the data is made available to a specific party.

Want to learn more about how to get your data strategy in order?
Then visit Ankeri for more information on the Data Connections Hub or contact us. We are happy to discuss your data strategy needs with you and how we can help.