The Best Decisions Are Based on the Right Information

The right business decisions are based on up to date and accurate data

Business leaders need to make decisions. In many situations it can be difficult to be sure whether or not the information, on which a decision is based, is accurate or fresh. However, with today’s data integration technology it is perfectly possible to have access to the latest accurate and up-to-date data.

The Problem

Projects create data. Lots and lots of information. Mostly this information is created in order to create a historical record and/or communicate something between team members and other stakeholders.

Unlike former times, in the World Wide Web serviced world of today most of this information is stored somewhere “in the cloud” (on a computer connected to a network). This means that we have, at our fingertips, the potential to access all of this information.

Annoyingly, we also live in a world of heterogeneous information storage tools that do not seamlessly integrate. Thus the exciting possibilities of all this information are hampered by frustrating technical integration issues.

It is well known that integrating and subsequently analysing this data can greatly assist the running of a business. Business Intelligence techniques and tools have been used for many years (Since 1865 according to some sources). Even on a smaller scale, running a project inside a larger business, Key Performance Indicator (KPI) Dashboards have been in use since the 1970s. However, the “Intelligence” acquired from the analysis is only as good as the data itself.

Fortunately there is a way in which we can collect real data, without errors, and base our analysis and decisions on a solid foundation.

The issue with common solutions

Most Business Intelligence tools or KPI Dashboards are intended to support a wide audience and hence provide mechanisms for integrating data from standard or generic data sources, such as:

• Relational (or other) Databases,
• Web Services,
• Excel spreadsheets

Often, and in particular in my field of work (Software Engineering), much of the data is not stored in tools that provided access to the data using such generic APIs.

The common solution is to ask a person (i.e. the project lead) to give a Status Report. The person uses his own resources to do the transformation of the real project data into a form that is suitable for the Business Intelligence tool.

Now although we, of course, employ dedicated, responsible, skilled people. People are only human, and make mistakes. People are also a costly resource, hence we do not ask them to spend their valuable time creating such reports too often. As such, the data is quite likely out-of-date and may have mistakes in it.

The silver bullet

A much better solution, is to collect data from the real engineering tools in use and automatically transform it into a form that is suitable for your Business Intelligence tool or KPI dashboard. Thus the data collected is, real, accurate and up-to-date. Additionally, good engineers did not spend their time creating reports, rather they were able to continue to focus on engineering tasks.

Although the individual engineering tools may not have a highly generic API, there is always a way to extract data. There are tools and techniques available today that make this possible and I work for a company (itemis AG) that has been doing this for many years for our own products and within multiple bespoke customer projects.

The technical solution

Firstly, develop a common API that is matured over many years and is based on the experience of extracting data from multiple different tools.

Secondly, build up a portfolio of multiple data extractors that are based on that API. Without the experience and maturity it is very easy to build an API that is quite simply, not generic.

Thirdly, learn how to address issues such as history, identity, and scale.

Finally, develop user-friendly mechanisms that make it possible to query and view the data once it is collected.

(For more technical details on an approach I have taken you can read this.)