Controlling Your Data — Part 1

Saturday, January 14, 2006 at 12:58PM

This is the first of two articles discussing technologies used to control data. This article covers data management strategies, while the second article will cover Extraction, Transformation and Loading tools, the technology most likely to be used for analytical applications in insurance businesses.

What do we mean by “controlling data”? The primary purpose of data management technology is to deliver good quality data. By this we generally mean:

• data that can be verified against its source,
• a single agreed definition for the data, and
• comply with data standards

In addition, there are a number of secondary goals that must also be met by any tool used to control data. It must be able to handle large volumes of data and move it efficiently. The system must maintain the security of the data. Processes must be verifiable and support audit and testing. Finally, depending on the environment and application, there may some additional requirements, such as the ability handle data from multiple sources, multiple platforms or in real time. There are three data management approaches:

• Consolidation,
• Federation, and
• Propagation

Consolidation

Consolidation involves moving data from multiple sources to a single consolidated source. The two types of tools designed for this are Enterprise Content Management (ECM) tools and Extraction, Transformation and Loading (ETL) tools. ECM is typically used for knowledge management, such as organizing and accessing information from documents, while ETL is used for data. Consolidation is typically very scalable and can support complex data transformation. Its main weakness is that the data is not available in real time.

Federation

Federation involves creating a virtual business view of the primary data sources. These tools are typically referred to as Enterprise Information Integration (EII). As a virtual view of the business, any changes in data are reflected in the virtual business view in real time. The downside of this is that data cleansing and transformation is limited.

Propagation

Propagation tools distribute and replicate or copy data between multiple systems, often in real time. These tools may be referred to as Enterprise Application Integration (EAI), Real-Time ETL (RT-ETL), or Enterprise Data Replication (EDR). Once again, the main advantage is that data can be available in real time, however performance can vary and the tools may be difficult to set up.

In Summary There are three fundamental approaches for data control — Consolidation, Federation and Propagation. Federation and Propagation can share data in real time, but are limited in their ability to apply complex transformations and manage large volumes of data. For this reason a consolidation strategy is of most relevance for data management for actuarial applications. ETL tools are the topic of the next article in this series.

Originally published at actuarialreview.com.