Startups Nation
Published in

Startups Nation

How Data Lineage Improves Data Compliance

With great power comes…

…great compliance regulations.

No, we’re not arguing on the truth of “with great power comes great responsibility.” But when entities with great power neglect their responsibility, they inevitably get slapped with compliance regulations designed to force responsibility.

Banks didn’t accurately assess their credit and operational risk and hold enough capital reserve, leading to the Great Recession of 2008–2009. Enter: BCBS 239!

Insurance companies misvalued or misreported on insurance contracts (which, to be fair, are notoriously hard to compare with precision). Please welcome: IFRS 17!

Commercial enterprises utilized consumer information for marketing without taking appropriate care to keep it private. Let’s hear it for: GDPR, CCPA and DCIA!

Now all is well with the world, right?

Wrong.

Taking pains is a pain

Data compliance may be good tidings for consumers, customers and clients, but it’s a big pain in the neck for the organizations that need to comply. It takes time, money, brainpower and manpower.

Efforts to automate regulatory compliance require investment of resources across in a variety of areas, and while no one tool or resource can supply all your regulatory compliance automation needs, the tool of automated data lineage is a critical asset in your regulatory compliance mapping efforts. It’s for that reason that even as the first BCBS-239 implementation deadline came into effect a few years ago, McKinsey reported that one-third of Global Systemically Important Banks had focused on “documenting data lineage up to the level of provisioning data elements and including data transformation.”

Let’s take a look at several regulatory standards and explore automated data lineage’s role in smoothing and improving data compliance.

Data lineage and financial risk data compliance

Money makes the world go ‘round.

Not enough money makes the world go topsy-turvy.

The banking system’s inability to deal with the Great Recession of 2008–2009 led to the passage of regulations designed to make banks more responsible in preparing for financial risk. BCBS 239 requires banks to appropriately evaluate risk, using more accurate and more objective standards, and to ensure sufficient capital reserves to deal with risks should they materialize.

Instead of using internal models to decide how much capital they need to cover risks, banks now need to use standardized models for many types of assets. Even when they are allowed to use internal models, the result still needs to reach a certain threshold (72.5% of the standardized approach). The ECB’s Targeted Review of Internal Models (TRIM) also has specifications about the models that can be used by European banks. All these models need to be informed by data, with operational risk assessment mandating loss data that goes back 10 years.

Can you find all that data? Can you prove it’s accurate?

This is where data lineage and metadata management becomes an integral part of BCBS 239, TRIM or any other regulations dealing with financial risk. Automated data lineage enables you to visually trace the entire path of any data point from origin to destination. With this visualization, you can quickly understand every transformation that happened to your data along the entire data landscape, maintaining data consistency and preventing redundancy or conflicts.

An automated data lineage solution gives you accurate data to use in your calculations and in your reporting. Almost as important, it gives you the ability to effortlessly prove accuracy to any auditor.

Data lineage and financial trading data compliance

If you thought evaluating credit and operational risk was tricky, evaluating trading risk is enough to make you bite your fingernails to the quick.

Another cause of the Great Recession was the lack of data transparency in high-risk financial sectors. Trading — stocks, bonds, ForEx, equities, you name it — definitely qualifies as higher risk than lending or borrowing money. In fact, the supporting data was so complex and unscrutable that price came to be a matter of being impressed by the seller’s personality rather than being reassured by his numbers.

Enter the Fundamental Review of the Trading Book (FRTB) regulations. They define how to measure market risk as regards how much capital reserve banks need to have to cover their trading activities.

In order for your risk calculations and models to be accepted by FRTB regulators, you need to have (and prove!) clear, granular, precise visibility into your data. Here, again, data lineage mapping gives you that visibility. Work back from any report, or forward from any source system, or backward AND forward from any data point within your landscape. The precise trail of your data, including what transformations it underwent, when and why, will be clear to see.

In addition, you can easily conduct impact analysis with models informed by automated data lineage. Ask the question — if this data point or condition changes, what will be affected? — and just look at the data path visualization to see your answer.

Data lineage and insurance data compliance

If you thought that we’d finally left the hard stuff behind, you’re in for an unpleasant surprise. Moving from financial trading data to insurance contract data is akin to jumping from the frying plan into the fire.

Insurance contracts are notoriously hard to evaluate and compare. For any given contract, you have to traverse a sea of questions, like: Is the valuation of the contract according to market value or present value based on cash flow? Which cash flows and liabilities are included? When do you value those cash flows? How does discounting factor into the cash flow value? How does risk change the valuation?

If your eyes are going glassy, you’ll understand why regulators instituted IFRS 17 (International Financial Reporting Standard 17): an accounting standard that intends to make financial reporting on insurance contracts more transparent and consistent across the insurance industry, allowing for apples-to-apples comparisons and reducing misvaluations.

Data transparency. Data consistency. Data accountability. Sounds like a job for… data lineage! And if you don’t want to make several new hires just to do all this new work, automated data lineage is the way to go.

Using automated data lineage as part of your IFRS 17 compliance strategy allows insurance companies to have a full grasp on their IFRS 17 data models and track down answers to questions (whether from an investor, an auditor or a manager) in record time.

Data lineage and consumer data compliance

What goes around comes around.

1984 expressed our primal fear of privacy invasion. Then came smartphones and FOMO overcame our privacy concerns as we gave away our personal information left, right and center in order not to miss the targeted coupons to the gas station we might be passing.

But the see-saw is swinging back the other way, and consumers are unhappy with the liberties that companies have taken with their data.

They’re demanding the right to be forgotten, the right to be left alone and the right to know precisely how companies are using their personal data.

GDPR (in Europe), CCPA (in California) and the DCIA (in Canada) are some of the most prominent — although by no means the only — consumer data regulations.

The key to compliance with any of these regulations is a strong handle on your data: how well you can control how consumer data is collected, utilized, and disclosed.

Are you masking or otherwise protecting consumers’ PII? If a consumer asks you to delete every speck of information you have on him, can you do it — and prove you did it? Can you show why your ad-serving algorithm made a particular recommendation to a given customer?

And — no less important — how quickly can you comply with these stipulations? The CCPA, for example, requires companies with an online presence to be able to perform an audit on any consumer’s personal information the company stores — or face a fine of $7,500 per violation. If performing the audit is a slow, manual, labor-intensive process, you may avoid the fine, but your bottom line may still suffer.

Automated data lineage gives you speed and accuracy when it comes to complying with consumer data regulations. Need to delete a consumer’s personal information? Find one instance of the PII in question within your data landscape, and automated data lineage will trace its journey forward and backward. This allows you to see what happened to it along the way and how it influenced other data in your system. It also allows you to understand the data flow, dependencies and impact analysis so that you can decide how to delete the data in a way that will not distort existing calculations or reports.

Data lineage and health data compliance

HIPAA (the Health Insurance Portability and Accountability Act) has been around for over 25 years, but still presents challenges. Under HIPAA, healthcare providers must secure the confidentiality, integrity, and availability of all e-PHI (electronic protected health information) they create, receive, maintain, or transmit. They must identify and protect against reasonably anticipated threats to the security or integrity of the information, impermissible uses or disclosures.

Especially as more aspects of our healthcare are digitized and more systems enter the chain of healthcare information flow, more thought must be given to protecting patients’ healthcare data. Owning the “chain of custody” for your data is critical to a healthcare provider. Knowing where e-PHI entered your data landscape and what systems it impacted is essential for reporting, auditing and correcting issues quickly when they arise.

Automated data lineage mapping shows the chain of information flow and dependencies quickly and clearly. It’s the key to HIPAA compliance and a healthy, thriving healthcare practice.

Data lineage: supporting “great responsibility”

Take great responsibility seriously?

Just don’t want to mess with great compliance regulations?

Either way, automated data lineage is the tool that supports great power with great insight, great speed and great metadata management compliance.

It’s gonna be great!

--

--

--

Startup Nation: Israel’s nickname

Recommended from Medium

How to investigate and clean data with Python Pandas

How the WHOOP fitness wearable can help you detect a pre-symptomatic COVID-19 infection

How automation in machine learning and data storytelling can overcome the shortage of data…

3 big data analytics projects that make the world a better place

If you are looking for a one-shot way to determine brilliance, stop reading now.

Why Should the Length of Your Hash Table Be a Prime Number?

Data modeling with Power BI

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Or Hillel

Or Hillel

Helps executive teams, marketers and data analysts leverage innovative digital strategies and emerging technologies to outsmart their competitors.

More from Medium

Three Challenges in Managing Big Data

Efficient Data Engineering with Versatile Data Kit

All About Data Engineering: Exploring the Key Concepts Behind Data Pipelines and Data Warehouses

Data Engineering By a Pragmatist — Thou Shalt Not Skew (Part 2)