4 Things You Need To Consider For Managing Access To Your Data
Introduction
Determining the right approach to managing access to data is a crucial part of defining and executing any organisation’s data integration strategy.
Many approaches can be taken to consolidate data silo’s within an organisation, data hubs, data lakes, data warehousing or data virtualisation, naming just a few.
Regardless of approach, the Achilles heel of these models often comes back to how granting access to data consumers is managed. A robust, pragmatic approach to governing and managing to access your data assets is as critical as the approach to integrating it.
When discussing data access, many people immediately start to think of authentication. Determining the authentication model is typically the easy part. The more challenging decisions are how to apply authorisation and access control within your data integration approach.
Finding the right balance for data access is difficult. Set access rules too tight, and data consumers will look to bypass the access controls or create new silos to avoid it. Setting access rules too loose, and it places the confidentiality and integrity of data at risk.
This article talks about best practise approaches to establishing a multi-layered data access model. Let’s first look at the typical challenges that this model addresses.
Challenges To Managing Data Access
Transitioning to data-centric access policies
Many organisations dealing with legacy data silo’s are likely still managing data access at a system level with potentially limited restrictions.
This approach becomes a more challenging issue when shifting into more modern data integration platforms that typically abstract the ultimate source of data, from the consumption of it. Maintaining those same controls using system-level access won’t work when migrating to a new model. Instead, data consumers will look to integrate using data-driven models for accessing data, regardless of the source system that provides it.
Shifting away from system-centric access to data-centric access can be difficult, and the approach taken to that transition is equally challenging.
Balancing usability with security
Another challenging issue to data access is to balance your need to the security of the data, whilst still enabling the increased accessibility that a modern data consumer expects.
The risk appetite of an organisation, in balancing restrictiveness against usability is different for everyone. It requires striking the appropriate balance.
Striking the right balance for restricting access to data will ultimately come back to your organisations appetite for risk. Highly regulated industries typically require tighter restrictions as opposed to those industries that can operate with more flexibility.
The user experience for data consumers is important to factor in.
The ultimate experience of data consumers is a crucial factor. Not capturing feedback upon poor experiences means the ‘people’ aspect of any solution will seek alternative ways for accessing data or build yet another silo to avoid using it.
Establishing automation to operate at scale
Manual workflows and approve processes becoming a challenge when transitioning these more modern approaches, they ultimately slow down consumers from accessing the data they need when they need it, obviously detracting from the business case, but also detract from the value proposition.
Maintaining manual processes also constrain good governance by enabling inconsistency in access provisioning. Though not everything can be automated, particularly where access is requested is for sensitive or critical data, the goal should always remove manual steps where possible.
Our experience has shown that data access processes need to be automated to scale with the data integration platform.
Approaches to establishing multi-layered data access
1. Data access policies need meta-data
Any data integration strategy will cover the need to establish meta-data management across information assets. It’s also a pre-requisite to establishing good data access management and governance. To put it simply, you can’t secure access to data if you don’t first know where it is.
Metadata management allows data consumers to search and identify information based on specific attributes. Data access needs to be closely aligned with meta-data management to enable and apply data-driven access policies.
Whether you look to define data-access rules based on data domains, fields or records, it needs to maintain an association with the metadata management system.
2. Data access policies need to be abstract and centralised
Don’t expect that consumers will come as just ‘one size’ in how they want to access data. Consumers will have different requirements to how data is accessed, whether it be via real-time streaming data versus batch ETL or consumption via Rest APIs versus legacy FTP.
Data access rules should be expressed in a high-level declarative language. That approach allows them to be applied within various technology services associated with the data integration platform.
Maintaining declarative access rules in a central location means that regardless of the requirement or the technology services providing it, access rules can be consistently applied.
3. Data access policies need to be defined as multi-layered
Data access rules should be defined across multiple layers. This allows rules to be defined based on the different stakeholders who will be responsible for managing and approving. That layered approach provides flexibility to manage the lifecycle of these rules and to allow for data access rules to be incrementally matured over time.
The following diagram below provides an example to how those layers can be defined.
The bottom layer of the diagram shows organisation-wide rules defined as the default. This rules should be reflective of your organisational data management and security standards. These rules apply for all consumers and will likely be the most restrictive. Rules at the bottom level are more static and defined by senior executive levels to reflect organisational requirements for data access.
The next level above is for data access approvals, allowing for groups of consumers to access data domains or data fields. The approval process will look at assessing the business requirements for data consumers to access particular data. The creation, update or removal of rules at this layer, is the responsibility of data owners or data stewards for the organisation.
The following level is for record-level access or consent. Record level access rules are anchored to consent granted by individuals who may not be the owner for the entire data set but have ownership over particular records associated to them. A good example is health care records where individuals may want to manage (or grant consent) to how data associated to them is accessed.
Lastly, the top-level represents exceptions. Data access policies need to cater for exceptions within any of the below layers. Regardless of how well you define your data access governance model, there will be exceptions that need to management.
4. Data access policies need to be automated
Managing the lifecycle for data access policies, particular for data access requests and approvals must be automated where possible.
Consider how you can automate access to data with a lower sensitivity or criticality by using pre-approvals for particular data access requests. Pre-approvals are useful (as opposed to just allowing open access) as they still prompt the consumer to evaluate and justify their rationale for accessing this data.
Pre-approvals reduce fatigue within the data access approval process and enable greater focus to be given towards the remaining requests, being intrinsically more ‘sensitive’ data.
The key concepts for ‘pre-approval’ are
- Data consumers are not provided access by default, but ‘on request’.
- Each request is tracked and audited, allowing visibility and insight across data access.
- Notification is provided to the relevant stakeholders for retrospective review. Data owners reserve the right to retrospectively revoke access, where it not appropriate.
Pre-approvals are also a good mitigation to the risk of accidental data loss, limiting the blast radius of any exposed data. For example, if a data consumer inadvertently executes a task to copy all data that they have access to, then only the datasets that they have explicitly requested via pre-approval are exposed.
Final Notes
Regardless of your organisation’s strategy or approach to data integration; how you govern and manage access to your data assets is critically important.
It’s essential to get things right up front when transitioning to a mature data integration platform.
What to learn more ?
More and more organisations are rethinking their approach to legacy data and application integration by adopting cloud-based solutions.
The Gyfon team are here to help organisations work through these challenges. Our consulting practice specialise in design for large scale data integration platforms in both Azure and AWS.
If you’re considering such a move or just want more information, we’re here to help. Simply visit us gyfon.com or email us at info@gyfon.com and we’ll be in touch.
Good luck!
