Top 7 characteristics of a modern data architecture

A modern data architecture (MDA) must support the next generation which is characterized by the ability to fully exploit data using exponential technologies like pervasive artificial intelligence (AI), automation, Internet of Things (IoT) and blockchain. In addition, an MDA must support a platform-centric business model that fully supports people, process and technology and is optimized around business goals.

These goals are admirable but difficult. In today’s rapidly-changing landscape, it is difficult to keep up with the latest technologies — AWS alone released over 1,800 new services and features in 2018, to their CEO Andy Jassy in Forbes — let alone the most optimal frameworks to deploy those technologies.

If you ask your product vendors for their thoughts, they tend to get really excited and rattle off their entire product catalog hoping to convince you of their approach, build a product-centric solution and meet their sales target for the year. If you ask your favorite IT person, you may get a narrow view based on a combination of his/her experience and a desire to learn a new marketable skill set.

To thwart these potentially damaging efforts, my goal is to equip you with a short list of my top seven characteristics of a modern data architecture, in no particular order.

Since I am a practicing architect, I need to provide a disclaimer that my full list of characteristics is definitely more than seven. And I’m sure there will be debate about the seven I selected. In fact, I’d love to hear directly from you with your top characteristics. But I aimed to start with a fairly succinct list that could be used as a checklist by you to keep your vendors honest.

To that end, the MDA can be characterized by the following:

• Cloud-native. Designed to leverage the strengths and accommodate the challenges of a standardized cloud environment, including ability to support elastic scaling, high availability, ironclad end-to-end security for data in motion and at rest, and cost and performance predictability.
• Robust and scalable data pipelines.The MDA needs to support real-time data streaming and micro-batch data bursts comprised of a set of functional architecture components and services that combine events, real-time integration, data, cognitive analytics and intelligent workflows in a single integrated framework. It’s where business processes can be orchestrated efficiently and cost effectively, and security and compliance is applied at its core.
• Seamless data integration. This means the ability to integrate seamlessly with legacy applications based on standard API () interfaces. A modern data architecture does not need to replace services, data or functionality that works well internally as part of a vendor or legacy application. Instead, it is optimized for sharing data across systems, geographies and organizations without hundreds or thousands of unmanageable point to point interfaces.
• Real-time data enablement. Enterprises need the ability to deploy automated and active data validation, classification, management and governance with complete and visible data lineage and associated metadata, incorporating a comprehensive and searchable data catalog updated in real time and enabling effective self-service data access and data discovery. This concept is driven by the adage that “whoever creates the data owns the data.” Policies are driven by that adage in which the architecture enforces governance in real time to efficiently integrate, curate and optimize an enterprise’s data-eliminating data swamps and never-ending, costly and unsuccessful post-processing data cleanup initiatives.
• Decoupled and extensible. The MDA is designed and built based on a loose coupling architecture where services are defined to do one thing and have no knowledge or dependency on another service. They also conform to open standards-based tooling, platforms and frameworks to drive interoperability and portability in standardizing to a core set of platforms, applications and tools across a multi-provider, hybrid IT environment designed to allow the rapid addition of new capabilities and functionality. Because of this extensible framework, MDAs can support the latest advancements around AI and machine learning, intelligent workflows and robotic process automation (RPA), edge computing, IoT and serverless processes.
• Domain-driven, event-based and microservice-enabled. The MDA is driven by common-data domains, events and microservices. It is centered around a common business information model (BIM) defined to fully represent the business data domains and act as the foundational cornerstone for the entire framework. Each data artifact, topic and microservice is associated with a business data domain based on the BIM. The MDA is also designed to be triggered by business events initiating the appropriate choreographed processes and workflows that are implemented using microservices. Because the approach is consistent and the architectural principles are sound, adoption by the business is rapid and complete.
• Balanced. The MDA helps businesses balance cost and simplicity with fit-for-purpose. It does not aim to use every technical “arrow in the quiver,” but instead uses a best practices approach to apply the best solution to solve the business problem and still deliver an appropriate ROI for the business. This requires full visibility and understanding of all the requirements and variables for each situation including resources, skills, licensing, or politics so that the best solution can be provided.

The MDA drives the interconnectedness of the cognitive enterprise and supports exponential technologies that are fueled by clean and contextual data in order to use next-generation applications on a multicloud environment. The MDA is not built in a day, however. It all starts with a holistic, business-driven data strategy to support business goals and strategic vision. Once that strategy is defined, then the MDA can be deployed across the enterprise in an incremental, prioritized fashion where starting small and iterating enables business benefits very quickly.

Connect with me on LinkedIn and share your thoughts on modern data architecture.

Originally published at https://www.ibmbigdatahub.com.

Disclaimer: This blog represents my own personal view and is not related to my current employer or past employers.