Packaged vs Headless CDP: Which one is right for you?

The interest and adoption of packaged Customer Data Platforms (CDPs) spiked over the last years, but now the times are changing. Many within our industry are questioning the effectiveness, future potential and cost of packaged CDPs.

In this blog series, we will discuss the points necessary in deciding between a headless CDP and a traditional, packaged CDP.

What is a Packaged Customer Data Platform (CDP)?

A packaged customer data platform (CDP) is a type of software solution that is designed to help businesses collect, unify, analyze, and activate customer data from various sources across their organization.

Unlike traditional customer relationship management (CRM) software that focuses on managing customer interactions and sales processes, a CDP is specifically designed to provide a unified view of the customer that can be leveraged by various departments within an organization, including marketing, sales, customer service, and product development.

A packaged CDP is a pre-built software solution designed to be deployed and configured quickly and easily, often with minimal IT involvement. These solutions typically come with a set of pre-built integrations with popular marketing and advertising platforms, as well as pre-configured data models and analytics dashboards to help businesses get up and running quickly.

Packaged CDP vendors claim their product is the solution to the problem but ignore that it will take an ecosystem to solve all customer data use cases across various tools and downstream teams.

Public cloud data (warehouse) platforms offer this ecosystem. Overlap with packaged CDPs is growing as it has become much easier to build, or more accurately, assemble, the functionality that existing CDPs are offering.

What is a Headless Customer Data Platform (CDP)?

A headless Customer Data Platform (CDP) is an increasingly popular solution implemented without a packaged CDP tool or suite where:

• A cloud data warehouse, such as GCP BigQuery, Snowflake, AWS Redshift, acts as the data foundation. Different data sources are ingested, transformed and joined.
• Actions and insights are created on all levels, not limited to customer data. Think of product and campaign-level data.
• Activation of data in channels and platforms is implemented directly from the cloud data warehouse environment where the central view lives.
• A combination of cloud-native (data) services and specialized tools or frameworks are used.

As discussed in our article ‘The real CDP revolution is in public clouds’, we noticed the movement to the public cloud among our clients and shared our observations and experiences.

Headless CDP in more detail

A headless architecture gained popularity in the Content Management (CMS) world and is widely used these days. By decoupling the back-end (content management) and the front-end (website), the CMS now acts as a central content hub. Through an API, content and actions can be shared with all sorts of applications, such as websites, apps or even dynamically generated emails. The front-end is no longer intertwined with the content management system. The result: all content is centralized in one system, and front-end applications can be developed independently from the CMS.

The headless CDP acts as a (marketing) data, decisions and activation hub where insights and actions can be created on all kinds of levels, and these are activated / integrated with channels, tools or platforms (think of audiences, events or triggers). This is done directly from the cloud data warehouse, without the need for a stand-alone CDP tool or suite.

However, there is no centralized user interface. The solution consists of a combination of services within the cloud environment (building blocks) and other (specialized) tools that fulfil a specific need (e.g. connectivity or decision-making).

Not limited to customer data

We are comparing the headless approach to existing stand-alone CDP solutions, which are mainly built around customer data. However, a headless data management system is not limited to customer data only. It supports interoperability with the enterprise data platform. Some examples:

• Identity management and an identity graph to link customer and organizational identifiers.
• Building a single (360°) customer view and (customer) audiences.
• Gaining insights into product performance and enriching product data / feeds.
• Creating models, such as product recommenders, engagement, conversion scoring, CLTV, churn prediction models or forecasting sales. These models can be used in batch and real-time. Directly in the platform, so the data doesn’t have to leave.
• Creating business rules or “triggers” (decisioning) for follow-up / Next-Best Actions.
• Sales and engagement reports based on multiple data sources. Creating notifications and alerts based on deviations within these reports.

The headless approach has a lot of overlap with the idea behind the modern data stack, also called the composable CDP. That is mainly because you can use both products from the cloud vendors and ISV SAAS solutions for every function of the platform.

A data warehouse ‘with benefits’

Many organizations recently invested in a data warehouse from one of the well-known cloud providers such as AWS, Azure or GCP. Data can be retrieved from various sources and joined together. Traditionally data warehouses were populated using periodic (daily/hourly) batches, but modern cloud data warehouses have native support for real-time data ingestion. Having live orders, delivery and clickstream data from websites and apps is more common in cloud data warehouse implementations.

Working with such a cloud data warehouse and all the different services within these cloud platforms became much easier. Services within the cloud platforms can be stacked on top of each other (building blocks), making it easier to import, transform and analyze large amounts of data. Skills that are important for working with such a data environment, such as SQL, Java or Python, are also increasingly present within organizations and their suppliers.

Reverse ETL, which means activating this data in channels and/or tools directly from the data warehouse, is what increases the overlap with existing CDPs, hence becoming an alternative.

The headless CDP approach vs. CDP tools: functionality comparison

In contrast to headless CDP, the packaged CDPs usually specialize in different areas. Some platforms prioritize activation, personalization, and AI, while others focus on generating insights, identity resolution, and a comprehensive 360° view of the customer. Despite their differences, nearly all of these systems require the collection and ingestion of source data. Typically, this involves implementing additional on-site trackers using Javascript, as well as uploading first-party data to combine online data with order or CRM data.

When talking about a packaged and a headless CDP when a cloud data warehouse environment is already present, the following comes up:

1. The data sources that already flow into your data warehouse, also have to be integrated with the CDP.
2. Capturing and collecting real-time behavioral data is most of the time already implemented by systems such as Google Analytics or Snowplow. So why, in the case of a packaged CDP, would you add another tracker on your website/app that potentially slows down your website and which measurement methodology certainly will differ from the measurement you are already familiar with?
3. Duplicating business rules already present in your warehouse environment, such as identity resolution, creation of a 360 customer view or audience selections.

If the above applies to your organization, there will be overlap. Most packaged CDP and data management tools do not work together in such a way that they can use on everything that is already collected and calculated in your data warehouse.

Disadvantages of a headless CDP

Stand-alone CDPs also have their advantages. First off all, if you don’t have a data warehouse (yet) and/or resources are limited, a packaged CDP suite could fit your needs and get you started quickly, seeing the out-of-the box functionality. The cost factor heavily depends on the type of system, volume and integrations. When zooming in, the headless CDP has the following disadvantages:

1. No central user interface & ease of use: a headless system does not have an integrated user interface where settings can be easily adjusted or insights can be given into all processes. It is certainly possible to gain insight into the processes of a headless system, but those insights have to be created yourself or are available in multiple systems. There might be technical / SQL knowledge required to set up audiences or journeys for example. An option is to put a low code application on top such as Google’s Appsheet or using Looker Blocks and Actions.
2. (Real-time) decision engine / flow builder: making decisions based on real-time data flows (e.g. web or app data). Creating such a real-time decision engine (e.g. online personalization use cases) within the cloud is possible, but is more complex than clicking rules together in a user interface because that is code based. Also, creating flows / journeys is easily done within most CDP solutions. Often we use Google’s Cloud Composer for batch orchestration. For real-time we use a Cloud Function and PubSub.
3. Connectivity to channels and tools is maintained by the platform. In a headless CDP, you have to build a part of the integrations yourself (mostly through API connections), although there are currently several (open-source) frameworks available, such as Google Tentacles and “reverse ETL” tools such as Looker, Hightouch, Flywheel Software, Census.
4. The time-to-market is longer because the platform has to be designed and built.

If your organization is not complex, if the use cases you are looking for are basic and if you don’t want to hire a services company like Crystalloids, then a packaged CDP is a good fit. However, be aware that if a packaged CDP fits your existing requirements, there is a great chance that there will have limitations when your maturity grows.

Reverse ETL

There are already a lot of tools that ingest data into your warehouse. More recently, a new segment has entered the market: reverse ETL. These tools can activate data to your marketing channels or other platforms. Reverse ETL tries to solve this problem. A reverse ETL tool enables you to focus on building use cases and not to worry about connector development and maintenance (e.g. Flywheel Software, Census, Hightouch, Looker Actions, Segment). Next to that, the tools are providing insights in the process and data that flows out of your cloud platform.

An example setup:

Next to using Google Cloud-native tools, also third party ISV SAAS point solutions can be used for ingestion, transformation, analysis or governance. If you do so, you get a hybrid composed CDP. Crystalloids helps you in determining what works best for you.

When to choose a headless CDP approach?

When your organization already has a cloud data warehouse or planning to set one up, the headless approach is interesting if you:

1. Don’t want to purchase and implement an (expensive) CDP tool, for example if you want to share audiences, triggers or other data directly from the data warehouse with your (marketing) channels or (external) platforms.
2. Want to realize advanced and more complex use cases based on different sources and combinations, which are not possible or hard to achieve in packaged/stand alone CDP tools (by clicking stuff together) or without moving the data out and in the CDP platform to perform the necessary actions.
3. Do not want to duplicate data and recreate business rules in different environments (single source of truth).
4. Want to optimally leverage existing investments in a cloud data warehouse. Even if your data lake resides on a non-Google Cloud, Google Cloud still remains the most suitable option for hosting your CDP.
5. Unlock the potential of and activate non-customer data, such as product or campaign data.
6. If you want to reduce vendor lock-in of systems and applications.
7. If the cost of the packaged CDP grows 10X while your business grows 2X. A headless approach cost only increases with additional processing and storage needed to handle the growth. Until now we’ve seen headless being more cost effective.
8. Using various cloud services and dedicated / specialized tools (such as reverse ETL), it’s possible to connect data and tools relatively easily, bringing flexibility of choice.
9. If you want to enjoy flexibility in on- and offboarding point solutions in the future. Which you will because nothing stays the same for always.

Why Integrating GCP Can Enhance Your Existing non-Google Data Lake Infrastructure?

What if your business already has a datalake on AWS, Azure or another cloud environment? It’s understandable that you want to safeguard your investments in these clouds and don’t want to replicate.

In such cases, we don’t build another datalake. We only bring the data to the CDP on Google Cloud that is needed for the marketing and sales use cases and link the results back to the datalake and other systems. Adopting a multi-cloud strategy is a trend where companies pick the best cloud for their use cases.

GCP offers seamless integration with various data sources, enabling you to unify your data and access it easily from a single location. This can save you valuable time and resources that would have otherwise been spent on managing disparate data sources. So, while safeguarding your investments in existing clouds is important, it’s worth considering the benefits of integrating GCP to unlock the full potential of your data.

Google Cloud is very well positioned for headless CDP for several reasons:

• the native connectors with advertising platforms
• the AI capabilities that are nicely tuned for marketing and advertising

Next to that, we adopt a serverless platform meaning that no technical maintenance is required.

Realtime headless CDP applied by Crystalloids

The data flows and integrations that we have developed leveraging the cost effective and real-time capabilities of the Google Cloud Platform, are pre-defined building blocks that can be tailored to your specific requirements. Instead of packaged software, where you depend on the existing functionalities and roadmap of the supplier, we build a solution that will always meet your present and future demands. Therefore our headless CDP (API driven) Data Platform offers more flexibility to define your own rules regarding aspects such as user stitching or building audience segments.

The advantages of a headless CDP

• More Flexibility

By using a modular, loosely coupled architecture and persisting data in a real-time cloud data warehouse, the headless CDP approach provides the flexibility necessary to quickly innovate and have an agile roadmap.

• Reduced Vendor Lock-In

Unlike pre-built CDP options, data is not “locked” into the headless CDP approach. The “single-source-of-truth” is available for all applications, reducing the risk of vendor lock-in.

• Real-Time Loop-Back

The headless CDP approach also allows for an easy and real-time loop-back to source systems that require consolidated data from the CDP.

• Cost Efficiency

The use of cloud-native components with a pay-as-you-go pricing model makes the headless CDP approach cost-effective with low total cost of ownership.

Conclusion

While traditional stand-alone CDP solutions have been a valuable tool for businesses looking to manage customer data, the rise of headless CDP solutions is a game-changer. By decoupling the front-end user experience from the back-end data management, headless CDPs provide businesses with the flexibility, scalability, and agility they need to stay ahead of the competition.

If you’re looking to take your customer data strategy to the next level, a headless CDP may be just what you need. With its flexible architecture, cloud-native design, and robust data management capabilities, a headless CDP can help you unlock the full value of your customer data and stay ahead of the competition in today’s fast-paced digital world.

At Crystalloids, we have architected and developed public cloud data (warehouse) ecosystems for over eight years. We call it a headless Customer Data Platform or Unified Marketing Technology Stack. When we started, the terminology “headless” was unknown within our industry. The headless approach has become mainstream, with headless CMS and more.

If you want good advice from wise and experienced solution architects on your data strategy, leave us a comment below. We also offer a CDP workshop free of charge to get you started.

ABOUT CRYSTALLOIDS

Crystalloids connects IT and business with scalable and flexible solutions. As a Premier Google Cloud Partner, Crystalloids is a specialist in end-to-end data management, including BI, data science and activation. Using transparent, agile development, Crystalloids ensures that use cases deliver immediate value and that our customers are in complete control.