Segmentation.ai: Automated Audience-Clustering-as-a-Service in Adobe Experience Platform

Jaemi Bremner
Jan 30 · 6 min read

This post describes our innovative audience segmentation service and how it empowers Adobe Experience Platform customers to easily explore and understand their audiences for effective targeting.

With the exponential influx of data and the growing customer expectation for tailored experiences, marketers can no longer heap an entire audience into a single bucket. Segmenting these audiences provides some guidance, but traditional segmentation based on age, gender, and income fails to take into account customers moving from device to device and channel to channel.

Adobe Audience Manager is a leading data management platform that reimagines audience segmentation. It enables modern marketers to seamlessly collect data from multiple sources, build focused audience segments (or clusters), and follow their customers’ journey to deliver relevant messaging in real-time.

It still, however, requires manual input to create each segment, along with some expertise to understand the data. With Segmentation.ai, we aim to simplify this process by automatically creating audience clusters and condensing insights into interactive graphics for easy exploration and comprehension.

Audience segmentation in Adobe Audience Manager

The problem with the current audience segmentation is that the overwhelming amount of collected data makes it difficult for users to create representational segments and explore important insights. Another speed bump is the need for users to manually define rules in Adobe Audience Manager for segment activation. A rule being, for example, “users aged between 25–30 and studying engineering.”.

To remedy this overbearing process, Segmentation.ai uses customer traits and segments to achieve the following:

  • Create an accurate timeline of user interactions.
  • Shrink lengthy user journeys — where each has differing numbers of traits and segments — into equal, 16-digit vectors for easier analysis.
  • Convert audience clusters into interactive, graphical visualizations that non-domain experts can swiftly explore and understand.
  • Create a streamlined, end-to-end solution for segment activation.

Additionally, this innovative service relieves customers from bloated, overly-complex dashboards; instead of allowing them to interact with their audience segments through a clean, focused, and user-friendly interface.

Segmentation.ai architecture

As shown in the figure below, the Segmentation.ai service is comprised of five stages: pre-processing, modeling, clustering, discovery, and ingestion.

Figure 1: Diagram of Segmentation.AI architecture.

To better illustrate each stage, we will explain using an example.

Pre-processing

Segmentation.AI then embeds these traits and segments into a user journey — building a timeline that accurately represents the user’s behavior.

For this, we run a PySpark job on Databricks for compute, along with Data Science Workspace recipes. The lookback window is currently seven days, and we automatically determine which traits and segments are the most meaningful for the customer.

Modeling

The service now has the UX design customer’s user journey, along with the user journeys of millions more who have interacted with Adobe products. Although some users have shorter journeys than others, and longer journeys can contain thousands of traits and segments. This results in huge amounts of uneven data that is understandably difficult to work with.

To solve this challenge, we used Tensorflow and a deep learning auto-encoder to reduce each user journey to just 16-dimensions — while preserving the user’s behavior. Now, Segmentation.AI can comfortably digest these smaller user journeys to train and score the embeddings model.

Clustering

Since every user is on the website for different reasons, the service shows high-level clusters for the Adobe products the users interacted with.

Here, we run clustering on the sampled embeddings dataset.

We also simplified the user data even further since 16-dimensional data isn’t easy to explain. So, we partnered with ShapeVis for topological data analysis. This allowed us to group user data and generate a summarized, two-dimensional graphical representation, all while maintaining its inherent structure.

Discovery

Now, the service is ready for the marketer to explore the clusters and determine meaningful audience insights. They can navigate to the Segment.AI module within Adobe Experience Platform and view the high-level clusters that represent the users who interacted with Adobe products. (See image below.)

Figure 2: Sample of clusters.

The clusters with darker colors represent the highest concentration of user traits.

Ingestion

Once the marketer has found clusters of interest, they can click to expand them and view meaningful insights about the audience, such as the main users for a specific product and what device they’re using. Then, to activate an audience segment, they can simply ingest it into the Real-Time Customer Profile.

For this final stage, we have two parts:

  • Segment expansion: Since the clusters showed are of sampled data, we allow the customer to expand a cluster and view all the users in the complete dataset. This is done by running a PySpark job on Databricks.
  • Segment ingestion: In the same job, we create a new batch in a dataset marked for Real-Time Customer Profile ingestion, along with segment metadata for the new audience. Currently, it can take a few hours for the segment to be ingested.

Implementation challenges

Another challenge was explaining the smaller audience clusters, as some traits captured much more information about the user than others. For example, a large cluster could easily reveal that most users were using iPhones; but a small cluster discerning the specific iPhone models would be much more difficult to explain.

What’s next for Segmentation.AI

  • Enhance the user interface: Currently, our customers rely on Adobe support for model configurations. We aim to build a simple and accessible user interface that will enable customers to take charge of this process.
  • Add unified data support: To extend the power of Segmentation.AI, we are adding support for different Experience Event datasets, such as Adobe Analytics data. Today, this technology only works with data from Adobe Audience Manager.
  • Add real-time segment description: With the addition of segment descriptions that function as queries, customers will be able to further refine segment creation and streaming.
  • Build objective-driven segment discovery: Using machine learning models, this type of segment discovery will swiftly create segments that optimize specific marketing goals.

With these improvements underway and many more innovations on the horizon, we continue to help Adobe Experience Platform users fine-tune their marketing efforts and drive powerful, customer-focused experiences in real-time.

To learn more about what our developers are working on, follow the Adobe Tech Blog for weekly developer stories and resources. For exclusive posts on Adobe Experience Platform, follow Jaemi Bremner.

Adobe Tech Blog

News, updates, and thoughts related to Adobe, developers, and technology.

Jaemi Bremner

Written by

Experience Technologist. Developer Advocate for Adobe Experience Platform. Passionate about technology, architecture, fashion, and design. Twitter: @jaeness

Adobe Tech Blog

News, updates, and thoughts related to Adobe, developers, and technology.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade