Choosing the right analytics tool: Insights from our Contentsquare journey
As Data Analysts at Welcome to the Jungle (WTTJ), we often face straightforward data questions like “What is the click-through rate on this button?”, “How many people visited that page?”, or “What are the main landing pages for a specific cohort of users?” Before we could find the answers, we often found ourselves going through a series of time-consuming tasks: Setting up tracking plans, ingesting and modeling new data, or crafting SQL queries of varying complexities. Hours, sometimes even days, could pass before we were able to provide what seemed to be simple numbers to outsiders.
As our company grew from 150 to 350 employees over the past 3 years, the demand for data-driven decision-making grew significantly. With more individuals seeking insights, we encountered constraints in our capacity as Data Analysts to quickly and efficiently answer those needs, resulting in team frustration. Acknowledging this challenge, we recognized the necessity to empower team members across the company to be able to independently access data. This strategic shift not only alleviated decision-making bottlenecks but also paved the way for promoting data autonomy throughout WTTJ.
That’s why we embarked on a quest for a complementary analytics tool. A tool that could improve the way we handled data inquiries. A tool to streamline the processes, boost efficiency, and ultimately unlock the full potential of our data-driven efforts.
It’s worth mentioning that, before joining WTTJ, I worked at Contentsquare, a UX analytics tool, where I held various data positions, including educating customers on technical data aspects of the tool, and later working on developing new features for the product. This multifaceted experience provided me with valuable insights that contributed to a nuanced understanding of our requirements and possible solutions to fit our needs. At WTTJ, we finally chose to work with Contentsquare, but it’s important to note that our choice was not influenced by my previous job, rather it stemmed from a comprehensive evaluation of the tool’s alignment with our needs.
In this article we will share our 2-year journey of selecting, implementing, and utilizing Contentsquare at WTTJ.
1. The limitations of Snowplow analytics: Why we needed more
In 2020 we started using Snowplow, an open-source tracking tool to capture data effectively. It quickly became the backbone of our data strategy, enabling us to uncover super-important insights and provide our organization with data-driven decisions.
Thanks to Snowplow, we had the ability to dig into granular and customizable tracking. This helped us get a clear picture of how users were doing things with our product. Plus, the cool thing was that we had full control over everything. It gave us tons of freedom, granting us access to raw data, preventing the black box effect, and ensured that we handled privacy and security concerns the right way.
However, as our company continued to grow and our interest in data increased, relying solely on Snowplow was no longer enough.
The example mentioned earlier in the introduction was merely a glimpse of the numerous challenges we encountered. The continuous updates to our tracking plan resulted in a long development cycle, which became burdensome. This cycle involved identifying new events or user interactions requiring tracking, devising detailed tracking plans to capture these elements, implementing the necessary code changes, testing for accuracy, and then finally deploying the updated tracking. The complexity and time required for each step sometimes hindered our ability to respond promptly to evolving data needs. Moreover, following UX changes with a standard tracking plan was complicated, as we needed to specify every possible scenario (if the user clicks here, send origin = A; if he clicks here, send origin = B, and so on), which led to endless tracking specifications. Additionally, the absence of a user-friendly interface left our non-technical teams without access to data insights. Even simple metrics like click rates or user segmentations required convoluted SQL queries, adding complexity to our path toward data autonomy.
Consequently, we began considering the integration of an analytics tool featuring automatic tracking alongside Snowplow, to enhance our data analytics ecosystem.
Snowplow always remains our source of truth, as it grants us access to the complete data journey, from its raw, unprocessed form to the meticulously modeled data, empowering us to diagnose and resolve any discrepancies or issues. Snowplow is the sole tracking data we ingest into our data warehouse, which we combine with other data through modeling and share internally and externally.
This approach sought to strike a balance, capitalizing on the advantages of both solutions to empower our data teams to analyze and utilize data effectively.
2. The journey to Contentsquare: The selection process
Over the course of a quarter, we embarked on a careful journey to find the perfect analytics solution to complement Snowplow. Here’s how we made our decision:
Identifying challenges
We began by listing all the existing difficulties we encountered with Snowplow, as well as new challenges we wanted to address, like UX improvement.
Extensive benchmarking
To ensure we explored all available options, we conducted a rigorous benchmarking analysis, evaluating more than 20 competitors in the market.
Prioritizing features
We categorized analytics tools’ features according to three levels of priority — critical needs, essential needs, and bonus features — aligning them with our specific requirements.
Comparative analysis
To gain a holistic understanding, we constructed a comprehensive comparison table, assessing how each of the 20 competitors would address our needs.
Interactive demos
We took the next step by booking demos with four promising contenders, digging into their capabilities and functionalities.
Thorough testing
Further narrowing down the options, we put two of the shortlisted tools through rigorous testing over several weeks, while also actively seeking feedback from clients who had experience with these solutions.
Finally, after careful consideration, we arrived at our decision to collaborate with Contentsquare because it not only provided all the functionalities we required in a more advanced way compared with its competitors, but also we had the advantage of having a power user already within our company (me, as a former employee), which greatly facilitated the adoption process and gave us huge confidence in choosing it as the solution.
3. How Contentsquare meets our needs
Certain functionalities of Contentsquare distinctly align with our requirements, and in this section we will expose some of them in detail, shedding light on why we ended up choosing Contentsquare after this long selection process.
- Optimizing data implementation: In our quest to simplify the laborious process of manually implementing data tracking for every new feature, Contentsquare emerged as a solution. Its JavaScript tag promises to automatically track all page views and events across our website. This eliminates the need for extensive manual tracking plans, which would be time-consuming and resource-intensive for our data, development, and QA teams to implement. What adds significant value for us is the retrospective accessibility of all this data. With Contentsquare, we no longer have to anticipate data needs in advance.
- In response to our need for greater autonomy among non-data teams and improved data accessibility, the Contentsquare platform made the data available through a user-friendly interface accompanied by intuitive visualizations. This accessibility extends beyond data experts, allowing individuals across various disciplines within our organization to analyze user behaviors effectively.
- Recalling an example from our introduction — “What is the click-through rate on this button?”, which previously required complex SQL queries from Data Analysts — the Contentsquare Zoning feature offers a promising solution. With its detailed and customizable heatmaps, distinct from conventional static ones, it provides a quick and simple avenue for accessing zone-specific data, including click patterns, hovers, and conversions. This accessibility greatly aids conducting thorough analyses and ensures that non-data teams like Product Managers and Designers can easily access vital insights, all without the need for Data Analysts.
- Customer journey analysis: Understanding how visitors navigate our website, from their initial interaction to their final destination, is critical for uncovering user needs and areas of frustration. This valuable analysis can shed light on opportunities for site optimization. Without Contentsquare, this is a complex task. Analyzing user journeys involves creating intricate SQL queries to trace actions from page to page, often leading to hard-to-interpret tabular results. Even when utilizing tools like Looker, visualizing user paths beyond the second page and across multiple entry points remained challenging.
Contentsquare’s sunburst visualization allows us to quickly and intuitively map user journeys. We can effortlessly filter and analyze specific user segments, start journeys from the landing page or from designated pages, and even create inverted sunbursts to identify paths leading to specific pages of interest.
Furthermore, Contentsquare’s capabilities enable us to delve deeper into identified journeys, with automatic segment creation from the sunburst streamlining our optimization efforts. This powerful tool has simplified our ability to decode user behavior and optimize our website effectively.
- Advanced segmentation: In our analysis, the need often arises for intricate and precise segmentation that goes beyond the capabilities offered by competitors’ simpler tools, typically limited to 1 or 2 conditions. Contentsquare comes to our aid by providing highly sophisticated segmentation options. This enables us to construct chains of conditions, such as identifying users who viewed specific pages then navigated to other pages and eventually clicked on specific zones. This advanced segmentation is invaluable as it allows us to conduct incredibly precise analyses of user behaviors, an essential aspect of optimizing our platform and providing the best possible user experience.
- Session recording: This feature serves as a valuable asset within our data-analysis toolkit, enabling us to directly witness how users engage with our products. This qualitative data is pivotal in informing our user experience enhancements and aids in the illustration and comprehension of specific user behaviors. While quantitative data from other features might flag issues on certain pages or zones, session recording goes a step further by allowing us to pinpoint precisely what is problematic. By observing users’ interactions in real time, we gain valuable insights into their struggles and challenges.
In addition to listing all those functionalities, it’s essential to emphasize that Contentsquare was chosen to complement our current solution, Snowplow. Unlike some of the other options we considered, which were too similar to Snowplow in terms of analytics and tracking, Contentsquare aligns better with our complementarity objectives.
4. The implementation journey
When it comes to implementing Contentsquare, the commercial pitch might promise a simple copy-paste of the JavaScript tag, but it can end up a more complex story, depending on your site’s tracking maturity. In our case, we embarked on two distinct waves of implementation that explored vastly different paradigms: One on a highly mature product with lots of existing data, and the other from scratch.
Firstly, in early 2022, right after signing up with Contentsquare, we implemented the tool on our well-established B2C site, which was already data-mature. Leveraging our existing setup with Google Tag Manager (GTM), where we implemented advanced event tracking and utilized variables within the data layer, proved advantageous. With a robust data layer in place, filled with most events and pageviews already sent through GTM, incorporating Contentsquare was as straightforward as adding the Contentsquare tag within GTM, with a few necessary adjustments, particularly concerning pop-in handling. After just a week of implementation and regular pairings with the Implementation Manager from Contentsquare, the mission was accomplished.
Encouraged by the successful adoption of Contentsquare on the B2C side by the end of 2022, we made the decision to implement it on B2B products. In contrast, these products were in an earlier stage of development concerning data infrastructure. Consequently, we were faced with a more time-consuming implementation process, given that we essentially had to create the data layer from scratch. This endeavor required weeks of dedicated work, involving the collaboration of our developers to establish and populate the data layer and address product-specific intricacies and other related tasks. While this implementation phase demanded additional time and resources, we received lots of support from Contentsquare’s Implementation Manager. Once the data layer was comprehensively populated, the integration with Contentsquare progressed swiftly, taking only a few days across four products. The primary complexities stemmed from Content Security Policies (CSPs) and the GTM script. However, once these hurdles were overcome, setting up the Contentsquare tag, variables, and mapping within Contentsquare itself proved to be a relatively straightforward process.
Although the implementation might seem challenging, it’s much less so compared with what other tools like Snowplow might need. For instance, with Snowplow, introducing a new page or feature involves a laborious process: Data analytics teams create tracking plans, sometimes requiring complex modeling when data doesn’t fit existing models. Then the development team implements the tracking, followed by QA.
However, with Contentsquare, once the tag is placed, releasing a new page or element automatically collects data without the need for data or development team intervention. The information is instantly available within the tool, eliminating the requirement to anticipate data needs and enabling retroactive analysis from the tag-implementation day, diminishing the stress of future planning.
After implementing Contentsquare, it was time to embark on the process of onboarding our teams to this new product. Our onboarding strategy had two key phases. Initially, Contentsquare guided our Data Analysts to become experts, preparing them to lead internal onboarding. This laid the foundation for independent team onboarding. We had access to rich learning resources, such as online documentation and masterclasses, enabling efficient training. Our organizational structure, with a dedicated Data Analyst per cross-functional team, allowed us to establish Contentsquare experts within each team. These champions were instrumental in disseminating knowledge and seamlessly integrating Contentsquare within our workflows.
5. One year later: Reflecting on Contentsquare and our usage
The standout feature of Contentsquare is its ability to collect data automatically, eliminating the need for detailed tracking plans and code implementation, and without requiring anticipatory tracking plans. We are no longer restricted by predefined tracking plans, which took hours to set up and test, making us more agile in responding to user trends and needs. We are now able to focus more on analysis and less on tracking implementation.
The visual interface provided by Contentsquare has revolutionized our analysis processes. Previously, complex SQL queries were required to understand user journeys and conduct advanced segmentation. With Contentsquare, we can easily visualize user paths, analyze engagement levels, and create custom heatmaps for specific zones on our website. This not only provides us with valuable insights into user behaviors but also serves as an initial guide, pointing us toward areas that might require more in-depth analysis to enhance the user experience and address potential conversion bottlenecks.
Contentsquare allows us to conduct in-depth analyses of page usage and evaluate the impact of new features or product releases. The ability to segment data based on various conditions provides profound insights into user behaviors, leading to better UX optimization and decision-making.
Consequently, since implementing Contentsquare, our data-driven decision-making culture has seen positive growth. Teams across the organization now utilize Contentsquare’s insights and functionalities. Product Managers and Designers have become more self-sufficient in analyzing user behaviors and tracking key performance indicators. This autonomy accelerates development and reduces the workload for Data Analysts, resulting in agile feature tracking, quicker iterations, and faster time-to-market results for new releases.
More specifically, on our day-to-day, we mostly use Contentsquare as follows:
- To quickly get simple usage data, like click rates on call-to-action buttons and scroll rates on new pages.
- For comparisons before/after implementations of new features or making changes in positioning or naming, to assess the impact of changes on user behaviors and experience.
- For session replays for analyzing “suspicious” user behaviors, leading to valuable insights regarding UX improvements. For instance, we observed users attempting to perform drag and drop actions, unaware that the feature was not available. This insight prompted us to consider developing the drag and drop functionality to enhance user experience and minimize confusion.
We’ve adopted a flexible approach to our analytics toolkit, utilizing both Contentsquare and Snowplow depending on the specific use case. For closely monitoring critical features or metrics over time, integrating user behavior data within our database, and sharing data with our customers, manual Snowplow tracking is still invaluable. This allows us to seamlessly incorporate this data into our visualization tool, Looker, and perform complex data analysis. However, for tracking the adoption of new features or understanding user interactions with elements like buttons, Contentsquare has proven to be highly efficient. Its automated data collection streamlines the process, saving us significant time and effort. What’s interesting is that we now have more control over where we want extremely granular data. In cases where we need that level of detail, we opt for manual tracking with Snowplow. However, Contentsquare handles the job seamlessly when we require less granularity, saving us considerable time and effort. This adaptive approach ensures that we have the right tool for the right job, optimizing our data analytics workflow.
6. Important considerations before using Contentsquare
Despite its advantages, as with any tool, there are some crucial challenges to keep in mind if you are considering adopting Contentsquare:
- Dynamic HTML IDs: To work effectively, Contentsquare requires HTML div IDs to be static and unique. Unfortunately, Contentsquare cannot accurately gather data from dynamic IDs, which exist in various parts of our products. This limitation hinders our ability to fully leverage Contentsquare’s capabilities. Addressing this issue required our developers to invest significant time and effort into finding a sustainable, non-manual solution.
- Zoning limitation: After implementing changes on our website, there is a 30-day window during which we need to remember to capture zoning data for the previous site state. If we forget to do so within this timeframe or fail to take a screenshot, we are unable to access zoning data for the older version of the site, even if we have access to a year’s worth of historical data. In fact, it’s not uncommon for us to experience moments of panic, wondering, “Oops, did we remember to capture the zoning data in time?”!
- With the implementation of Contentsquare, a challenge that arose was the determination of data ownership once it became accessible to other departments. As Contentsquare provides easy access and user-friendly features for data analysis across various teams, the question arises: Who should be responsible for conducting analysis using Contentsquare? Should it be the Designer, the Product Manager, or the Data Analyst? Although we recognize that this issue can be perceived as a “rich man’s issue,” it is essential to address potential frustrations and ensure a streamlined and effective data-utilization process. As we continue to reflect on this challenge, we are actively contemplating the necessary organizational adjustments that come with adopting a new tool — who will be responsible for its use, how it will be integrated into our workflows, and how it can best serve our collective objectives.
Challenges to come
Over the past year, Contentsquare has played an important role in enhancing our approach to data-driven decision-making. It has offered teams an intuitive interface, automated tracking, and extensive analysis features. Despite some challenges, the tool’s impact has been significant, offering valuable insights and simplifying data utilization. As we progress, Contentsquare remains an essential ally in our ongoing quest for data-centric improvement. Looking forward, we anticipate a series of new challenges on our data-driven journey. We are committed to deploying the tool across new products and onboarding new users effectively, ensuring that its benefits are felt company-wide. Moreover, as we continue to become more data-centric, defining clear ownership of data and its analysis will be crucial to avoid fragmentation and ensure consistency. We also recognize the importance of not falling into the trap of vanity metrics, prioritizing impactful data and meaningful data analysis that drives actionable insights. These challenges are the next frontier in our data-driven evolution, and we are dedicated to meeting them head-on.
Written by Aurélia Kain, Senior data analyst @ WTTJ
Edited by Anne-Laure Civeyrac
Illustration by WTTJ
