Data Tracking Development: Cross-functional Involvement and Its Instrumentation
Author : Fatia Kusuma Dewi
One of the most important parts of building data literacy and data-driven culture within an organization is making sure we are collecting all necessary and sufficient data. Although the infrastructure for saving enormous amounts of data points is getting cheaper and cheaper (thanks to the cloud technology!), it does not necessarily mean we should collect all data. Huge amounts of saved data does not necessarily positively correlate with the huge information that will be extracted.
On the other hand, we are aware that we do not want to miss any important data points from users. As such, we may be thoughtful in deciding and designing which data we should collect and store. Let us first recall two systems that play a role for data collection in a digital product application: (i) the back-end (BE) system that holds essential information and; (ii) the front-end (FE) tracking system that collects user feedback data. The latter is well-known for developing our understanding of the user’s experience in an application.
In this article, we present a framework for the data tracking process on a FE system, which also includes specific roles and responsibilities of cross-functional team members for building the accountability and ownership of the data tracking system. We identify four main functions that are involved throughout this process, namely Product Manager (PM), Data Analyst (DA), Software Engineer (SE), and Software Quality Analyst (SQA). However, it is important to note that different companies possibly have different functional structures and preferences on who and how to be involved. We are lucky to have team members who were alumni from various tech industries and gave tremendous help during the process of developing as well as implementing this framework. As of our limited understanding, this topic has not been widely discussed, as opposed to the comparison of tracking tools, which ranges from open source, paid to in-house tracking systems (e.g. comparing GA, Amplitude, and Mixpanel, comparing 5 analytics tools, comparing Plausible and Fathom )
The working flow is divided into three-fold, namely (i) requirement gathering and detailing, (ii) design assessment, (iii) development and assessment of the tracking design. We refer to those three elements as the tracking development step. We elaborate the contributions of cross-functional team members, in the next following sections.
Step #1: requirement gathering and detailing
Summary: The tracking design starts with the identification of users journey by a Product Manager (PM) when a new product or feature will be released. The PM holds main ownership on the data tracker requirement of their product, which includes initiating a tracking requirement document, overseeing the product performance metrics and translating the metrics to more detail potential events. Next, the Data team plays roles for reviewing, detailing, and giving further recommendations on the requirements. We share here an instrument for tracking requirements template (WIP), which has been extensively discussed and reviewed by cross-functional teams.
The main involved-functions and their ownership
- Product Manager (PM)
Role: A PM holds the main data-tracker ownership, because of these two following reasons. First, a PM well-understands the product’s vision as well as the product performance measurement. A high-quality product can be developed through constant measured-improvement, in which the existence of data tracking is one of the key prerequisites for a team to do so. Second, implementing a tracker requires additional effort in the development (or engineering) team, which may reduce their capacity in building the product/feature itself. Unfortunately, this effort is vulnerable to being pushed-away, due to focusing on more “tangible” products for the end-users, which results in the data tracker being later or, even worse, never being implemented. As such, it is important for a PM to have high ownership of the data tracking on their product.
Responsibility. The data tracking requirement starts from when the product team is expected to determine the user flow journey. The ideal instrument is the high fidelity design mockup from the Design Team, an example can be seen on our data tracking requirement template here . When such an instrument is not available, any documentation on user journey and expected interaction between users to the app will also help at this stage. Next, a PM identifies potential insights as well as product performance metrics, which it is then translated into tracking requirements.
2. Data Analyst (DA)
Role: A Data Analyst plays a role as a consultant from the previously initiated tracking requirements by the PM. Tracker design is suggested to be mastered by the Data Team because it will be exhaustively consumed by the DA later. The role includes on how to manage the information level on the tracker attributes, how to define the tracker nomenclature in the most structured form, and on how to (or not to) group the data-trackers.
Responsibility. A DA gives reviews, feedback and gives more technical detail on the data-tracking requirements (that are previously initiated by the PM). The DA is also responsible to identify any missing points from initial requirements, as well as to give suggestions on the alternative tracking design, as such it satisfies the product’s goals.
Tracker requirement instrument
The instrument is a tool or a form that serve these following goals: (i) enabling PMs to write down information about the goal on each prospective-tracker; (ii) accommodating PMs to write the requirement in a high-level language — i.e. non-technical engineering language; and (iii) elaborating the specific page, in which the tracker is implemented (usually in the form of page screenshots). The instrument necessarily accommodates details of a page of web or apps (for example in a screenshot form or link to high fidelity design, as such the DA and other relevant functions can clearly point to where the expected tracker to be implemented is. We gladly share our tracker requirement instrument here, as an iterative tracker requirement template from a previously published template.
Step #2: assessment of tracking-Design
Summary. Having the PM and the DA aligned, the tracker requirement is then assessed for its feasibility and is passed to the development team (engineering team). Shall there be any additional requirements and/or changes on the requirement after the assessment, the step may return to step #1
The main involved-functions and their ownership
3. Back-end (BE) and front-end (FE) engineers
Role: Engineers hold accountability for the technical decision on the data tracking requirements.
Responsibility. Engineers give assessment and decision points on whether the data tracking can be implemented, as well as give recommendations on the alternatives implementation of the data-tracking (if any).
4. Software Quality Assessment (SQA) engineers
Role: As it is reflected in the function’s name, a SQA is involved in the assessment phase (in collaboration with BE and FE engineers) and plans the testing for later.
Responsibility. An SQA may give input to the implementation of the tracking requirements and may start to make a testing plan for the later.
Please note that although the development team are main actors at this stage, all the process may be facilitated by a PM (e.g. at the grooming session with engineers). If there is any changes in the requirements, the PM and the DA may be informed and re-do the step #1.
Step #3: development and evaluation of the tracking design
Summary. Having the requirements set and its feasibility assessed, the tracker design is then implemented. Next, the evaluation is conducted to ensure its functionality and its compatibility with the predefined-requirements.
The main involved-functions and its ownership
- Back-end (BE) and front-end (FE) engineers
Role : Engineers (BE or FE) is the role for implementing the tracker corresponding to agreed tracker requirements.
Responsibility. The circulated tracker requirements then become a guideline when engineers develop the trackers in the apps. Engineers also hold up the responsibility to fill up some value on tracker attributes corresponding with any available variable on the apps due to make the value on attribute become consistent. For example there exists an attribute for “section”, whether the value will be “product carousel” or “product_carrousel” or “carousel” is expected to be decided by the engineer to maintain the consistency across the apps.
2. Software Quality Assessment (SQA) engineers
Role : SQA will be responsible for ensuring that the implemented tracker is the same as tracker requirements in any available test case scenario
Responsibility. Tracker must be seen as a feature where a set of behavior is expected and the failure could happen hence it must be tested to minimize failure. This is a crucial part that is often overlooked by many because the tracker does not have any direct impact on our users, so the attitude of seeing the tracker as a feature is important. SQA may perform some test case scenarios on the implemented tracker for ensuring it follows tracker requirements and all expected behavior, thus the data collected from the tracker can be trusted and could give confident insight for the product and business.
3. Product Manager (PM)
Role : As an owner of the product and data tracker, a PM decides the readiness of their product and tracker. If the implementation of the data tracker requirement is not ready yet, the PM may postpone the product launching to prevent failure measuring product performance.
Responsibility. Tracker development is also a part of overall product development. There might be a case when the release should be postponed either because of delayed feature development or delayed tracker development. The PM should make a call whether they want to defer the product release if the tracker is not ready yet, as it will affect product success measurement.
Tracker requirement instrument
At the same instrument, the engineers are expected to fill some value of each attribute, so the data team will know what to expect when querying the tracker once it is implemented. This is to avoid guessing games when querying the tracker after the implementation. The Quality assurance team also should be able to write when they perform the test and when is the time of roll-out, thus all teams that need the tracker could rest assured if they know the tracker has already been tested.
Acknowledgement. At our organization, this framework has been formulated and developed by a dedicated task force who initiated and built this data tracking from scratch, as well as put lots of effort for alignment across functions and for the socialization process (Markus Kevin, Nada Haroen, Muhammad Nassirudin, Abdul Somat) and the Data Analytics Team (Hana Rotinsulu, Krisna Aditya, Bagoes Rahmat) for enormous help and for giving constructive feedback to iterate the documentation.
