Pama Team: Hypothesis
Part 1 of 3. Hypothesis generation and testing
Many software developers run their own projects in their spare time. Most of them develop small highly specialized frameworks or libraries, but there are examples of implementing large projects, such as Nginx, for example.
Many of those who are reading this article now, I think, at least once asked themselves the question: if I have 2–3–4 hours of free time every day, what kind of project can I do on a long distance?
I want to tell you about one of my completed projects, for which I was “One Person Team” and to which I have dedicated 3 years — Pama Team platform.
Working in large IT companies that used agile methodology, I constantly faced the fact that for ceremonies for different frameworks we created the whole toolkit ourselves: somewhere we used miro, somewhere we used separate tools, for example, online poker. When I moved to another team, the toolkit changed completely, which depended very much on the maturity of the team.
Unfortunately, I never managed to gather a full set of specialized tools: there was a wide range of tools for task estimation (poker planning), but I didn’t find a quality retro ceremony tool.
Other tools required a common context — so that information, or data, such as tasks, could be shared between them. If there is no common context, then usability can be forgotten.
These factors, combined with feedback from my teams, led me to the idea that if there was a ready-made set of Scrum tools that were linked by a common context when the team was formed, it would greatly facilitate new teams to get up and running faster and reach a consistent velocity.
My observations also suggested that these tools would be good to have in the form of a mobile app, not only because their CSI is on average 50% higher than desktop ones, but also because it is simply more convenient: they can be used anywhere and anytime.
To restate the idea, my main hypothesis is:
hyp 1: The Scrum team prioritizes the Pama Team’s pre-built set of context-aware Scrum tools implemented in the mobile app, rather than building their own set. As a consequence, new teams launch faster
PamaSync
To test my hypothesis hyp 1, I needed to create new custom mechanics for a mobile app that would enable online team ceremonies, because I wanted to remove the indispensability of voice conferencing tools like zoom from the team collaboration equation.
Through brainstorming and prototyping in Sketch, conducting lengthy UX studies on friends (sorry, guys ✌️), I derived the ground rules for such mechanics:
1. User actions on the mobile application screens are synchronized with other users as if they were the ones who performed these actions
This is done so that the users’ behavioral pattern is not destroyed: the mechanics are new, but the expected behavior is familiar to the user.
For example, if a user changes a text field, the same field will also change for other users
2. Changes are made synchronously for all users without delay, that is they are made in real time.
3. The interaction area is not limited to only one visible area and one screen.
If one of the users opens a new screen, for example, to create a subtask, the other users will see this screen as well.
4. All users are equal in broadcasting their actions
I added this rule after A/B testing on real users. It turned out that the team itself comes to a point of equilibrium — a facilitator naturally appears when and where needed. Although at first glance it doesn’t seem very logical.
Proof of Concept
To keep the focus on the core product idea, I use JobsToBeDone: I distinguish Core Functional Job and Job Stories (table 1).
The Core Functional Job in Pama Team is a set of mobile tools for scrum ceremonies and team events, which was my initial hypothesis.
Job Stories is a set of additional supporting functionality, or functionality that increases CSI and NPS.
At the initial stage of product development, I faced with the fact that to get objective CSI/NPS data on PamaSync, I had to implement a large amount of functionality, and quite possibly, when I would have implemented it, it would have been a) not relevant b) the PamaSync mechanics underlying the product would not be liked by users and they would not use it.
Since we want to save money at the start, it is important to get confirmation of the main hypothesis (hyp.1) in as short a time as possible and with as little investment as possible.
In such cases, I often use $100 Prioritization, the essence of which is very close to Pareto’s law, but allows you to iteratively build an end-to-end prioritization of tasks at the zero stage.
Without diving into the details, I have come to the following conclusion:
1. We need to implement a limited set of ceremonies (Proof-of-Concept) built on PamaSync that can work without a common data context and platform function implementation
2. Since Proof-of-Concept tools are based on PamaSync, more complex Proof-of-Concept+ will have CSI and NPS values not less than Proof-of-Concept (3 parallel lines principle).
On the one hand, this allows us to create conditions for the feedback to be objective so that we can draw the right conclusions. On the other hand, since there will be limited functionality to test the hypothesis, we will have the opportunity to get feedback from users as quickly as possible without having to wait for some of the platform’s functions to be implemented.
Another source of savings could be the reduction of iOS and Android app development costs.
To reduce development time and unify software components, I chose the Xamarin Forms mobile app development platform, it allows you to develop one unified app for iOS and Android at the same time.
This approach allowed me to realize Proof of Concept in 4 months, half of which I spent on creating the application architecture, UI, UX and diving into the nuances of mobile development, as I had previously developed only backend and desktop applications.
Proof of Concept results and getting feedback
The implemented applications received a version index of 0.5 (fig 1 and fig 2): iOS was distributed using TestFlight, and the Android app was not presented in the Stores — it was distributed by apk file.
During the Proof of Concept lifecycle, which amounted to 2 months, 3 digital product development teams of 28 people with standard competencies took part: product owner, scrum master, analyst, developer (front + back), QA.
Of course, the teams used Jira in their work and using my tool with it was not seamless — it was noticeable that the teams felt discomfort due to lack of integration.
However, through the use of my tool in the day-to-day work of the teams, I received detailed feedback that matched my expectations, estimates, and projections.
The feedback received confirmed the usability of the developed PamaSync mechanics, while the incomplete set of tools and lack of a unified context did not allow to reveal all its possibilities.
Reflection and restarting the idea
One of the main existential challenges of early stage development projects is to move towards a structured and conscious approach to product development, because while in the initial stages, they test their hypotheses, usually without any methodology.
The modern concept of digital product development, which involves targeting a customer problem and evaluating its effectiveness as a result of small increments of functionality, is product design .
Using this concept has allowed me to set up a pipeline not only for generating ideas aimed at solving specific user problems, but also for implementing them and then evaluating their effectiveness.
Special care should be taken with the parallel realization of ideas going to this pipeline, because they have: 1) strong mutual influence, they can correct each other directly and 2) each idea after implementation and evaluation can change, thus influencing other hypotheses/ideas (fig. 3).
As for me, I mitigated this risk by building a prioritization model and working only in sequential mode.
The transition from Proof-of-Concept to Proof-of-Concept+ was a kind of watershed for the transition to an iterative and measured product development model. Another notable event also happened at this point: I started using my platform for task tracking myself.
New UI and UX design principles
The work done on Proof of Concept, as well as the valuable feedback received, showed the need to revise the UI/UX to improve user scenarios. It was evident that some of the scenarios were not detailed enough and some were highly complex.
When designing user UX, it’s important to prioritize intuitiveness — make the system 1 in Kahneman’s classification.
Realizing that you are on the right track can be done in several ways, for example, a UX study should show you that user scenarios are walked through intuitively “in the same breath” without reading manuals. During the prototyping stages of a new UI/UX version, I started using a separate KPI for the number of “what’s next/how does it work” calls — the higher it is, the worse the service is and the system is closer to type 2, where more unnecessary conscious actions are required for the system to work. This helped me greatly simplify the UX of some sections (fig. 4).
Proof of Concept+
The mission of Proof of Concept+ is to create the final product image by adding features to the task tracker platform as well as extending Core Functional Job for the final confirmation of the hyp 1 main hypothesis.
Table 3 shows the list of functionalities by version and Core Functional Job/Job Stories.
For clarity, I have shown the user functionality in terms of version and step of the scenario, organized in the form of User Story Mapping (fig. 5).
The user scenario here is the end-to-end workflow of a Scrum team member: from creating a new team space on the platform, to completing a sprint and forming a product iteration. Of course, a lot of attention is paid to Scrum ceremonies, and the focus of the solution itself, as I said earlier, is on the interaction of team members.
ChDeveloping Proof-of-Concept+ took me almost a year: I completely redesigned the architecture of the mobile and backend applications, it was essentially a new solution.
Looking back, I realize that using product design and choosing the right moment in time allowed me to solve technical problems in an evolutionary way: the backend and mobile app architecture that was created for Proof-of-Concept+ is used in the latest version with minor cosmetic changes.
I will note that the most important idea acquisition for me was the use of the Signal R library, which allowed me to build real-time user interactions and it breathed new life into the PamaSync mechanics.
Once all applications were tested and published, I started focus group studies to build CJM (fig. 6)
User portrait: a member of a small scrum development team who cares about the convenience of on-the-go ceremonies
The results “wrapped” in CJM not only confirmed my hyp1 hypothesis, but also showed that the assumptions made at the beginning of this project were correct. The intermediate Proof-of-Concept stage allowed me to do some work on my mistakes, and also introduced additional safeguards into my scheme if users did not rate the solution very highly at the initial stage.
In the next article, I’ll talk about the minimum viable product and the task prioritization methods I’ve used.