Animesh Koratana of PlayerZero On The 5 Habits That Can Accelerate Product Development Cycles

An Interview With Rachel Kline

Iterate fast and don’t settle — Don’t take any shortcuts in design, prototyping, and creating thought frameworks. The most expensive place to do design is in the code editor.

All new products start with an idea and then continue through the stages of development. What are the 5 habits that can accelerate product development cycles? In this interview series, we are talking to product managers, founders, and thought leaders who can share stories and insights from their experiences about how to accelerate product development cycles. As part of this series, we had the distinct pleasure of interviewing Animesh Koratana.

Animesh Koratana is the founder and CEO of PlayerZero.ai, the AI platform that is helping product teams build and nurture world-class experiences by operationalizing their quality practices. Animesh started PlayerZero while at Stanford University studying under the founders of leading data and AI companies, such as Databricks, Sisu Data and more. Animesh and his team are now based out of Atlanta, where they are excited to be building a next generation AI platform in one of the fastest growing startup ecosystems in the world.

Thank you so much for joining us in this interview series! Before diving in, our readers would love to learn more about you. Can you tell us a story about what brought you to this specific career path?

I think in order to tell the most meaningful story about where it all started for me, I like to reflect back on my time helping my dad with his startup when I was 8 years old. My dad was an entrepreneur and, like any startup, had limited resources; he’d use me as his live-in QA to help him test his product and make sure it would always live up to the “white-glove service” he was obsessed with. As the company grew, this obsession became core to the culture of their enterprise and became a key lever that would differentiate their company against the competition around them. Quality — that was the only and most important difference.

After high school my journey continued at Stanford, where I had the opportunity to work with amazing people and work on building data intensive systems at the Stanford DAWN lab. I got to work with some of the smartest people I’ve ever met to design and build the future data + AI systems. The core idea we’d always think about was, “How will AI systems become useful to real people doing real work?”; I fell in love with the prospect that these systems could bring to the world, and began considering my place in helping make that happen.

After seeing a few fellow lab members spin companies out of the technology we were working on, it only felt right that I would take what I had learned from my time working with my dad and combine it with my education. The problem I wanted to tackle: how to drive and operationalize software quality with AI and data.

Do you have any mentors or experiences that have particularly influenced your approach to product development and user experience?

Yes, definitely. I previously mentioned my work at Stanford, and I’d love to get a bit more specific there. I was lucky enough to work under the tutelage of Matei Zaharia (Databricks) and Peter Bailis (Sisu Data) in the legendary DAWN lab. This is where some of the greatest minds came together to push the bounds on how to create democratic access to data and AI systems. I have continued to stay close to all of my lab-mates as time has passed — they are a constant source of support and inspiration as we all find ways to build better products using this core technology.

But building a company is more than the technology — and I think this is where I’ve learned the most in the last few years. As we were putting together PlayerZero I had the unique opportunity to surround myself with operators who build some of the world’s most iconic companies — builders of products that I was inspired by and we use every day. Just to name a few that have been incredibly influential for me, mentors and investors such as ChenLi Wang, Guillermo Rauch, Dylan Field, and Oliver Jay have helped shape our company and our vision for how to reinvent our space.

To create a successful company, I know how important it is to find a balance between engineering and go-to-market in every decision we make as a team. Having these guys in my corner as constant sounding boards empowers me to keep moving fast and stay focused on delivering successful outcomes for our team, and more importantly, our customers.

It has been said that our mistakes can sometimes be our greatest teachers. Can you share a story about the funniest mistake you made when you were first starting? Can you tell us what lesson you learned from that?

That’s a funny question — I feel like I make mistakes every day. Perhaps what makes them funny is hindsight — seeing how the dots connect looking backwards is always more obvious than thinking about how you felt in the moment.

Perhaps one of the biggest mistakes I made early on was underestimating the very problem we were working on. Product quality is a big deal — it’s the reason we buy an iPhone over an Android (hot take?), buy a Bounty vs the store brand, or go to Target over Walmart. But if you talk to enough engineers and ask them what the biggest blocker is between their product today and a high quality product, the universal answer will likely be “we need to test more”.

Turns out, that’s only part of the problem. Only after building a QA product did we realize, “oh crap, turns out this doesn’t solve the problem the way we’d expect it to.” We’d painfully stumbled onto a bigger opportunity; we realized that product quality has a goal alignment problem. Product quality isn’t a QA problem, it’s an exercise in finding ways to systematically make sure everyone in a company is aligned with creating a high quality product & experience for their customers. That’s a hard problem — and that’s what we want to solve.

What do you feel has been your ‘career-defining’ moment? We’d love to hear the lead-up, what happened, and the impact it had on your life.

This one is pretty clear for me — dropping out of Stanford to pursue entrepreneurship full-time was the biggest leap I have taken to date.

To resurface my time at the DAWN lab, this is really where it all started from an inspiration perspective, but it definitely received a huge boost from the pandemic. When I was sent home to Georgia to take online classes, I quickly realized that the lab is where I did the most learning. A fun fact is that my professors were really a main driver in me taking the leap and stepping away from school; they knew the “point” of Stanford for me was lost if we weren’t able to be in person, building and socializing.

Outside of the structured learning element of my decision, the company, which I had officially been running as a student for a year at that point, was starting to get commercial traction. We had landed a few large customers and it was clear that there was value there, so it was time that I committed my full, undivided attention to delivering great experiences for those customers.

I’m extremely fortunate to have had the opportunity to walk away from Stanford with an opportunity to do what I love everyday, with people that I respect and for customers that inspire me.

Can you tell us a story about the hard times that you faced when you first started your journey? Did you ever consider giving up? Where did you get the drive to continue even though things were so hard?

There’s a concept called the “Idea-Maze” that is really relevant for this question. I first heard it from Marc Andreessen, but I think it originated with a partner of his, Balaji Srinivasan. Anyways, the idea is that founders must go through an extensive journey of trial and error before ever getting to a place where they can see success. In some cases, this can take up to 5–10 years! This is something we have embraced as a team over the past few years as we continue to look for product-market-fit.

One early checkpoint in our idea-maze journey was our realization that “quality” does not actually mean what most people think it means, from an operational perspective. When we first started out, we were a testing company that used AI to test applications in ways that mimic real user behavior. It was a fantastic product that did exactly what people told us would move the needle for how their team assured quality. But it fell flat, and this reality was extremely hard to grapple with. As we chatted about earlier, I worked in QA for my dad’s company, so my pretenses were being challenged as our team struggled to find fit in the market.

The insight we came to is that quality is not a role or a department, it’s a way of operating. The best companies in the world don’t just “assure” quality, it lives in every decision they make and is built into every process they have. I like to use the metaphor of building a house. You don’t build the entire house all at once and then hold a postmortem to test the plumbing, electrical, and supports! You do it as you go, one key element at a time. Like a house, building a product means you’re constantly building on top of something, so you’d better be sure that what you’re building on is safe and performs as expected.

Giving up has never really been an option. We have a talented and committed team that has been with me every step of the way, and for that I am extremely grateful. We’ve had our share of downs, but the ups are continuing to grow higher and higher.

How do you stay on top of market trends and developments in the product management space?

Product managers are what many organizations call founders of their own product, within a larger team. It just so happens that these are the types of people that I tend to jam with the most when it comes to what’s new in the world of tech. I have a lot of friends that have matriculated into Product roles at companies of all sizes. We all frequently get together and talk about what’s happening in the industry, and the frequency of these meetings is increasing with the ever-changing landscape led by AI.

Outside of my direct network, we do a ton of discovery research as a team, speaking with dozens of Product Leaders every week about the state of tech and its implications on Product roles, and how organizations are coping with change. I’m lucky to have built an awesome network that is always open for conversation or brainstorming, which keeps us on our toes and pushes us to deliver great experiences.

From a technical perspective, I have a background in data intensive ML systems which helps me understand the boundaries of AI and data and how they might change the way we work. I read academic research daily on which trends are emerging and which technologies are going to drive the next wave of innovation.

What role does cross-functional collaboration play in accelerating product development cycles, and how do you foster effective collaboration across different teams and departments?

Cross functional collaboration is the neurons firing in the brain that is the organization — it’s the reason you do anything. How am I going to lift my arm if a specific neuron doesn’t fire telling my body to do so? Or maybe it happens too late and I’m unable to get my hand up to protect my face from the incoming football careening across the field? The same goes for an organization. If the support team doesn’t deliver insights to engineering in time, then maybe your most important user runs into an issue again, gets frustrated, and churns. That’s a real black-eye moment right there.

A few ways you can foster better collaboration:

• Alignment towards business outcomes: In order to foster better collaboration, everyone on your team should be aligned around the same north star: what are the key business outcomes that we need to deliver, and how is the work the team is doing supporting that goal? The entire team, from sales to engineering should have a clear understanding of what you’re doing and why you’re doing it.
• Democratic access to data: Cold hard data is always the best way to talk through a problem or hypothesis. Unfortunately, there are only a few departments in an organization that have access to data, and oftentimes, it’s completely unique to only the work they are doing. Every team has an impact on the end state of the product and customer experience, and this is data that should be tracked and observed by the entire organization. Only once you’re able to see the entire picture can you make smart decisions and collaborate successfully.
• Meaningful ownership: Make sure people can feel ownership over the impact they’re making! Too often teams that are further from the end user (i.e. engineering) get left delivering on inputs to the product experience, without feeling the output. This creates a natural emotional detachment, and can lead to a less engaged team. Everyone on the team should understand success in the same language and be able to celebrate when they deliver wins for their customers.

Thank you for all of that. Here is the main question of our interview. Based on your experience, what are your “The 5 Habits That Can Accelerate Product Development Cycles”? If you can, please share a story or an example for each.

1. Iterate fast and don’t settle — Don’t take any shortcuts in design, prototyping, and creating thought frameworks. The most expensive place to do design is in the code editor.

We have consistent feedback meetings setup with a wide range of people (existing users, cold outreach, advisors, etc) getting constant signals on what’s working, what’s not, and everything in between.

2. Operationalize quality — Find a way to apply the learnings of the past to the quality output of this iteration.

With every release, we track the impact of our work on our user’s behavior. This gives us a tight feedback loop to allow our users to contribute to our quality initiatives everytime they use the product (without them knowing it).

3. Use a machete — Once the initial design is in place, be aggressive in asking why things are the way they are, and don’t be afraid to scrap things and try again. Create a clear scope and boundary for what needs to be done to create the value needed.

We’ve torn down and rebuilt a lot of our product many times. We’ve persevered through over two years worth of pivots… it’s hard, but the team understands what it takes to catch lightning in a bottle.

4. Plan & communicate early (mental replays) — Give people time to think about, digest, and formalize the execution strategy before they’ve broken ground — ideally while they’re working on the previous sprint. Giving a lead time to think before opening code makes a big difference in the quality of code and how much it’s thought through.

At the end of every week, we do a big picture review and projection, we sync on the successes and failures of the week, and challenge each other to start off the next week with great momentum and focus. We like to tie in larger concepts that get us inspired to start off the next sprint with a fresh take.

5. Create a checklist — Create a running checklist of pre-release verifications. These can be tests, emails to send, upgrade scripts, or reminders. Do this to have a central place where the entire team can reference what the key steps are before a successful release. Don’t rely on the team’s memory, it’s always dampened by the craziness of a release.

To kick off the week, our team has a list of “to-do’s,” or inputs that align with a goal/output to achieve by the end of the week. Each day, we briefly sync on these objectives, track our success, and help each other progress.

What are some of the common pitfalls that you see product teams fall into when trying to accelerate their development cycles, and how can these be avoided?

People tend to work hard, not smart. It’s so easy to tie value to time spent working (or total input), but the reality is, measurable output is all that matters. Are your customers getting the experience that they’re paying for, and are you continuing to find unique ways to delight them and keep them coming back?

For most teams, acceleration is a resource allocation problem instead of a scale problem. It’s important to have a team of aligned people, who understand the problem, where you need to go, and how you’re going to get there. Once you’re able to map out these goals, you can accurately map what’s possible, and what’s not. Trying to create scale without efficient resources is a bad recipe.

So you must ask yourself, before jumping in, are you understanding the problem from all angles? Have you thought about and mastered the topic and are you confident in your positioning? Have you reflected on the successes and failures of your teams before to understand pitfalls and opportunities?

In the case that you’re not positioned for success, you must be resourceful in finding ways to fill the gaps. But again, it all starts with a complete understanding of the situation in front of you, the resources and the goals, and what it’s going to take to deliver.

Can you share an example of a time when you had to make a tough tradeoff between speed and quality during a product development cycle, and what was the outcome of that decision?

Early on, startups always prioritize speed, which makes total sense. You are trying to prove, as quickly as possible, that the need you are solving is one that’s worth paying for. The idea of “move fast and break things” is exciting for entrepreneurs and builders alike, but it comes with its own pitfalls.

For our early iterations, we prioritized speed. We had an insight and we were heads down building a solution that we thought would deliver real value. But we took some shortcuts, and that hurt us.

Bad product quality can actually limit the amount of signal and feedback you get from customers. (Yes, I realize it’s ironic. A quality platform with poor product quality…) By delivering an early product iteration that was not up to snuff on quality, it created biased or misguided feedback from our users and in our discovery research. This made it extremely difficult to untangle the feedback that was net-new and novel from the stuff we already knew.

One key contributor to keeping this from happening since has been our heavy commitment to focus and refinement. Limiting the amount of variables we are testing, while also giving more thought to potential scenarios before launching, has given us the ability to eliminate speed-induced feedback spaghetti.

How important is a data-driven approach to product development, and can you share a story where data significantly influenced your decision-making process?

Extremely important. There’s no other way to put it.

If you’re not using data to inform your decisions, you will sooner or later get beaten by an organization that is taking data seriously. There are so many colorful stories and insights that live in data that are seemingly impossible to find at scale. If you’ve grown without it and are seeing success, then congratulations, but my money is on your luck running out one day; your process will start to get bogged down, your product will start to bloat, and you’ll lose a pulse on the product decisions that move the needle.

For us, it’s hard to pinpoint only one story, because we’ve always been so data driven (even when we barely had any). Every pivot we have made has been a result of data, and we’be made a few ;) One instance I’ll highlight is our decision to move on from Testgram (as a product and a brand) and rebrand with a new product and direction.

To pull back for a quick second to explain exactly how we approach data and decision making, we’ve always required a holistic approach — leveraging both quantitative and qualitative data from inputs across the organization. Engineering devtools, customer feedback, user behavior changes, all of these are crucial data points when we want to better understand the performance of our product and the strength of our decisions. Having made a few successful directional product changes using this mindset, and having seen continuous improvement towards a solution that delivers real value, we actually decided to focus on delivering this same opportunity to our users in our product. By leveraging some of the key AI principles that inspired my lab-mates and I in the DAWN lab, our team at PlayerZero was able to democratize data for teams of all sizes.

Can you share an instance where user feedback led to a significant pivot in your product development strategy?

I’m going to double down on the Testgram story here. What’s important to zoom in on here is that at PlayerZero, user feedback is, at its most basic, data. It’s data that has a tone, purpose, and intent. So we try to make sure we infuse this in every decision we make, most notably the switch from Testgram to PlayerZero.

We are very blessed that very prominent leaders read this column. Is there a person in the world or in the US with whom you would love to have a private breakfast or lunch, and why? He or she might just see this if we tag them :-)

Andrew Huberman is definitely top of my list. We actually met briefly at a Stanford event, but I’ve always been a huge fan of his work. What really intrigues me about his work is his ability to tie underlying scientific principles to practical use cases, and then help people take action on those insights

Thank you so much for this. This was very inspirational, and we wish you only continued success!