Snapdeal’s Head of Architecture and Ex-CTO of Zomato, Prashant Parashar, spoke at the fourth edition of CTOTalk by OrangeScape on 22nd September 2018.
As someone who has spent the last 16 years deeply entrenched in the Indian tech landscape, and having experienced the ecommerce boom of the last decade, Prashant is a highly knowledgeable industry thought leader. At this month’s CTOTalk, he broke down the process of experimenting when rolling out product features. He also explained Snapdeal’s approach, with anecdotes and examples.
In his own words…
The need to run experiments
With a new business model, you need to win customers–that’s why you experiment. We initially tried experiments like huge discounts. Then, our customer base grew exponentially. The economy was good too, so we experimented. We analyzed customer behaviour, and tweaked the way we did things.
Prashant addresses a rapt audience at OrangeScape’s CTOTalk
Last year, we turned our eyes towards profitability. Since July this year, we’ve become profitable. A scalable experimentation framework is the need of the hour. We have to optimize every last thing we can. Experimentation becomes unavoidable after a certain stage.
Keeping certain features beyond a point may be detrimental to your organization.
Product Management
Feature creep can pose a problem. Instead of loading (and bloating) the feature list, find the features that work. Experimentation is key here.
How to experiment when rolling out features
First achieve a critical mass. The methods vary depending on which stage you’re at. Look at it clinically–CxOs should embrace pragmatism.
Our old approach to launching a product feature was to get every component in order–design, marketing tactics, and so on. Then we’d create a lot of hype around the new feature. The issue, though, is that we often get personally attached to it. You might fight to keep it successful in spite of it having some fundamental flaw. The focus is on the launch of the feature.
Culturally, move away from the “launch a feature every so often to stay relevant” mentality. Instead, wait till you hit a feature you can justify. Roll it out gradually. People won’t know when you made a major change, but your product is still evolving in the right direction. Be focused on the outcome of a new feature instead of the launch of said feature.
This will help you build your tech, product and marketing teams into a unit that works cohesively to slowly roll out features by completely understanding customers.
Do not assume you know your customers
Product management teams need to have a clear vision when they come up with a rollout plan.
There should be no know-it-alls when it comes to experimentation. “I know something, but I’ll have to experiment for the best results” is the best attitude to have here. Understanding user actions is the aim.
The humbling fact all engineers, programmers and even product managers and CxOs need to realize is that you don’t ever fully know your customers. There’s no definitive “this will work”. Experimentation and validation is the way to go. What’s innovative and exciting from your perspective may not be what customers are excited about. Think pragmatically.
The ideal product feature cycle: controlled rollout → measure → optimize → repeat
You’ll need the right tech for:
- Rollout
- Measurement
- Changes — rapid experimentation
Conclude — keep the tenure of experiments in check. You need to have checks in place for what qualifies as a success, and what qualifies as a failure.
Tech should be:
- Easy to set up
- Easy to measure
- Controlled
- Configurable
- Avoid latency to end-user
- Scalable
Snapdeal has, for instance, about 300 microservices, over three billion messages every day, low latency, and a single-responsibility design. Every feature is built on solid thinking and planning.
Snapdeal’s experimentation engine
At Snapdeal, we created our own data platform to process all the data we needed to. All user events and transactional events are handled here. This could track user history, creating opportunities for fruitful experiments regarding user behavior.
We split experiments internally into ‘inbound’ and ‘outbound’ experiments. Inbound experiments concern those features we show users who visit our site or use the app. We choose what to show users–entire pages like the home page and the listings page are fully configurable. Outbound experiments are those emails, messages and other things we send to people. This is basically notifications and corresponding content. We experiment with target demographics and the best time for communication.
Prashant answers his audience at the post-talk Q&A session
Prashant then gave CTOTalk participants more details about Snapdeal’s experimentation engine. After that, speaking of the road ahead, he spoke of AI-driven sample sets, conclusion and decision making regarding experiments, and configuration. He also spoke of how experimentation, which had been used for optimization, would be used to fuel growth going forward.
During the 30-minute Q&A session, the audience, a mix of tech founders, architects, and engineers threw out some interesting questions at Prashant. A few of them:
Q: Are sellers on Snapdeal aware of these experiments?
A: All the experiments increase order value, margins, shippability, or deliverability. Sellers are more concerned about their volumes. Ultimately, the results benefit the sellers as well.
Q: How is your team structured?
A: We have 4 verticals: buyer-facing platform, mobile API, mobile apps, and website and ordering. Buyer journeys are covered by the first vertical. The supply chain side handles logistics. Seller side handles inventory, products, etc. The tech platform vertical handles the experiment engine, search, data platform, et al.
Over 90 techies participated in the CTOTalk. A new addition to this month’s CTOTalk was an exclusive “Lunch with the CTO” session, where select startups / tech founders who sign up in advance, get to spend some quality time with the Guest Speaker, over lunch.
