Startup Guide: Empowering product and marketing teams to fail-fast on lean experiments

In growing startup, one of the fact is that there are plethora of hypothesis to test and validate, however bandwidth and costing always exists as constraint for companies striving for profitability. Further you can’t stop experimenting or avoid failures, so you can just fail fast if you want to succeed sooner.

Thanks to Optimizely and VWO for providing us A/B test framework, though that’s just necessary but not sufficient tools. Our growth hacking team charts out all variations in beginning as required, but requirement of new variation comes at anytime based on data and need to be implemented in no time to move forward quickly and achieve success of failure in least time.

Giving one dedicated full stack tech developer bandwidth each to marketing as well as product growth hacking team, just for experiments, has never yielded ROI. Getting it done through standard development sprints too is not an option as it takes 1–2 weeks to deliver on variation of experiment and above it, it gets hard to accommodate so many runtime requests on regular basis. We have faced this problem first hand as well as I’ve also checked the same across all startups in my network.

Back-end team is always occupied in company’s critical projects and hence we needed to figure out way to have experiments created and scaled anytime, without back-end tech team as well as independent of sprints. Goal was set to increase our experiments count, decrease our failure time & increase our success rate as well.

In TravelTriangle, we have achieved this goal through right set of collaboration and approaches that were set for our growth hacking teams. We decided to use only front-end team bandwidth for such cases, along side growth hacking team for any experiment.

Few of the mandates created were that they have to find solutions to create variation by just using front-end skills (HTML + CSS + JS) or economical 3rd party plug-ins. If all options gets exhausted and few back-end changes are needed, it gets done by creating one time framework in a way to enable growth team to create any kind of variation(s) later on by just using frontend code changes.

Our rockstar growth hacking and front-end team eventually figured out ways to solve most of the solutions using seamless front code injected in VWO and Google Tag Manager(GTM). They also started using GTM to inject 3rd party tools directly without needing any deployment or other team’s bandwidth.

We did hit blockers while applying this approach onto pages built using AngularJS but as expected, team solved it too again by just using fronted code.

Growth hacking team becomes their own QA. They limit targets of experiments in such a way that their test cases becomes less and faster to get done with. Over it, they launch experiment on very low scale and observe it for few days so as to eliminate chances of any other bugs being present. We eventually removed dependency on QA bandwidth as well.

Our back-end development team too stepped up here and contributed equally by implementing seamless framework in little time, few of such examples mentioned below:

• Upload static HTML pages on to production and make it live instantly, using Amazon S3 — needed to test totally different looking pages and variations which needed complete HTML & CSS code revamp.
• Enabling missing APIs to integrate at front-end & interact directly with back-end code flow.
• Request tagging framework, leveraging elasticsearch and logstash, to tag any request onto server — needed by team to see whole lifecycle of customers coming through a particular variations, to compare accordingly.

Post cracking this up, team scaled up quickly but soon hit yet another blocker as our experiments were evolved to next level. They wanted to store additional or totally different inputs taken from our users, over & above what was supported in our database schema or back-end code. This is where we discovered combined power of Google Sheets and Google Scripts.

Tech team built a lightweight JS plugin which can store such inputs onto Google Sheets from variation pages and wrote another plugin using Google Scripts which can sync relevant data from that sheet onto back-end server to process them in real time. New inputs were moved onto excel reports by growth hacking team to analyze and evaluate new data to check if they are useful or just another failed hypothesis.

Next few frameworks in sight:

• There are even more complex experiments coming up now, which need modified business logic and hence different API on server side. Back-end team is working on uploading code snippet and loading it dynamically through reflection. This way, we can enable new API for growth hacking team without needing to change any server side code and hence need not following standard development and QA cycle.
• Getting all analytics at one place to have predefined data reports for experiments, generated even before starting experiments and used later to just monitor periodically without having to look or dig here and there across multiple places.

On final note & presenting bit of objectified analysis, above effort and approaches have increased our experiments number by 3X, taking our success rate up to 2X with time reduced to almost half to conclude any experiment. Team now remains all time motivated and fearless to try new ideas and experiments as now the power lies in them.

I’d love to hear about any other ways that you might have seen or getting followed up in any startup so as to fail-fast on lean experiments without getting blocked on tech team bandwidth.