EXPO Summit 2020

Anthony Tang
Feb 7 · 8 min read

2nd Annual A/B Testing Conference in Carlsbad, CA

EXPO SUMMIT 2020 provides the opportunity to learn about A/B Testing process, analytics, and technology.

Register Today! Registration closes March 3, 2010.

On March 10, 2020, EXPO Summit 2020, the 2nd annual A/B Testing Conference organized by Walmart Labs will be hosted at the Carlsbad Conference Center in Viasat. This year, the conference will feature speakers from American Express, Convoy, Hulu, Lowes, Microsoft, and Walmart Labs.

Attendees will hear from industry experts about how Experimentation and Optimization is done and the technology behind it, from experimentation platforms and process to the reporting and analytics performed. There will be also be a live panel hosted by the Test and Learn Community, as well as opportunities for Q&A.

Check out last year’s conference to get a taste of what to expect this year, then come join fellow A/B testing product and experimentation program leaders and practitioners for a day of deep learning and networking this year.

Schedule

8:30am — 9:30am — Registration/Breakfast
9:30am — 10:00amOpening KeynoteFrank Zhu
10:00am — 10:40amStarting With the Viewer: Experimentation at HuluChris Gorski and James Wann
10:40am — 10:55am — Break
10:55am — 11:35am‘Two Week Tests Must Die’ : Myth-busting Experimentation Best PracticesChad Sanderson
11:35am — 12:15pm One XP Experimentation Platform: Key Successes and Learnings Greg Sylvester II

12:15pm — 1:30pm — Lunch

1:30pm — 2:10pm The Holy Grail to Building an A/B Testing Team From ScratchKenya Davis
2:10pm — 2:50pmDesign Experiments that MatterChris Kahn
2:50pm — 3:00pm — Break
3:00pm — 3:40pmProblem Solving through A/B TestingKartikay Sahay
3:40pm — 4:00pm — Break

4:00pm — 5:00pm — Live Panel with TLC

5:00pm — 5:15pm — Closing Remarks

5:15pm — 7:30pm — Networking/Happy Hour

Register Now!

You must register by March 3, 2020. There will be no on-site registration.

Opening Keynote

Frank Zhu has worked in the analytics area for over 20 years, building and running analytics teams covering product, marketing and business performance. Currently he is VP of Product Analytics at Walmart Labs. His team is a major user of the Expo A/B testing platform, managing experiments and analyzing results. As such, he has been heavily engaged in the development and improvement of the platform to meet the needs of experimentation and optimization at Walmart.

Starting With the Viewer: Experimentation at Hulu

Controlled experiments have become an increasingly important part of how Hulu builds products for our viewers. Hulu has taken experimentation from a pilot program to a full fledged practice, capable of running experiments on the web and dozens of devices using Hulu’s in-house platform. In this session, we’ll share technical and organizational lessons learned along the way.

James Wann leads data applications engineering at Hulu supporting real-time operational intelligence pipelines and tools. Prior to Hulu he was an entrepreneur with focus on building automated content engagement tools. He received Bachelors degrees from UC Riverside in Computer Science and Mathematics.

Chris Gorski leads the experimentation technology practice at Hulu. For the past 4 years Chris has embarked on a journey to bootstrap, evolve and grow the experimentation platform and delivery functions at Hulu. Prior to Hulu, Chris spent his formative years building experiences in game development. Chris received a BS in Computer Science from UIUC.

‘Two Week Tests Must Die’ : Myth-busting Experimentation Best Practices

Experimentation as an industry is filled with best practices and tribal knowledge, some of which started with experimentation software and other-times with top practitioners like Microsoft, Google, and Amazon. When put to the test, how many of these best practices hold up to common sense scrutiny and what should we do to improve our tools and programs moving forward?

Chad is the current Head of Product for Data Platforms at Convoy, and has worked on Experimentation teams for Microsoft, Sephora, Subway, and Oracle. He is a passionate believer in the power of optimization and the value of measurement.

One XP Experimentation Platform: Key Successes and Learnings

Experimentation is part of a culture at American Express and is fundamental to understanding our customers’ needs and wants to design the world’s best experiences. This talk will provide a view into an internally built experimentation platform One XP launched in 2019, why we started building an in-house platform, challenges we are running into, what we have solved, and what’s next for the platform. I will provide the top trials to move in the direction with a testing program that starts and ends with data.

Greg is an established product leader currently working as a VP of Content and Data Platforms at American Express. He has worked at multiple companies leading product, design, engineering, and data science for approximately 19 years. Such notable companies are eBay, PayPal, Intel, Honeywell, Las Vegas Sands, and IBM. His education spans Computer Science, Accounting, Biomedical Engineering, and Psychology at both Undergraduate and Graduate levels. He loves to spend time with his wife and kids reading, traveling, running marathons, biking, and humanitarian efforts.

The Holy Grail to Building an A/B Testing Team From Scratch

You’re a big business with big problems. How do you start solving them? Starting up an A/B testing team from scratch is a daunting task. Understanding what A/B testing is to your company and what technology to use is even bigger, and most of all, it’s time consuming. There are also topics of how to isolate the tests to increase velocity with your growing A/B Program, how to get leadership buy-in, and the end to end process of running every test to consider. Let’s dive into the Internal Optimization Start-up company your big business needs to invest in.

Kenya Davis created and now leads the team of A/B Testing analysts that run dozens of tests a quarter across all Lowe’s digital platforms. Her team works with many stakeholders across Lowe’s including product management, marketing, merchandising, and innovations. Her motivation comes from the desire to weave the fundamentals of scientific experimentation into the culture of digital A/B testing. Her fast growth in her career from a Resolutions Analyst, to an Online Operations Manager to now the leader and creator of A/B testing for Lowe’s is proven in the results of her team. She prides herself on her ability to grow and develop diverse teams while delivering revenue gains. Kenya is a proud alumnus of UNC Charlotte with her Bachelor of Science in Applied Physics with a concentration in Astrophysics.

Design experiments that matter

How do you scope and build experiments that focus on business priorities so that you use your program resources to maximize impact across the organization and continue to grow in scale and influence. Chris will be discussing three tactics with illustrations from his own program — build an advocate and SME network that provides an outside perspective, develop reportable program goals that align with overarching leadership goals, and communicate structural obstacles to leadership.

Chris Kahn is a senior program manager at Microsoft currently focusing on purchase path experimentation and piloting personalization tactics and strategies as the program has evolved from sporadic testing to building, launching, and enabling dozens of experiments every month globally. Previous to Microsoft, Chris worked at a venture backed start up and at Expedia in strategy roles. Chris has a literature degree from Princeton and started his career as a naval officer and SEAL before getting an MBA focusing on marketing analytics at the University of Pennsylvania.

Problem Solving through A/B Testing

While advantageous to other methods, AB testing is still a sampling method to understand population behavior with inherent risks in misunderstanding insights. In this talk, using examples at Walmart Labs, we’ll discuss problem solving and framing a hypothesis, avoiding common pitfalls, responding to unpopular outcomes and driving an effective learning arc.

Kartikay is a Senior Product Manager at Walmart Labs, focused on marrying supply chain opportunities to a delightful customer experience and building processes to learn and iterate faster. Prior to Walmart Labs, Kartikay held engineering and product roles at Yahoo. He holds an MBA from Carnegie Mellon and a Bachelor’s degree in Computer Science from India.

Live Panel with TLC

The Test & Learn Community (TLC) is a community of experimentation practitioners and subject matter experts who connect regularly to discuss and debate hot topics in the industry. All conversations are recorded and loaded to the TLC YouTube Channel. At this year’s EXPO Summit, Kelly Wortham, founder of TLC, will be leading an expert panel conversation on topics including the decision to build vs buy experimentation technology, organizational model impacts, and privacy implications to testing.

EXPO Summit Details

When

Tuesday, March 10, 2020

Where

Carlsbad Conference Center at Viasat
2456 Town Garden Road
Carlsbad, CA 92009

Cost

Free!

Schedule

8:30am — 9:30am — Registration/Breakfast
9:30am — 10:00amOpening KeynoteFrank Zhu
10:00am — 10:40amStarting With the Viewer: Experimentation at HuluChris Gorski and James Wann
10:40am — 10:55am — Break
10:55am — 11:35am‘Two Week Tests Must Die’ : Myth-busting Experimentation Best PracticesChad Sanderson
11:35am — 12:15pm One XP Experimentation Platform: Key Successes and Learnings Greg Sylvester II

12:15pm — 1:30pm — Lunch

1:30pm — 2:10pm The Holy Grail to Building an A/B Testing Team From ScratchKenya Davis
2:10pm — 2:50pmDesign Experiments that MatterChris Kahn
2:50pm — 3:00pm — Break
3:00pm — 3:40pmProblem Solving through A/B TestingKartikay Sahay
3:40pm — 4:00pm — Break

4:00pm — 5:00pm — Live Panel with TLC

5:00pm — 5:15pm — Closing Remarks

5:15pm — 7:30pm — Networking/Happy Hour

Register Now!

You must register by March 3, 2020. There will be no on-site registration.

WalmartLabs

Using technology, data and design to change the way the world shops. Learn more about us - http://walmartlabs.com/

Anthony Tang

Written by

Leading the experimentation platform team at WalmartLabs building Expo, the A|B testing platform for Walmart.

WalmartLabs

Using technology, data and design to change the way the world shops. Learn more about us - http://walmartlabs.com/

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade