How do we ensure users receive accurate info in their calendars?

A huge part of our success comes from our ability to deliver accurate information to users via in-calendar content. It’s even what prompted Stanza’s launch back in 2014. And generally, you all love it:

Especially now, as global events have been canceled and postponed en masse, we know users rely on us to deliver accurate events to their calendars.

In addition, based on a 2019 user survey, we learned that maintaining accurate information in-calendar is our single most valued product feature.

Words like “teams” “accurate”, “games”, “favorite”, “events”, “schedule”, and “upcoming” showed up most prominently when we asked the question “What is the main benefit you receive from using Stanza?”

Senior QA Engineer Eugene Still led our most recent project to ensure users are seeing accurate information in-calendar. The overall objective was to eliminate duplicate and inaccurate events for Stanza users.

It’s rare that one person gets to own a project that involves so many different parts of the business: engineering, QA, customer technical support, user support, product operations and development operations. We love projects that solve multiple problems at once…but it’s even better when they’re objectively trackable and measurable. Data forever!

Goals & Timeline

Our engineering team uses web scrapers (pulling content directly from text on web pages) and APIs (retrieving data through a programmatic interface) to extract accurate, up-to-date data for customers. In this document we’re referring to fetching data via web scrapers or API as “scrapers/scraped events”.

Our planned action items, in chronological order:

• Create daily automated tests and track results
• Fix bugs so we can contact customers more rapidly
• Enhance customer dashboard to reduce duplicate/inaccurate entries
• Enhance ticketing partner rules to ensure customers are paired with the proper ticketing partners (TicketMaster, StubHub, SeatGeek, etc.)

Daily Automation Test Results

We run over 7,000 daily tests each day for customer calendars. Failure rate is almost always below 1% (and is often below .01%). Failed tests, however, do not necessarily mean there’s a problem — oftentimes, if a customer adds a new event, it can cause the test to fail without a bug actually existing.

Failure rate percentage for calendar accuracy is well under 1%

Scraper Ticket Resolution Time

Our engineering and product teams use JIRA (a ticket-based platform) to track and resolve bugs. An important focus for this initiative was to reduce the amount of time between when the issue is flagged and when it gets fixed. The goal was to complete scraper tickets as quickly as possible to minimize the chances that users would be exposed to inaccurate calendar events. Some highlights include:

• Tickets are opened proactively based on automation results.
• Scraper tickets are prioritized based on customer event frequency and are addressed faster.
• Q3 2019: avg ticket life was 26 days vs. January 2020: avg ticket life was 4 days

Hit us up at learnmore@stanza.co or on Twitter @stanzacal if you want to know more about how we’re working to ensure customers and users are seeing accurate information.