Calculating First Reply Time for Intercom Conversations

I don’t know if you have noticed, but Intercom definition of built-in First Reply Time metric is a bit unusual. Here is how it works according to Intercom:

The median first response time is the time within which 50% of the messages received were responded to. It’s a better measurement of response times than average, because with average response time, a few messages received at the weekend or during non-business hours could take much longer to reply to and skew your numbers.

Let’s make sure we are on the same page with the definition. So to calculate this number, Intercom takes all new incoming messages and find 50% of conversations with the best first reply time. Now, if you take the worst first reply among this 50% of conversations, you would get your Median First Reply Time.

For example, if for a specific day, your median response time was 45 minutes, it means 50% of your conversations were responded to in less than 45minutes, and 50% were responded to in more than 45 minutes.

I see few problems with that definition:

• Whatever number you get, it is hard to build an intuition if it’s a good value or not. For example, if your first response time is 2 hour for 50%, but for other 50%, it’s 2 days. Is it good or bad?
• It doesn’t account the other 50% of your conversations with users. For example, you might be doing a good job during your business hours and have 2 hours first reply time for 50% of your customers, but your other 50% might be waiting for 8+ hours.

• This definition is not standard across the industry, so if you want to compare your numbers to companies using tools other than Intercom, like Zendesk, Help Scout or Salesforce, you are out of luck.

In addition to built-in median first reply time measuring, I recommend you also track average first reply time as defined here. Metric is calculated in calendar minutes from when the ticket was created to the first agent reply. The calculation is for all tickets created in the specified timeframe. For example, if your team received 3 new tickets yesterday. The first was replied to in 2 minutes, the second in 5 minutes and the third in 8 minutes. Your average first reply time for yesterday is 5 minutes.

• Tracking first reply time for all conversations would make sure you are not sugarcoating the results. If you have outliers that are making numbers look bad, you have to take those into account. Every customer message deserves a fast reply.
• Said that, you can also find flaws with calculating averages. So try to track this in addition to other metrics including median provided by Intercom.
• Even better, if you look into different problematic conversations one by one, to see the real reason why your first reply numbers are lower than expected. This would guarantee you have a full understanding of the problem.

To automate First Reply Time metric calculation, you can use Rasper or write a script to load all conversations from Intercom API and calculate metric manually.