What Is The Real Value Of An Online Brand Community? Impact, not ROI.

Richard Millington
Oct 6, 2018 · 10 min read

The industry of community professionals isn’t alone in struggling to prove its value.

Public relations, mainstream advertising, human resource initiatives, and many (if not most) things an organization does don’t have a direct revenue generation attached to them.

The difference is most of us know what results we expect to see from these above activities and we value those results. The difference with community is it’s a term which encompasses so many types of social interactions which in turn produce such a diverse range of results (activity, posts, feedback, ideas etc…).

In the past few years, the community sector has largely been divided into two camps about community value. We can call these the ‘True Believers’ and the ‘ROI Advocates’.

The True Believers

The ‘True Believers’ generally feel an organization should have a community even they can’t prove the value of it. In their minds, it’s the right thing to do. This camp is best epitomised by people like Gary Vaynerchuk and his famous retort: ‘What’s The ROI Of Your Mother?’.

Not many of us have bothered to calculate the ROI of our mother, so don’t communicates have equally obvious apparently value?

In short, this group believes some things just have innate value and (like your mother) it’s nonsensical to even question it.

Isn’t it obvious that it helps to have strong, intimate, relationships with your best customers? Isn’t it also obvious that it’s useful to have a place where members to connect, solve each other’s problems, and build close relationships? Of course it is.

True Believers are most common among the newcomers to the field and technology startups (the latter often led by bosses or organizations who have benefited enormously from communities in the past).

There are two problems with True Believerism. The first is opportunity cost. No-one really doubts, all things being equal, that it’s better to have a community than not. But all things are not equal. The money spent on the community could be invested in other areas where it might generate even better results.

The obvious retort to “What’s The ROI Of Your Mother” is “What’s the ROI of cutting the community budget and investing in better customer support, in-person events, training courses, or better product people instead?”

The other problem with True Believerism is it leads to isolationism. It requires your colleagues to share the same (or similar) experiences which led you to become a True Believer. Many True Believers spend a huge amount of their time trying to convert others without realising their words can only take them so far. True Believerism ultimately requires a True Believer boss (and True Believer colleagues) to work.

It’s worth remembering too that others are also trying to impress colleagues with the importance of their field. Everyone wants more resources, support, and respect. PR, Sales, Marketing, Human Resources are all advocating for the unique benefits of their discipline.

The other problem with True Believers is it’s an engagement game. You impress colleagues by showing high levels of engagement. But engagement can’t keep growing forever. Worse yet, engagement only takes you as far as your colleagues value engagement. If they don’t much value engagement much, you’re going to struggle.

Even the most supportive environments can be hit by a sudden change (your boss leaves, a bad financial year, or a new strategY) which leads the community out in the cold. Even the most supposedly supportive bosses have no problem making big cuts and redundancies to the community team. Brian Chesky might call himself Airbnb’s Head of Community (he’s the founder and CEO), but Airbnb has been known to cut the community team and budget. Bosses move on, times change, and priorities shift.

The appeal of True Believerism to community professionals is obvious. It means they can focus on driving engagement without having to question whether engagement is valuable. But it’s also a self-limiting which confines a work into a small silo of likeminded believers.

(note: There are three important exceptions here. The first at hobbyists building a community or group for their passion. The second are advertising-supportive communities, where more engagement directly drives more revenue. The third are communities where the community is itself the product (often associations/member-driven organizations). However, even here it’s not enough to have a huge amount of activity.)

The ROI Advocates

The second group are the ROI Advocates. These are people who realize if an organization is investing significant resources in the community, they’re going to want to see something valuable to show for it. After all, if the community can’t deliver anything, it begins to look suspiciously like an extravagant expense instead of a worthwhile investment.

This group believe it’s critical (and very much possible) to measure the ROI of a community to a clear financial value (if not the actual efficiency percentage). Like True Believerism, this viewpoint initially makes a lot of sense. If the community is proven to deliver a great financial return on the investment it makes sense to increase the investment. If you’re delivering great results, showing a clear ROI is a powerful way to show results that everyone can understand.

ROI advocates tend to stumble in three areas. The first is organizations are not as logical and systematic about how and where they invest money as we might imagine. ROI is essentially an efficiency ratio developed to shrink a wide range of corporate activities into single percentages which can be compared to one another when making investment decisions. In theory, CFOs should invest in areas with the highest ROI.

Back in 2006 we created a huge resource explaining how to calculate the ROI of an online community.

Yet, almost no-one can prove the ROI of what they do. Human resources, PR, advertising all struggle. This makes meaningful comparisons almost impossible. More importantly, very few organizations prioritise efficient spending over hitting their main strategic priorities. These priorities might be maximizing growth, creating the best possible products, or having world-class customer support etc… Given the choice they would rather not waste money, but they’re not going to worry much if they do make some inefficient investments as long as they hit their major strategic goals.

This leads into the second point, very few people in any organization care about the ROI. If proving ROI was as critical as its advocates believe, the entire community profession would be out of work by now. The fact we still have jobs suggests its not quite as important as we think. No-one you work with is too worried about the ROI of their activities, they’re too busy hitting their immediate goals than efficiency metrics.

Perhaps the biggest problem with ROI is it’s extremely difficult to prove and our current approaches are wildly inaccurate. This often leads to absurd claims. Dell might claim every idea on Ideastorm generated $10k in revenue, but can anyone name a single innovative idea since launching the platform? It can easily be argued Dell has been one of the least innovative tech companies since launching Ideastorm.

The ideas implemented on IdeaStorm don’t exactly scream innovation.

A common approach is to compare members with non-members. This always reveals community members buy, create, or stick around a lot longer than non-members. This is because the people most likely to join a community are your best customers/members/supporters (i.e. the more Apple products you own the more likely one will break and you will join a community to get help). Unless you run a controlled test, the result is meaningless.

So many things influence buying behavior, it’s hard to establish the community’s value.

Perhaps the most interesting approach is call deflection (or the networked value of an answer). This approach establishes the value of an answer and multiplies this by the number of people who see the answer. Customer support is most common. If people get the answer online without having to file a support ticket or phone a support rep, that saves money. If you can find out how much each call costs (i.e. total support costs / no. calls) and how many people successfully get an answer you can theoretically calculate the number of calls you saved support staff from answering.

This might be the best approach we have, but it’s also incredibly flawed. The definitions of support costs and, most importantly, a successful search are so vague that you can come up with any answer you like (hence the 2500%+ ROIs on communities using this method).

For example, Ted’s definition of a ‘successful search’ here is a user who clicks on a search result and doesn’t search for anything else within a short time period.

This metric might include satisfied customers who got an answer, but it would also include people who didn’t get an answer and went elsewhere (i.e. perhaps even filed a support ticket). It would include people who came via search and landed on the wrong answer. It may even include bots crawling the page. The number of successful searches are likely in the minority. Using survey results (pop-up or otherwise) is slightly better, but the majority of visitors (the disgruntled ones!) won’t ever complete them which biases the results heavily.

This also neglects whether the answer was the best answer, whether it was kept up to date, and whether members browsed several similar answers before making their choice. Worse yet, estimating the value of an answer allows plenty of room to rig the final metric positively in your favour. Do you include just staff costs? Technology? Sunk overhead costs? Even really small changes here (i.e. changing $3.5 to $4 per call) has a 14%+ difference in the final calculation.

The biggest problem with call deflection is it doesn’t actually measure cost savings, it measures theoretical cost savings. Cost savings aren’t realized until the organization genuinely cuts the customer support costs i.e. you might estimate $10m savings on call deflection, but has the customer support budget genuinely been cut by $10m? Usually the support costs remain the same. Sometimes, the calculated savings even exceed the entire support budget.

(quick aside, if the budget is to cut the customer support budget (i.e. reducing headcount), don’t expect much support from the customer support team. Do we really want to be measured by the number of customer support staff who lost their jobs because we persuaded members to work for badges instead of pay-checks?)

Too often ROI calculations reveal extremely high theoretical returns which never show up on the balance sheet. Sooner or later, someone (likely the CFO) is going to notice that and behave accordingly.

Given the choice, it can help to know the ROI. Some CFOs in larger organizations do demand to see the calculated financial return. But this is incredibly rare and the time and effort used to determine is time far better deployed elsewhere. ROI Advocates also tend to fall into the trap of looking to see what was valuable instead of knowing what results they tried to drive in the first place. We need another approach here and another set of metrics.

Pragmatists Measure The Impact Of Their Work

By now we know we need to prove value, but specific financial metrics aren’t the best fit.

What we need is to focus on the impact of our communities.

Everyone in an organization has some idea of the results they want to see from their work. We need to align and measure our community’s impact on those results.

There are three rules of impact.

  1. Impact is specific and visible. Focus on tangible results you can show people rather than vague metrics you can tell to people (more on this shortly).
  2. Impact is something someone is responsible for. Your colleagues probably don’t have responsibility for ROI or the brand’s mission, but they do have responsibility for their team, showing results from their work, and keeping customers happy/productive etc..
  3. It has to be something (a) colleague(s) care about. Whatever the community will impact, your colleague(s) have to want to improve it. There is no point impacting something someone doesn’t care about immensely. It’s possible to be responsible for something, but not have a great interest in improving it.

When you focus on impact, you know what results you want to see before you begin. This prevents the ‘hunting for value’ problem. You also have support from your colleagues because you’re beginning with something they already support.

The difference with impact is that it’s usually quite specific and targeted. Some examples include:

  • Getting great feedback to fix product problems.
  • Identifying and fixing problems/bugs.
  • Sourcing great content ideas.
  • Testing and improving marketing campaigns.
  • Getting the best insights about (or from) customers.
  • Reducing time spent looking for information.
  • Reducing newcomer churn.
  • Increasing the number of customers who develop their first apps/make their first use of product.
  • Turning members into advocates.
  • Identifying new sales leads.
  • Increasing awareness of new products.
  • Driving more search traffic.
  • Building a repository of all our best official and unofficial product information.
  • Collect great case studies for our sales/marketing teams.
  • Get top members publishing reviews on major comparison sites.

All of these (and many, many, more) are specific and tangible. You can see reviews published by members, case studies sourced, or insights generated. You can clearly see a decline in newcomer churn or the results of testing marketing content ideas. All of them is something that someone is likely to be directly responsible for.

Impact comes in many forms with varying levels of influence within the organization

Once you know what impact you want to see, you can determine what you need members to do to achieve it. This often means you need far less activity from far fewer members than you might imagine. You don’t need anywhere near as much engagement as you do today to get most of the benefits above. It completely changes the game of what we measure and how we work. We can focus on building closer relationships with fewer people and helping them to do the things that have the biggest impact.

The key to making this work is finding out what your colleagues want (or need) and aligning the impact of your community to match. Don’t come up with the impact first and see who cares about it. The impact has to be sourced from your colleagues. This stops you spending the next few years trying to persuade people why that impact matters.

It’s clear we need metrics to fill the voice between True Believerism on the left and ROI zealotry on the right

If you’ve done this right, you shouldn’t have to explain the value of community often. But, when you do, you can show the clear and specific impact of the community. Better yet, once you’ve determined the impact you’re trying to make, it becomes far easier to have that impact.

Richard Millington

Written by

Richard Millington is the Managing Director of FeverBee, a community consultancy which has worked with Apple, Facebook, SAP, and 250+ other top brands.