A Social Scientist’s Guide to Diversity Strategy
Evidence, evidence, evidence, evidence, evidence, evidence. Where’s the evidence?
It’s an exciting time to be working on diversity in tech. The challenges are acute: lack of diversity is well documented and uniquely bad. But the opportunity to disrupt biased hiring and promotion practices is even greater — if tech leaders act on the industry’s principles of transparency, innovation, and data-driven decision making.
Already, tech leaders have made good on calls for transparency. Here are some beautiful graphs showing some pretty ugly statistics. And there is no shortage of innovative ideas for diversity interventions: unconscious bias trainings, blind interviews, applying the NFL’s Rooney Rule, embedding engineers at schools, and more.
The problem is that many companies are not applying the same data-driven standards to their diversity strategy as they do to build their products. Too many companies are relying on so-called “best practices” that research suggests do not work. And because companies are scrambling to find solutions, they aren’t properly testing the effectiveness of their interventions. Both strategic failures are costing companies time, money, and the gains associated with diverse talent.
Companies need to bring the same rigorous, evidence-based decision making to their diversity strategy that they bring to all other aspects of their business. Here are a few suggestions from a social scientist’s perspective:
- Start with programs that work
There are dozens of “best practices” out there. How should a company choose which to implement? Just because Google or Facebook is doing it, doesn’t mean it’s guaranteed to work at your organization (or even that it works for Google or Facebook, yet).
Luckily, there is some solid data on what interventions have been shown to increase diversity. Sociologists Frank Dobbin and Alexandra Kalev examined over 30 years of data on the gender and racial composition of American companies. Their data come from the U.S. Equal Employment Opportunity Commission (EEOC), which requires that all companies with 100 or more employees report demographic data on their employees every year, as well as from a random survey. In a 2006 study and a follow-up 2015 study, Dobbin and Kalev tested the effects of over a dozen corporate diversity practices on outcomes at over 800 companies.
They found that the following programs have positive effects on increasing diversity (listed in order of the size of the effects): establishing a diversity task force, hiring diversity staff, special recruitment programs, management training programs, mentorship programs, transparent promotion criteria, and implementing work/family policies. If you’re going to invest in diversity, why not start with these programs?
Notice that two of the most hyped and ubiquitous interventions, diversity training and networking groups, are not on this list. That’s because, unfortunately, there’s little evidence that either succeed in increasing diversity. In fact, some studies find that diversity training can have negative effects. Similarly, Dobbin and Kalev’s study found networking groups (think Women@YourCompany), on average, had negative effects on the representation of people of color at the companies they examined.*
Each company is unique and should carefully measure the effectiveness of diversity interventions in their own specific case. But at the very least companies should start with interventions that have been shown to have positive effects. Unfortunately, a random survey of large private companies found that twice as many companies have networking groups as have a diversity task force, for example, even though the latter has been shown to be more effective in increasing diversity.
2. Collect baseline measures
Put simply, the goal of a diversity strategy is to decrease bias and to increase diversity. In order to measure a decrease or increase of any kind, we have to start with baseline data. This is a critical but often ignored first step in the rush for solutions.
Organizations should measure diversity, inclusivity and bias with as much detail as possible. The most basic step is to conduct a demographic analysis of current employees and applicants (Buffer’s dashboard is a great example). Other baseline measures should be linked to specific areas where your organization wants to improve.
Concerned that female employees and employees of color are having negative experiences at work? Gauge baseline employee satisfaction with a survey that incorporates demographic questions. Worried that your hiring process is missing out on top talent because the resume is from Jamal rather than from John? Conduct a resume audit to find out. Is a failed or non-existent diversity strategy costing you cold, hard turnover costs? Conduct a demographic analysis of retention and link it to turnover costs. Do women systematically earn less than men, even when taking into consideration years of education, experience and other relevant variables? A salary audit will help you identify if and where this is a problem.
You can’t manage what you don’t measure. So measure early and often.
3. Experiment with a critical eye
No one has discovered the magic bullet for reducing bias and inequality in the workplace. Sociologists spend a lot of time diagnosing the problems that lead to a lack of diversity, but very little time developing solutions. Social and organizational psychologists test solutions in labs, but they’re often limited to using undergraduates in experiments, leaving serious questions about the applicability of their conclusions in the real world.
Meanwhile, companies are perfect laboratories. The opportunity to try new interventions and test their impact in controlled settings is enormous. This is why I believe data-driven diversity interventions are our best hope for disrupting bias. The key is to treat exciting but untested innovations as an experiment.
That means interventions should be anchored in theory and tested through thoughtful study design. Don’t know how to run a successful study? Hire a social scientist! They’re experts in social theory, designing studies, assigning test and control groups, and analyzing results.
4. Scale programs that work, get rid of the ones that don’t
Often it seems that organizations continue lackluster diversity programs because of path dependence rather than positive results. It’s understandable: someone, or a team of people, worked hard to design and implement those programs. No one wants to feel like their work was pointless.
Alternatively, have a rigorously experimental mentality when implementing diversity strategies. Every social scientist has had a brilliant hypothesis that just didn’t pan out in the data; they had to move on and test new theories. This ability to shift gears is critical because some interventions will undoubtedly be duds. Instead of being tied to a pet project, it’s important to move on and make sure resources are allocated based on evidence.
Start with programs that have been shown to work: 1) programs that engage managers rather than seek to limit their discretion and 2) programs that seek to change behaviors rather than attitudes. Collect baseline data. Be as specific and goal-oriented as possible. Innovate, but with a critical eye.
You expect your company’s product teams to rigorously A/B test, iterate and be data-first. If you are truly committed to hiring diverse talent, why would you expect any less from your diversity strategy?
* While diversity training and networking groups may have negligible or have negative effects on increasing diversity, as far as current research can tell, that’s not to say that they are not beneficial in other ways, like increasing retention or engagement among current employees. The point is that we need more data and that neither strategy should be seen as a panacea.