Q-Interview: Creating the Get HYPE Philly! Dashboard: What We Learned
Welcome to The Q — an interview series where we invite the Equal Measure team, clients, and colleagues from the field, to share their insights on evaluation, philanthropic services, emerging trends in the social sector, and more. Inspired by Stephanie Evergreen’s blog “The Problem with Dashboards (And A Solution),” we took a different approach to developing our recent evaluation report for the Get HYPE Philly! initiative. Here to share what we learned by taking the road the less traveled are Equal Measure Director Kim Glassman and Bill McKinney, PhD, Director of Research and Evaluation at The Food Trust.
Q: Bill and Kim, to set the context, can you describe the Get HYPE Philly! initiative and its objectives?
Bill: In a nutshell, Get HYPE Philly! is a collective impact project in Philadelphia, in which 10 partner organizations focus on supporting leadership, healthy living, and healthy choices for young people. We want folks to exercise more, we want them to eat healthier, and we want them to become stronger leaders. And because this is a collective impact project, our aim is to support how the partner organizations work together, expand their networks, and build sustainable programs that can last for a long time.
Q: So let’s turn to the dashboard, which Equal Measure developed in lieu of a more traditional evaluation report. What was the impetus for shifting toward a more visually compelling medium to communicate the impact of Get HYPE Philly!?
Kim: I think a major factor was that the field continues to move away from thick, text-laden reports to a more visual presentation of data and findings. We wanted to dip our toes into those waters, and explore whether we could develop an interim evaluation report in more of a dashboard style. We also knew that The Food Trust was putting together its own interim report about Get HYPE Philly!, in a more narrative, comprehensive format. We thought that our two reports, presented in different styles, would complement each other. I think we were able to design a report that balanced data visualization with some more narrative elements.
Bill: Kim raises an important point about balance. As researchers and evaluators, sometimes we do things in extremes. Either we produce a report that is very academic or technical, or we create a one-page dashboard, which is limited in depth. I think this dashboard sits in a nice space — it has a good mix of analysis and narrative, yet it also can engage people and is relevant to a broader audience.
Kim: We wanted to present the vast amount of qualitative and quantitative data that we were collecting in a more coherent way, and a dashboard format allowed us to neatly categorize our key findings according to Get HYPE Philly!’s four levels of impact.
Q: What are those four levels?
Kim: There’s the individual level, which is centered around youth participants, but particularly youth leaders. There’s the collective level, which is the ten partner organizations. Then there is the community level, which covers the community of Philadelphia. And the communications level focuses on how Get HYPE Philly! shares its impact and builds momentum for the initiative.
Q: Bill, you mentioned the importance of reaching a broader audience with this dashboard. For both of you, whom do you consider as the target audiences for the dashboard?
Bill: Right off the bat, I’d say our partners. Since this is a formative evaluation, they are part of the process. So being able to share information with them in an accessible way will help them make adjustments in their programs, if necessary. If we presented this information in a more technical manner, that was difficult to digest, we would not have the desired result.
I think there are aspects of the dashboard that can be very interesting to our current funder and potential funders. And as the backbone organization, we too are an audience, in a way. One value of this dashboard format is that we can share specific pieces with different audiences — whether its partners, participants, specific communities, funders, or policymakers. There is information in the dashboard that’s useful for any of those groups.
Kim: I agree that Get HYPE Philly!’s partners are a top audience, and they were also important contributors to the development of the dashboard. Over the course of the evaluation, we rely on the partners for a lot of quantitative data. They collect youth leader surveys. Partners also enter data on the youth they serve, the programs they run, use of social media and the organizations they partner with. We think the dashboard format offers partners an opportunity to combine different sources of data, to paint a more vibrant picture of the entire initiative.
Bill: That’s right. We rely a lot on partners to collect a lot of the data, and the more value they see in this activity, the more they’re going to be willing to participate. That’s why it is so important to communicate to them, ‘this product is what came out of your partnership with us in collecting these data.’
Q: Developing the dashboard took a number of months. What did you learn from this process?
Kim: One thing we learned was that the dashboard should be an evolving tool that is updated regularly, rather than a static report produced every year. Bill really pushed us on that. That realization took a while, particularly since I was used to thinking about presenting data in the style of a more traditional report. Now we’re at a point where we already have data for 2015 and 2016, and can pop in data for 2017 in the next iteration of the dashboard.
We also added an introductory slide and a concluding slide for each of the levels — to more clearly explain the data and findings to our audiences.
Bill: I think we learned a lot about the time it takes to create a quality dashboard. We probably underestimated the time needed to get it right. We could have ‘finished’ sooner than we did, and moved on. But we would have sold ourselves short, and I think everyone else short. I think for the future, we probably need to build more time for development and planning.
Q: So as you went through this process, were there any ‘ah-has’ that surfaced?
Kim: One of the biggest ‘a-has’ was Bill’s suggestion to track data back to the beginning of the initiative in 2015. We were able to see a consistent rise in different metrics over time, like with GHP’s consistent growth in social media impressions. And we could see that growth through the different data visualization graphics we used in the dashboard.
Bill: For me, it was that this is not a dashboard for a traditional midterm report. I think this dashboard serves a much more important purpose. It is an opportunity for people like us — researchers and evaluators — to communicate data and information to people in a much more visually compelling way, and to set the findings in the broader context of Get HYPE Philly!’s growth since the beginning of the program.
