Build a Better Performance Measurement System: Step 2
Step 2: Scope the Work
If you like the order of “Plan, Do, Check, Act” with an emphasis on ‘Plan’ then I’d bet my bottom dollar you will love this step. If you’re more a “Fire, Ready, Aim” type then get ready to step outside of your comfort zone — because this might be the most important step in the process of building a truly better performance measurement system!
How you scope the work for designing and implementing new data system will greatly influence your success. By success I mean — whether your final product delivers to meet your needs, and whether it gets deployed on schedule and within your budget.
First, let me dispel a myth. Just because a product claims to be tailor made for your needs — does not mean it actually will meet your needs. Most products have significant limitations when it comes to configuration to adapt to your way of doing business.
To illustrate…I was recently on a business call with a volunteer management software company seeking to sell us their product. The product was clearly designed around a volunteer use case where volunteers sign up to attend a one-time event or have a predictable schedule. While that is sometimes the case for our organization, our most critical volunteers serve as tutors and mentors whose schedules are flexible and often change from week to to the next. The assumption that all volunteers will sign up to attend an event at a certain day and time to create a “schedule” that then serves as the keystone for volunteer management does not hold in our organization. And yet, that assumption is core to their product and is not able to be changed. Sure, we can strategize how we work around that, but it shows a product’s limitation.
There is no such thing as a perfect data system. Identifying a product’s limitation is the point of Step 2. An excellent scope of work helps you to highlight a the strengths and limitations of a particular data system, as well as the partners who will help you design and implement it for your organization.
The scope of work helps your team evaluate the options and make an informed decision with the best information available to you.
A scope of work answers the questions:
- What are the specific requirements of the data system?
- Who will use it and how?
Specifying Your Requirements
The best way to define your scope of work for a data system is to create a Product Specifications Document or “PSD” (mentioned also in Step 1). A PSD communicates the expectations and the requirements of the new system. This document distinguishes features that are “must have” from those that are “nice to have.”
Key elements of our PSD included: Purpose, Priorities, Workflows, and Sample Reports. Need examples? Read on!
Purpose of the data system
Make the purpose of your data system clear and explicit in the PSD. This will ground the work in the vision you articulated in Step 1.
At Boys & Girls Clubs of the Peninsula, our purpose highlighted the need to effectively and flexibly capture information on what services students were receiving, where, and how often. By effective, we meant — accurately and easily. This was spelled out when we described the requirements and specifications later in the document.
Flexibility was paramount because our services and programs are always changing. Now that we are in the midst of a global pandemic limiting our traditional in-person services, we have reaped the benefit of a flexible system. Within hours we created and deployed new virtual services in our Salesforce / Exponent Case Management data system. This allowed us to began tracking virtual service delivery as soon as services began. This would not have been possible in our previous system.
Priorities
Discerning “must have” requirements from the “nice to have” is not easy, especially when you gather input from many different stakeholders. Trying to do it all is a recipe for failure. But what do you do when a colleague makes an impassioned plea for a feature you know is going to take the project off track or require outsized resources to accomplish?
We handled input on priorities in a couple ways:
- We created a multi-year, phased-in approach that allowed us to identify the highest priority items relevant to achieve the goals of Phase 1, and address other relevant but less immediate goals at a later date.
- We established boundaries for input by focusing Phase 1 on enrollment, class assignments (rosters), and attendance tracking. We kept returning to these 3 big “P” Priorities over and over again, especially as new fun features cropped up that threatened to sidetrack us (and our timeline and budget).
- We tracked all the priorities people mentioned in a shared spreadsheet; this showed we were listening and taking feedback into consideration.
Managing input from our stakeholders helped us rank priorities from highest to lowest and helped us stay focused and on-track.
For each priority, we did our best to provide specific, measurable goals with clear specifications. When possible, we shared how we currently operate and pain points we wanted to avoid. This information set the stage for the user stories that our data system implementation partner would use to configure the data system to meet our specific needs.
No matter how well thought out your PSD is, be prepared to adapt and make changes along the way. We had to make many choices after Discovery. Sometimes issues that were overlooked at the start of the project, became pain points that required time and money to change later on. For example, we hadn’t initially specified the need for students to be able to attend classes at other sites without being enrolled there.
We also learned that certain specifications would be easier…err…less expensive…to meet and would be adequate. For example, we had initially specified wanting to be able to assign multiple students to a class and multiple classes to a student. The former was do-able within the current system. The latter was not and would have required customization and development beyond the scope of our timeline and budget. At the end of the day, having the multiple students to a class option was adequate, and we’ve heard no complaints from staff to suggest otherwise.
Workflows
The way our Boys & Girls club staff interact with our performance measurement system is different than the way that other nonprofit staff interact with their system. Through the Discovery and Design process, I learned how important it is to make sure workflows are clear and consistent before designing or configuring a technical solution. Data systems should reflect your business processes not determine them.
Take enrollment, for instance. Here are just a few of the questions we had to answer to create a clear and sustainable enrollment solution for staff:
- Are applications received online or paper only?
- When and how often are families expected to complete an application?
- How do we know if students are in the same household?
- Are applications received any time or only after families attend an orientation?
- Who enters application information into Salesforce and when?
- When an application is received and entered how, when, and who determines if that student has a spot in the program?
- When a student is confirmed with a spot, what needs to happen next?
As you can see, these are not technical questions. These are questions that inform the business processes or “workflows” related to enrollment. Setting up an effective ‘Intake’ (i.e., enrollment) in Salesforce required us to have clarity and some level of consistency in how enrollment was managed by different sites and for students in different grades.
When we went into the Discovery process, we hoped we could pilot, or even launch, an online application. BGCP had used paper applications exclusively to register students. While there are benefits to paper applications, it is very burdensome from a data entry perspective, as well as for our families, who are asked to complete the same registration information year after year and sometimes twice in a year (for school year and summer enrollment). In some cases, it prevents us from being able to serve students — especially those in high school — quickly.
However, when our implementation partner interviewed our site leaders, it became clear that our school sites and clubhouses had very different workflows when it came to enrollment. Some only collected applications for students after parents attended an orientation, while others allowed anyone to sign up. Staff were unsure most caregivers would complete online applications on their own.
Because of the different workflows being utilized by sites, we were faced with a decision: change the way we enroll students at sites to accommodate online applications or continue to utilize paper applications. We decided that enough change was coming with the switch to Salesforce that we were not ready to upend our enrollment process during Phase 1. Rather than walk away from the idea entirely, we kicked it to Phase 2. We decided to later pilot an online application for our high schools students, where the desire and need was greatest. (Sidenote: Coronavirus has accelerated our need for an online application solution and now all sites are on board to utilize online sign-ups).
Reports
Getting data in the system is only half of the equation. Reporting on that information to inform decisions and take action is the other, better, half.
At the end of Discovery, the team at Exponent Partners surprised me by saying, “Your organization really uses a lot of reports!” I agree, but hadn’t realized this was anything exceptional. Not only do we generate a large volume of reports, but many require nuanced analysis. For example, we want to calculate average daily attendance, but not accidentally include weekends or days when sites may be open only to a small number of students (e.g., due to a field trip or school closure day). In our annual report, we communicate breakdowns by gender, race/ethnicity, and frequency of attendance, but not for all students served, only for those that attend 2 or more days a week (or who participate in our literacy or advising programs).
Leading up to writing our PSD, we experimented with various types of reports. We utilized a small (but growing) number of embedded reports that gave service counts, including daily attendance, as well as information about our students. These were able to be filtered, but staff were not able to customize, save, or build their own reports in our previous system (spoiler: not good). Reports generated in our previous system also did not have their own unique URL to link to in an email or Slack. The absence of a report-specific URL made flagging reports with data for follow-up unnecessarily laborious. Because staff were not using reports routinely, our team ended up sending data tables weekly to site leaders with their site-specific information. Note: Do not do that; it sets a terrible precedent that is hard to un-do and creates over-reliance on a centralized team for monitoring staff and program performance.
In addition to creating embedded reports, we began piloting Tableau to address limitations in table lookups/joins and we started using SQL queries to analyze data more efficiently. Learning a new platform, especially Tableau, was no small feat, but was aided by initial support and training from the Project Evident technical assistance team. However, while the volume of data we analyzed in our old system increased, we were not becoming more efficient. Our new system had to address this challenge in order for our performance measurement system to be sustained.
So when it came to defining our reporting priorities in our PSD, we specified which reports mattered most. Then we identified which reports which required (relatively) simple calculations and were (or should be) used frequently enough to embed the coding and information in Salesforce. In the end, we invested in creating a system that would enable us to report easily on daily attendance (unduplicated counts), frequency of attendance (based on a % of program days attended), and a unique field specifically to identify “active members” based on several calculations.
At the end of the day, not all of the reports we specified in our PSD were feasible to embed in our data system. For example, we still can’t view a report of average daily attendance in Salesforce. I mean, it’s possible, but we haven’t found it yet is worth the time or effort to use Roll-up Helper to do it. Most of the time, staff and leaders would prefer to view daily attendance as a (line) chart anyway, so they can easily spot trends. When average daily attendance is needed (some funding sources require it), we export the data to Excel and quickly calculate it when needed. And (thankfully) everyone in our organization can do this, too.
It’s Only a Plan, After All
Having a strong PSD is only a start. And no matter what, once the work starts you will learn things about your organization and the selected data system that will require you to rethink and iterate on your priorities and your plan.
Be patient with yourself. Be gracious with others. After all, “Great design is iteration of good design.”
If you have read this far, give yourself a pat on the back. This was a 9 minute read!?! And please “clap” so others see this, too.
