Build a Better Performance Measurement System: Step 3
Identify the Platform / Vendor
If you are in the position of rolling out a new performance measurement system, be prepared for some sleepless nights. What will bring you comfort is a great partner who understands your business needs and works with you to design, configure, and train your staff to utilize it well. Choosing the right platform is as important as choosing the right partner to bring your vision to life.
The Great Debate: Closed vs. Open API
In many cases, the company that sells you the technology is the same as the company who will configure it to meet your nonprofits’ needs. Performance measurement systems fall into two categories: Closed or Open. A data system with an Open Application Programming Interface (API) has been developed with the intent of it being accessible to outside developers. A major benefit of systems with an Open API is the ability to connect data systems to each other (no more double data entry) and to work with contractors outside of the company to configure or customize a data system to meet your needs.
“Closed” data systems often require that you utilize their staff to make updates or changes, as a way to profit from having captured you early on. Want to make a change? “Sure. That’s another $5,000.” Need a special report that is not “baseline”? “Well, we configured it for someone else, but since you’re asking for it and it’s not baseline, that will be another $10,000.” Want to hire someone else? “Sorry, no dice.” The only people with access to the backend architecture of a closed system is…the vendor. There is no. where. to. go. for. help.
While this might sound terrible, there are also good reasons to go with a data system that is a closed API. One reason is that your in-house capacity to get a system up and running is low. A second is that your data system needs are a match for the closed system out of the box. If these two hold true, a closed system may deliver exactly what you need (and be your only option).
However, if how you capture, report, and use information is complicated or may be changing, but prepared for some serious frustration.
Why do I rail on closed data systems? Well, I inherited a data system disaster and I do not want you (or your successor) to suffer the same fate.
The Cautionary Tale: What Not to Do
In 2017, a year before I joined #teamBGCP, Boys & Girls Clubs of the Peninsula was looking for a new system to monitor program effectiveness and track outcomes. Staff had been using a patchwork quilt of an outdated Access-based system and Excel to capture daily attendance, and leadership knew it was time for a change. They were ready to implement outcome-oriented programs and needed a more robust CRM to tie students’ participation in BGCP programs to school success outcomes.
At the time, BGCP and its advisors compared four performance measurement systems, and chose to deploy a data system with a Closed API (most are) for performance measurement. They seemed to go through many of the right steps. They spoke to several companies about their products, brought together a cross-functional team to evaluate the demos. They had a scorecard with a numerical ranking. They checked the boxes. (Notably, they did not create a Product Specifications Document — see Step 2).
However, in the end a team selected a system that I had not heard of, but whose sales team promised could do all that we asked for and more!
The organization learned, the hard way, how easy it is to get sales’d.
Demos showed the team what was possible, rather than probable. Sales reps touted features that weren’t yet rolled out or required additional customization work that was not included in the initial contract. Sure, all our requirements were “possible,” in theory, so long as our budget and timeline allowed.
To avoid getting sales’d, we must separate the possible from the probable.
Three sentences cut through the bull: “Has this been done?” “With whom?” “Show it to me.”
This is why a clear and detailed Product Specifications Document is so important. Without a clear set of requirements, the sales pitch will get you every time! After all, you are up against (pretty) smart sales representatives whose jobs — and livelihoods — depend on you thinking that they are the answer to all your data system woes. Sales people are just doing their job.
Here’s how you can do your job, as well or even better. Watch out for:
- Universal “Yes” — Watch out when the response to every question about a requirement is “Yes, it’s possible.” Remember: ask when and how it was done, ideally with a client example. No data system is perfect, and you want your vendor to be realistic about what is possible for your organization.
- Flashy Features — It’s easy to get caught up with flashy features that are fun or interesting, but are unnecessary. Keep your vendor (and any team evaluating your options) focused on your priorities. If a vendor can’t stay focused on what matters most to you during the demo, that behavior will likely carry on through design and implemention.
- Roadmap Promises — Every thriving company has a roadmap of features or new products it plans to rollout in the next 6 to 12 months. Sometimes, these features will be requirements of your system, and other times they will be nice to have. As an optimist, I know it can be tempting to bet that those roadmap features will rollout just as you need them to at exactly the right time. But in reality, roadmaps are simply well articulated dreams. Don’t place your bet that they will come to fruition. You need to make a choice assuming that the roadmap could change and that hoped for feature never gets developed.
- Sales-only Demo — If you are having a technical demo you need someone who is actually technical (a developer or troubleshooter from customer support) from the vendor on the call. A sales rep will not help you evaluate your options or advise the best way to scope the work. A technical expert will often be your ally and best reality-check of not just what is possible, but what has been done.
Be prepared to step in and steer the conversation in ways that will help your team best evaluate the data system and partner who will make your performance measurement system successful.
Successful Partner Selection
In the end, we selected to use Salesforce as our performance measurement system and Exponent Partners as our implementation partner. How did we get there? Let’s break it down.
First, in our case it had only been 18 months since we had rolled out the (failed) data system. So I reviewed the prior team’s evaluation of vendors for the initial data system project. I learned that Salesforce came in a close second. We knew everything we wanted would be possible with Salesforce, but weren’t sure yet what was probable. I also had relationships with a few executives at nonprofits whose program teams utilize Salesforce for performance measurement and were happy with it.
As I looked at my options, Salesforce represented the future of nonprofit performance measurement. It had an open API and would give us access to a broad pool of talent and contractors. We were in a great position to pioneer a true outcomes-focused data system, and I wanted security and stability. Salesforce fit the bill.
Our next steps assumed we would move to a system built on Salesforce, and it was a matter of determining whether we would work with a partner to develop a custom solution for BGCP or utilize a “managed package” that would eliminate some of the risk with customization. If you’re wondering, what risk? Reach out me. (I was told by a veteran tech executive if a vendor said “customize” to me I should run in the opposite direction!).
Salesforce is somewhat unique in that its employees do not design and configure the platform for your organization, instead you must hire an “implementation partner” to get Salesforce to meet your needs. There are MANY potential implementation partners. There are large firms with a long track record of working with nonprofits, as well as independent contractors and start-ups. Finding the right fit is critical to your success.
How do you find a great implementation partner? Ask for recommendations. When I asked around my network, I learned that there were a few nonprofits who recently hired implementation partners to help them get started with Salesforce. I was specifically looking for recommendations for the type of data system we were building — a performance measurement system that enabled us to connect our activities to our outcomes.
These initial conversations to get recommendations helped in a few ways:
- Identified the strengths and limitations of Salesforce — the people I spoke with were not universally happy with Salesforce. I dug into what they were unhappy with and kept these issues in mind as we approached the design of our system.
- Surfaced alternative approaches to outcome tracking in Salesforce — the biggest ah-hah! was that nonprofits were sometimes entering aggregated or summary outcomes data in Salesforce. Knowing this, we could make sure the implementation partners we interviewed knew that we were interested in a more client-centered approach.
- Narrowed our list — there are so many options, and these conversations helped me narrow the list of potential implementation partners from 100 to 10.
Once we started getting recommendations for implementation partners, I started making calls. I had an initial call (no demo) to get a sense of fit. Since Salesforce “out of the box” is designed for — wait for it…sales — making the system intuitive and effective for performance measurement does take some imagination and expertise. If a partner only had experience configuring Salesforce for fundraising or volunteer management, I knew not to pursue that relationship further.
Of the 10 calls I made to recommended implementation partners, 4 had potential. I had a follow-up call with each of the 4 candidates individually. During these calls, I communicated our Phase 1 priorities, and I shared our Product Specifications Document. We discussed who would be attend the demo; an internal task force of program and operation leaders to help evaluate our options.
About a week or two later, we had 90-minute demo’s with potential partners.
The demo’s were telling. We knew we had some out of the box ideas for how this new system would need to be used. For example, we needed to be able to take attendance through scanning ID cards, as well as marking students present for a class or club. We wanted to know how an organization had done it or would do it. One vendor was able to show us a barcode solution they had used for tracking bicycles. Great! Not exactly our use case, but they had experience with integrating scanning functionality. One potential partner ignored that requirement entirely during the demo. Did we hold it against them? Yes, actually we did.
However, we did not expect potential partners to be perfect. In fact, providing feedback to potential partners after the demo, and seeing how they responded gave us as much information about their fit as the demo itself. In our case, Exponent Partners took our feedback and ended up recording a video of an enhanced solution. I liked what I saw and set up a second demo. What had been a lackluster first demo round, blew us all away the second time. It wasn’t just that they had more time to prepare a demo, either. The clincher was that they truly listened and responded to what we were asking of them.
So remember a few things as you select your partner:
- Ask your network for recommendations and use these conversations to better understand pros/cons of working with a platform or partner.
- When it comes to your top priorities, ask a partner to show you how they have done it or would do it. Don’t take “it’s possible” for an answer.
- Prep partners so they understand your priorities ahead of the demo. Don’t be side-tracked by irrelevant fun or flashy features.
- Keep your expectations high, yet be willing to give feedback so that a partner can put their best foot forward. After all, don’t expect someone to really understand what you, uniquely, need after a 30 minute call.
Follow these guidelines and you’ll be well on your way to designing a performance measurement system that meets your organization’s needs with a partner who will help you sleep (a little) better at night.
