Setting a ResearchKit & ResearchStack Budget
There’s an app for that. Are you thinking about developing an app for your research study? First, you’ve got to create a budget. ResearchKit (and ResearchStack) can be expensive. It is these costs (and issues) that lead us to develop ProofPilot when we were consultants.
Apps have created efficiencies and solved problems in just about every industry you can think of. While researchers have adopted electronic data capture, the full-featured, high-concept solutions innovating other industries have been a little slower to appear in the human subject research space.
But, in March 2015, Apple introduced ResearchKit. The software development kit, from the world’s most valuable company, legitimized consumer-facing technology in the stodgy research world.
So now the question is: How much does it cost for you to create your research app? What kind of budget do you need to set aside in your next research grant proposal? Are there alternatives that don’t require you to be a software development project manager?
Most researchers, particularly those in the medical and social sciences are not experts in managing software development. While this article won’t tell you specifically how much your app will cost (every study is unique), it should give you a set of realistic expectations.
Initial ResearchKit & ResearchStack Budgets: Development
Getting a cost estimate from a contractor. Unless you are a software developer yourself (and have the time to devote to development), you’re going to need to find contractors or technical employees. To get an estimate of how much to budget, it would make sense to detail the scope of work for the developers and ask for a quote and timeline.
But, be careful. Working with software developers is a lot like working with homebuilders. Software development cost and time estimates are notoriously wrong — often by a factor of two or three. So, it’s worth going into a discussion with a contractor with your eyes open.
Developing an app with a barebones budget is possible. But that app has to stand up to the thousands of others on the app stores. It has to establish credibility and safety so your participants will share sensitive data with you.
To get a sense of how much a more credible app costs to develop, let’s look at the potential development costs for an Instagram clone. Most of us are familiar with the remarkably simple photo-sharing app. You log in, review, and upload photos. That’s it.
How much should an app cost?: So let’s say you went to a development firm for an estimate. They say it would cost them about $70k to develop something like Instagram under contract.
But, don’t forget, the developer’s estimates is almost always overly optimistic. And just because they offer a fixed price contract, it doesn’t eliminate your risk. Small, hungry development firms that take on contracts at super low costs to get the client can quickly get overwhelmed and go under.
On The NextWeb, a leading tech industry website, venture funders and developers more realistically estimated that the tech development costs for Instagram to be in the low six figures. In fact, a massive review of app development costs that went viral a couple of years ago puts that $70k on the very low end of the app development cost spectrum. Apps written by contractors with the experience and resources necessary in healthcare will likely charge anywhere between $100k to $450k.
Costing the certainty of change: Lowball costs assume that specifications during development don’t change much — this is an unrealistic assumption. Almost 70% of software development efforts fail, in large part because of unclear or changing requirements. With a moderately complex app taking several months to build, you should expect risks to increase including the likelihood of delays, extra development time, less than ideal product — and at worst a never completed effort.
As the study and its protocol changes, the app will have to change too. Do you require an IRB review? Does your organization require a security audit? That kind of back and forth can easily add significant business risk double costs and additional resources.
Ensuring time and resources for testing and quality assurance. Many lowball development estimates assume that development will complete without a mistake or hitch. You’ll be able to launch your app and everything will work correctly. That NEVER happens.
A user notes an important bug in a ResearchKit app review.
Good development firms will including testing within their estimates. If they don’t, testing and quality assurance will add additional costs to your budget. Even something obvious like eligibility criteria (as you see in this review of a ResearchKit app) can escape an insufficient testing process and cause credibility issues in the long run.
Costing Requirements beyond the basics. All this and most research studies are a lot more complicated than a “first iteration” Instagram. They require randomization, appropriate security, unique data collection and processing. Many need to be available in multiple languages to reach target populations. They need to be available on multiple platforms.
Instagram released an Android version a year and a half after their first iPhone version. You’ll probably want a web version of your application for participants and study professionals who don’t have smart-phones, and an Android version on Day 1 … the costs increase.
Ongoing ResearchKit & ResearchStack Costs
Once the application is developed, there are still additional costs you need to budget for.
Hosting and maintenance costs. You have to store the data for that study on servers somewhere. If you have a web version, you have to host the web experience. You’ll want the experience to be speedy. The unique regulatory, privacy and security issues mean that the average server isn’t going to be sufficient. Encryption is required. You’ll need a location that has adequate physical security. You’ll want backups and fault tolerance.
Buying a server and putting it in your closet probably won’t cut it. But thankfully cloud hosting providers like Amazon Web Services have brought costs crashing down over the past five years. Are cloud servers secure? Absolutely. The CIA has a contract with Amazon to host its most sensitive data. But you’ll still probably need the expertise to ensure HIPAA compliance. Online services like Aptible can help with that.
Despite dropping costs over the past decade, it is still probably going to cost $1,000 to $1,500 a month to host your services. If your study attracts hundreds of thousands of participants, watch the costs grow. Some organization (like Salesforce.com) offer free hosting to non-profits. But you’ll still need a systems administrator to manage everything. And those free levels available to non-profits may not provide all of the features and functionality you need.
Other ResearchKit Expenses to Consider
So far, we’ve only discussed the development and hosting of a participant facing application using something like ResearchKit. But that’s only one component of your research study.
Additional components to support the ResearchKit app. Do you want a web administrative experience to manage participants, view simple reports and download data on your own terms? Do you want a designer to provide an experience that participants remember? What about email, and text messages to remind participants to engage in your study? They all contribute to the initial development and ongoing management costs.
Sending email isn’t free. For a modest-sized study, you’ll probably spend between $250 and $500 per year for an e-mail system like SendGrid or Mandrill (part of the MailChimp family). Likewise, text messages via service like Nexmo or Twilio will run a couple hundred dollars a year for a modest study.
Designers are nowhere near as expensive as developers, and developer kits like ResearchKit have a default look. However, remember you are competing with hundreds of thousands of other apps on the app store. You are competing for participants attention and loyalty. It’s likely you’ll want to give them an experience they’ll remember.
Estimating costs for a ReserachKit or ResearchStack app based on existing programs and RFPs
The reward for the Robert Wood Johnson Foundation’s ResearchKit Mood Challenge is $120,000 for the app design and prototype phase. For one finalist, they award an additional $200,000 to continue development and submit to the app store. The average National Institute of Health R01 award (the largest and most prestigious NIH reward grant) was $427,000 in 2014. That means a mobile app would eat up three-quarters of the budget even before the study started.
The Michael J Fox Foundation collaborated with Sage BioNetworks in the now famous Parkinson’s ResearchKit app. In an article for BusinessWeek, CEO Todd Sherer said he did not know how much the ResearchKit app cost to build. But the biomarker study as a whole that the app is part of has a budget of about $60 million. That’s many times the largest NIH grants that fund most academic researchers.
And Your Normal Research Costs
While this app you’re developing may help you reach out to a broader group of participants, it may not reduce your traditional costs all that much unless you consider adding marketing and logistics expertise to your the app design process (which incurs additional development costs).
In fact, you may be exacerbating your recruitment process. Yes, it’s true, tens of thousands of people registered for the first Parkinson’s ResearchKit app when it launched in 2015. It got a major push from Tim Cook during one of Apple’s extraordinary launch events. To date, there are more than 1,000 media mentions on Google News. Apple’s PR strategy is so legendary that it’s the subject of Harvard Business School case studies.
But, Apple isn’t able to announce every ResearchKit App. The venture community warns startups that “Build it and they will come,” is a fallacy. Researchers should heed this warning as well. Half of U.S. smartphone users download zero apps per month. This means you’ve just exacerbated your recruitment problem. Even if participants do download the app initially, you’ve got to keep your participants using it. The average app looses 77 percent of its users in 3 days. You’ve exacerbated your retention problem.
Less Expensive Alternatives to ResearchKit and ResearchStack Apps
But don’t despair. There are alternatives to building your own mobile research application that are far less expensive nor anywhere near as risky.
Partner with an existing app. There are thousands of digital health apps on the app store. The University of California San Francisco partnered with Argus’s Heart Rate Monitor app to pool heart rate information into a heart health study.
Use a Software as a Service Solution. Tools like ProofPilot and TrialX allow non-technical individuals to create an online study. Use their web-based solution to design your study and then launch to participants.
Access data shared by participants. Projects like OpenHuman allow participants to share data with researchers from other studies, like their sequenced genome, and consumer tools like RunKeeper. You can pool data from these different solutions and analyze to get findings.
Summary
The reality is the cost of developing and managing even a simple app, approaching $300k, is outside the budget of all but the most well-funded studies. App development costs could eat up more than three-quarters of an average NIH R01 Grant. But with your eyes open, you can budget accordingly, and consider potential less costly alternatives.
Originally published at www.proofpilot.com on September 12, 2018.
