Testing grant application form questions
We wanted to learn about 3 questions from the £10,000 — £250,000 grant application form. We were particularly interested in how applicants understand them and how do assessing staff use them in their applications.
- Will capital work be part of your project?
- Does your project involve the acquisition of a building, land, or heritage items?
- Do you need permission from anyone else to do your project?
These questions are a vital part of this application form, but it felt like they could potentially overlap whcih would make it more difficult for people to answer them consistently or well.
We conducted a limited round of comprehension testing with previous and potential applicants to understand how people applying for funding may understand these questions and the differences between them. We spoke to 1 potential applicants and 2 applicant for up to 30 minutes.
We also conducted interviews with a limited number of investment staff to learn about the information staff need from these questions and how they factored the answers from these questions into their assessment. We spoke to 2 Investment managers and 1 Senior Investment Manager for up to 30 minutes.
We made a mistake when getting ready for this research
In the research we showed people the question:
Does your project involve the acquisition of a building, land, or heritage items?
Tell us the name of the building(s), collections, landscapes or habitat;Tell us:
If your organisation has the freehold of the building or land, or own outright the heritage items;
If your organisation has the lease of the building or land and how many years are left to run on the lease;
If your organisation has, or are you planning to take out, a mortgage or other loans secured on the building or land, or heritage item;
If so, give us details of the lender and the amount of the mortgage or loan.
This isn’t the right question — it’s an accidental blend of two questions from the £10,000 — £250,000 grant application form. In the current form, there are a series of questions which cover the same topics that we were looking at in the research. These questions rely on specific routing based on the answers that were given before.
When preparing for the research, we accidentally copied one questionwith hint text from one of the follow on questions.
This meant that the question and the hint text were talking about two different things which really confused the participants. The question was about buying a building, land or heritage items and the hint text was from the option where an organisation owns land, building or a heritage item that will be involved in the capital work.
“There are two separate questions — the bit at the top is fairly clear if asking are you getting new things, but latter bullet points are potentially separate.” — Participant 3 — Previous applicant
“That’s confusing, as if we’re expecting them to acquire it during the project then they won’t have that yet. It’s quite confusing, it seems to relate to two different things. That’s what it looks like to me, it looks like two different questions.” — Participant 5- Senior Investment Manager
This mistake means we didn’t learn as much as we needed to about either the acquisition question or the ownership question.
What did we learn about
Even with this mistake, we did learn about some useful things that will help us improve these questions.
People don’t know what capital work means
In the research, we asked people about the question:
Will capital work be part of your project?
By capital work we mean repair, conservation, new build, digitisation, or work to stabilise the condition of objects.
Examples of capital work:
- conservation of a heathland
- repairs to a war memorial
- digitisation of a photographic archive
Most people understood this question to mean, “are you doing any building work?”. This applied to staff and applicants.
Capital work — something that is physical, building with ‘a significance of sorts’ maybe renovations/repairs, and memorials. “Bricks and mortar” — Participant 1 — Potential applicant
“To me capital is buildings or objects, physical items not digital items” — Participant 3 — Previous applicant
Digitisation being included in the question as an example surprised people. They wouldn’t have considered digital work to be something they’d need to tell us about in this question.
“Wouldn’t have occurred to me that digitisation of a photographic archive is capital works. I guess you’re producing something tangible, but…” — Participant 2 — Previous applicant
Staff also talked about digital work not being commonly thought of as “capital work”. They talked about this being used for more commonly physical projects.
This is something we’ve seen before and had worked to improve with a bigger range of examples, but we still need to do more.
If neither staff nor applicants understand what we mean by “capital work”, we need to do more to make this easier to understand what it means. Until we start doing the hard work to make this simple, we’ll continue to see people not classifying things as capital work when they should be.
A lot of our examples and language links to tangible heritage and not intangible heritage
The questions we tested should apply to a range of projects — not just those with a building element or a physical heritage item, but most people interpretted them as being about those more tangible projects.
This applied to the language of the questions and the hint text that went with the questions.
If the application form is going to be suitable as the Fund continues its move towards supporting more intangible heritage, the questions and examples given should reflect that.
“Examples feel very specific to buildings” — Participant 3 — Previous applicant
“Examples photographic as the only non built example there” — Participant 7 — Investment Manager
Staff need a narrative explanation of the situation
A number of members of staff talked about needing there to be some explanation of the specific project’s situation. Every project is different, so a yes/no/not sure answer isn’t helpful for assessment. There should be space for the applicant to give specific details about the circumstances of their project.
“I’d want narrative against that, unless there was a follow up question where they describe it. Maybe expand on ‘describe your project’ to guide them to describe capital works explicitly. Yes/no isn’t particularly helpful by itself, more helpful with a narrative.” — Participant 5 — Senior Investment Manager
This applies to questions capital works, acquisition, ownership and permissions.
What’s happening next
Now that we’ve done the research, we have a better understanding of what staff need from these questions. This helps us make sure the design for these questions captures the information we need.
For now, we’ll be building the questions in a fairly familiar pattern so that we can see how those work in the new service.
In the future, we’ll be conducting some more analysis looking at how people complete this questions in the current form and the answers on the current system. This helps us gain a wider picture of how we’ll they’re working now.
Having built the question in the new service will mean that we’ll also have some analytics tools available to us that we can use to see how people are completing the form. We can combine these analytics with our analysis of the answers people gave in the current system to build a rich picture of how these questions work for applicants and staff.
