Measuring Digital Skills in South Bend
A look at how we approached our first digital literacy survey, and the results
Much of our work in the Department of Innovation & Technology focuses on making technology as accessible and effective for the public as possible. This means expanding the availability of public internet, providing residents with computing devices, and creating digital resources for City services. However, these resources are not genuinely accessible if residents do not first have the digital skills to use them. Also, as they are increasingly essential for the workplace, social engagement, and civic participation, digital skills are critical to helping all members of the South Bend community thrive.
As we begin new initiatives to empower the public with technology, our team wants to understand where there might be gaps in residents’ digital skills, and how we might structure educational programs to close those gaps. In keeping with our team’s motto to “listen first and build with,” we launched a digital literacy survey to ensure we can build a future together in which technology fully empowers our city.
Learning from Previous Studies
Though digital skills are increasingly vital, they are not commonly measured at the community level.
Large-scale studies, like the National Center for Educational Statistics’ Program for the International Assessment of Adult Competencies (PIAAC), do a great job of estimating proficiency in the skills needed for full participation in society. In addition to literacy and numeracy, PIAAC measures competencies in digital problem solving at varying levels of complexity. PIAAC offers insights into nationwide trends in digital skills, and can capture demonstrated proficiency rather than expressed comfort levels.
In 2017, PIAAC found that 24% of respondents fell below the most basic level of digital proficiency, with respondents of color, older respondents, and those with less education seeing significantly lower levels of proficiency on average.
While PIAAC offers valuable data on demonstrated competency at a national level, our team turned to the City of Seattle’s Information Technology Department for a model of digital skills surveying at the local level. In 2018, the City of Seattle contacted 20,000 residents with an email or mailed survey packet requesting their participation in a technology access and adoption study. With over 4,300 responses, Seattle’s study dashboard provides a clear view of residents’ technology access, connectivity options, digital skills, and potential barriers to full digital participation.
In 2022, our team decided to launch a survey of our own to gain a similar view of digital skills in our community, looking to Seattle’s study as a blueprint.
Our Survey
Following Seattle’s lead, our team put together a brief survey asking residents about their source of home internet, device access, and expressed comfort completing 19 basic digital skills. Again following Seattle’s lead, we produced printed and Spanish versions of the survey to ensure accessibility to as many residents as possible and thanked participants by giving away 10 gift cards of $50 each.
In the summer of 2022, we shared the survey across social media, at community events, and via mailers that also promoted our home speed test campaign. To reach residents who might be less comfortable completing online forms, we put ballot boxes at each of our 8 St. Joe County Public Library branches in the city, as well as 3 popular community centers. Next to each ballot box, we placed two stacks of surveys — one English and one Spanish — and replenished the stock when necessary. This ballot box approach was responsible for a significant number of our total responses, and helped us leverage community partnerships to expand our reach in a cost-effective manner.
By the end of the summer, we had received over 300 responses and began analyzing our results, which we de-identified and shared on the South Bend Open Data Portal.
Results and Analysis
Our sample appears to be roughly representative of the South Bend community, with some points for caution in the key distributions of age, education, and race/ethnicity.
The 30–39 age range is over sampled in our survey, while the 60–69 and 80+ age ranges are under sampled. Given that older adults on average have lower digital skills proficiency, this suggests our citywide proficiency estimates may have an upward skew and that further analysis may help us more accurately understand the needs of older residents.
Furthermore, our respondents have, on average, higher levels of education than South Bend as a whole. This again suggests our citywide estimates may have an upward skew and that additional analysis may be needed to best understand the needs of residents with less education.
Finally, non-Hispanic white residents are over represented in our survey (60%) compared to South Bend as a whole (52%), while Hispanic/Latino residents are under represented in the survey (7.54%) compared to South Bend (15.8%) based on the 2022 American Community Survey 5-Year Estimates. Additional outreach and engagement in communities of color, particularly the Hispanic community, will need to be an important aspect of our team’s work to understand and bridge digital skills gaps in South Bend.
With these points for caution in mind, our data give us a first glimpse into the state of digital skills in our community with important insights for training programs, outreach, and advocacy. We have created a dashboard that summarizes these insights with 3 key sections: a South Bend summary, an overview of skills by proficiency level, and a distribution of proficiency level by age.
South Bend Summary
First, our South Bend summary reflects the demographics of our sample while providing a proficiency score distribution for each demographic group of interest. “Proficiency score” measures the percent of digital tasks which a respondent expressed at least “somewhat” comfort in completing.
As shown above, our survey found that South Bend saw an average proficiency score of 72.72%, meaning that, on average, residents were at least somewhat comfortable with about 14 of the 19 tasks we asked about. We also see that 33 respondents (10.8%) were comfortable with 1 or fewer digital tasks in our survey, while 118 respondents (38.7%) were comfortable with all or nearly all tasks in the survey.
On the dashboard, you can select a particular group of interest to see the demographic and proficiency score distribution for that group.
For instance, selecting “Bachelor’s” in the education distribution would show that respondents with bachelor’s degrees have a higher average proficiency level than the city as a whole at 82.5%.
Similarly, selecting “Black or African American” in the race/ethnicity distribution would show that Black residents see lower proficiency scores on average at 63.15%.
These insights, as well as others users can explore in the dashboard, suggest that though many South Bend residents are quite comfortable with digital tasks, there is a sizeable number of residents for whom these tasks remain a significant barrier. Moreover, this barrier seems to disproportionately affect South Bend’s Black community and residents with lower levels of education.
Most and Least Comfortable Skills
Turning to the second page of our insights dashboard, users can see the digital tasks in which residents expressed the most and least confidence in completing. Hovering over a particular bar will reveal the survey results for that skill.
While email and online searching stand out as very comfortable for a sizable majority of residents, many basic tasks see varying levels of comfort. Spreadsheets, presentation applications, and collaborative documents are least likely to be “very comfortable” for residents. Despite their increasing importance since the start of the pandemic, videoconferencing and online banking are among the tasks residents are most likely to be “very uncomfortable” with, and only 43% of residents are “very comfortable” using online government services.
These results can inform digital skills training programs in the community, and also suggest that maintaining analog and in-person communication alongside digital services within our government is critical to serving all of our residents.
Digital Skills and Age
In the final page of our digital skills dashboard, users can see the relationship between age and average proficiency score among our respondents. Again, hovering over a bar or point on the graph will show the relevant survey results.
In our survey, older residents saw lower proficiency scores on average, with stark decreases from the 40s age group to the 50s group and from the 70s age group to the 80s plus group. Again, however, it is important to note that the 60–69 and 80+ age ranges are under sampled in our survey, meaning that these results should be interpreted with some caution, and that additional analysis would be particularly valuable to understand the needs of older residents in digital access. Especially as an AARP Age-Friendly community, South Bend is committed to better connecting our older residents to help them more fully thrive.
What’s Next?
With these results in hand, our team is eager to begin bridging the digital skills gaps in our community while working to understand them better. We are currently partnering with organizations like the St. Joe County Public Library and La Casa de Amistad to promote digital skills trainings for residents, and hope to expand our partnerships in the near future as new state and federal funding is distributed to support digital equity work.
We also are mindful of the limitations of our results. As discussed previously, our sample is not completely representative of South Bend, and we can do more to engage with communities of color, those with less education, and older residents. Moreover, our survey captures expressed comfort in certain skills at a surface level; we did not capture demonstrated proficiency as PIAAC does, nor did we hear residents’ nuanced thoughts like we would in an interview. Again learning from Seattle, our team could leverage focus groups and follow-up studies to paint a more complete picture of our residents’ perspectives and needs.
This survey marks an important step in our work to listen first, build with, and empower our community members to thrive in the digital age.
