Lessons for the Field: How to Sustain Proven Programs that Drive Student Success

If we, the higher education field, are fully committed to the attainment agenda, and I believe we are, why are programs that positively impact student retention and completion rarely sustained? How can we follow the path to continue and expand programs that we know are effective? This blog explores these questions by diving into two programs: ASAP Ohio and the SUCCESS Initiative supported with technical assistance from MDRC.

ASAP: Expansion of an Evidence-Based Practice

After MDRC’s rigorous analysis of the Accelerated Study in Associated Programs (ASAP) at City University of New York (CUNY) demonstrated the program’s effectiveness, the New York-based social policy research organization collaborated with CUNY and the Ohio Department of Higher Education in 2014 to provide technical assistance and evaluate ASAP’s replication at three community colleges in Ohio.

Again, MDRC’s evaluation of ASAP models in Ohio showed positive effects at each of the three participating colleges:

• Three-year graduation rates basically doubled
• Positive effects were seen in enrollment patterns
• Positive effects were seen in credit accumulation

And just like CUNY ASAP, while the per student cost of the program was higher than average, the cost per degree earned was actually lower. In a state that provides funding to its community colleges through an outcomes based funding formula incenting student persistence and completion, one might expect these initially grant-funded programs to continue and even scale across other Ohio community colleges. However, only Lorain County Community College (LCCC) in the greater Cleveland area sustained its ASAP-based program (called SAIL for Students Accelerating in Learning).

SUCCESS: Rolling Out a New Program

In 2019, MDRC engaged nine community colleges across four states (California, Indiana, New Jersey, and Ohio) in a new effort Scaling Up College Completion Efforts for Student Success (SUCCESS), which sought to “improve graduation rates for traditionally underserved students at colleges, by helping states and institutions align their resources with evidence-driven practices.” SUCCESS builds on the research demonstrating that comprehensive student support programs can dramatically increase student success. In addition to ASAP, examples of such programs also include One Million Degrees and the Dell Scholars program.

Participating community colleges collaborated with MDRC to tailor unique programs built around four components: proactive advising, financial incentives, accelerating academic momentum, and the use of real-time data on student progress.

These programs rolled out in Fall 2019 and soon thereafter ran up against the impacts of the COVID pandemic, requiring immediate programmatic adjustments, such as modifications to planned coaching sessions to help students navigate the transition to remote learning and to connect students with local basic needs resources and emergency aid.

New Jersey was one of the first states to feel the full brunt of the pandemic in 2020. Stefani Thachik, Director of Policy and Outreach with the state’s higher education office, describes how integral the SUCCESS program was to supporting students early on:

“As students faced increased challenges, the access to dedicated coaches and ability to receive financial incentives, were likely a lifesaver for many students during the tumultuous time. In particular, both SUCCESS colleges took the initiative to find and provide material hardship support, such as addressing food insecurity or lack of access to technology, to students in the program.”

Beginning in Spring 2020, colleges temporarily adapted their programs to respond to practical challenges resulting from the pandemic, including moving to fully virtual services and providing more flexibility to students around requirements, given the unprecedented context. The study will shed light on whether these adapted programs were able to increase academic progress during the pandemic, as well as their effects when implemented fully, during post-pandemic times.

The SUCCESS colleges have committed to serving students in the program through 2024 as part of MDRC’s study. They are also beginning the process of making plans to expand the program to additional students and financially sustain the program beyond the study period. When that time comes, SUCCESS colleges will face a choice between continuing an evidence-based program that initially shows promise, or ending the program and, very likely, undertaking a new grant-funded initiative.

Perhaps Lorain County Community College’s SAIL program can provide some insight for how to sustain effective programs. Marisa Vernon White, Ph.D, the college’s Vice President for Enrollment Management and Student Services shares:

“Although designed as a pilot program, we never viewed it that way. Rather, we approached implementation with the long-term view of sustainability and an opportunity for us to redesign our services guided by the goals of student success and a culture of care. SAIL was an opportunity for overall service redesign, this motivated us to find system-wide efficiencies, opportunities for cost savings and process streamlining, and encouraged us to combine previously isolated funding sources in a way that both sustained SAIL beyond the pilot and strengthened student success efforts overall.”

LCCC began their ASAP program with the goal of sustainability front and center. Campus leadership recognized the initial grant funding provided the opportunity to critically assess current operations and only continue those supports and services aligned with student success. The evaluation findings demonstrating the large positive impact on student success were used to inform and justify their scale and sustainability efforts. More colleges should approach their pilot projects with a similar mindset. CUNY has also been remarkably successful at scaling and sustaining ASAP within New York City. Their program now serves about 25,000 CUNY students annually.

Call to Action for the Field

How can institution leadership, staff and faculty, technical assistance providers, state policy makers, and philanthropic partners more effectively help pilot programs that work transition into ongoing (and funded) endeavors? Each of these constituent groups has a responsibility and critical role to play to ensure programs proven effective through rigorous evaluation continue to operate:

• Institutional leadership, staff, and faculty can commit to student success and, by doing so, be willing to discontinue ineffective practices, freeing up resources for sustaining those that work and transitioning initially grant-funded initiatives into base-funded operations. In conversations with policy makers, institutional leadership can leverage the efficiency and ROI arguments for additional resources to support the scaling up of effective programs.
• Technical assistance providers can focus on sustainability from the onset of any grant-funded initiative helping pilot institution staff engage in case making with their leadership and sharing best practices from prior projects (see Lorain County Community College). Further, TA organizations and successful programs should continue engagement beyond the grant period. Consider what the CUNY ASAP team has done to help other colleges across the country implement their model.
• State policy makers should, quite simply, provide targeted resources to keep successful programs going beyond the pilot phase, along with financial support for expansion and scaling across more institutions of higher education in their state.
• Philanthropic organizations should set an expectation that grant recipients take the necessary steps towards sustainability and, like TA providers, provide guidance for how to do so. However, funders also should be willing to continue supporting effective programs beyond the grant period in cases where additional resources are still needed to transition into a long-running, sustained program.

The COVID pandemic and its inequitable effect on college students has highlighted the importance of wraparound supports and services which are a key component of the SUCCESS program model. I am optimistic the SUCCESS colleges will find ways to sustain these programs, and, like ASAP at CUNY and in Ohio, become a go-to and oft-cited best practice for supporting students.

— — — — — — — — — — — — — — — — — — — — — — — — — -

In an exciting development, while writing this blog, another Ohio ASAP institution, Cincinnati State, reconstituted its program (C-State Accelerate). Through strong leadership from its president, the program is now embedded within the college’s strategic plan and a key student success program to help students graduate and reduce generational poverty. It is now viewed among college leadership and staff as an ongoing program, as opposed to a grant-funded pilot initiative.

I want to express my thanks to Katie Beal and Melissa Wavelet with MDRC for their guidance and assistance with this blog post. Further, thank you to Stefani Thachik with the New Jersey Office of the Secretary of Higher Education and Alison Musser with Lorain County Community College for their assistance. It was an honor to highlight the great work these individuals and organizations do for student success.