Fostering Collaborative STEM Learning: Project with Citizens Schools
by Kristin Stoeffler and Yigal Rosen
ACTNext + Citizen Schools
A major challenge in the rise of interest in cultivating higher-order skills in the classroom is the ability to determine whether the programs and projects that are being implemented to engage and improve these skills are, in fact, achieving their intended effects. Are these programs really engaging and improving students’ skills in critical college and career readiness areas like collaborative problem solving, creativity, and critical thinking?
ACT is partnering with Citizen Schools to explore this very question. Building on its evidence-based afterschool program, Citizen Schools developed a new model, named Catalyst, to provide science teachers in 6th — 9th grade with the curricular tools and additional supports necessary to integrate high-quality project-based learning experiences for students that include exposure to STEM professionals from their local communities.
The intention of Catalyst is to help adolescents in underserved communities develop the skills, mind-sets and networks they need to thrive in the rapidly changing 21st century economy. Also, the Next Generation Science Standards (NGSS) call for instruction and assessment to be focused on the integration of cross-cutting concepts, science and engineering practices, and core ideas — precisely the integrated skills and mindsets that students need for success in the 21st century workforce — and the emphasis on “doing science” in the NGSS also creates an unprecedented opportunity to meaningfully engage STEM volunteers in classrooms and schools implementing these standards.
Feedback from Catalyst teachers and STEM volunteers through the two-year pilot phase was quite positive, with nearly every individual expressing interest in continuing to participate in Catalyst in the future. For example, in the Catalyst Thermal Energy unit, students engineer a heating and insulation system for a model building and research and conduct interviews with local builders and heating experts to understand the environmental, social, and financials costs and benefits of heating systems.
One Catalyst teacher implementing the Thermal Energy unit shared, “Our Catalyst volunteer, from the U.S. Environmental Protection Agency, was able to captivate the students and lead them in a discussion about differences in temperature and why they may be experiencing unpredictable results. She visited several times throughout the project and ultimately played a huge part in engaging the students and having them eager to learn more about thermal energy and how it impacts their community!”
Since the 2017–18 school year, ACT and Citizen Schools have iterated through two interactive and engaging game-based assessments of students’ collaborative problem-solving with the second used by students in Citizen Schools’ classrooms this spring in New York City and North Carolina.
A core component of the Catalyst model and Theory of Change is the utilization of actionable assessment data by Catalyst teachers to measure student growth on critical skills including collaborative problem-solving. We are excited to partner with the ACTNext team on this innovative assessment project to better understand student skills and mindsets before, during, and after their Catalyst classroom experience as we continue to refine the Catalyst model and expand to reach many more schools and classrooms nationally in the years to come.
– Amy Hoffmaster, Managing Director of Catalyst
ACT had developed games and adaptive scenario-based tasks to measure skills like collaborative problem solving so it made sense that a collaboration between ACT and Citizen Schools could lend some exciting insights for both groups and to advance the field as well.
When we talked with Citizen Schools about the questions that they were trying to answer, and the innovative Catalyst STEM education program that they are developing in the classrooms, we knew right away that this would be an exciting and insightful collaboration.
– Yigal Rosen
For the most recent collaboration ACTNext and Citizen Schools explored current design team work occurring in the Catalyst classrooms. The team-based environments in Catalyst classrooms provided a great foundation to focus on real-world collaborative problem solving to solve real-world problems. This provided a great opportunity for ACT to provide support through evidence-based and actionable student assessments for middle school students.
“We are building new tools to foster and measure higher-order skills such as creative thinking, collaborative problem solving, and computational thinking, and had been exploring a range of scenarios to use for the new tasks. From the learning science standpoint and the opportunities in K-12 setting, these learning solutions work best when coupled with well-defined domains such as math, science, engineering and social sciences,” noted Yigal Rosen, Sr. Director of Learning Solutions at ACTNext, “It just made perfect sense to collaborate with Citizen Schools’ Catalyst STEM initiative to gain insights about ways to foster these critical skills through innovative learning and formative assessment technologies.“
This line of research and development at ACTNext builds on work in the field of collaborative learning in science that was highlighted in a recently published scholarly journal article in Computers in Human Behavior.
Science Fair: Design Challenge
What came out of the most recent collaboration is a learning and assessment tool called “Science Fair: Design Challenge” which uses a chat-based environment to guide students through some of the more challenging scenarios involved in working in a design team. The environment includes multiple ‘agents,’ or chat participants, that along with the student participant are members of a fictional design team. The fictional design team includes both peers and an expert advisor, with the majority of the chat occurring between the peers themselves. The tasks explore both the social and cognitive skills involved in collaborative problem solving. To do this the tasks make use of a range of innovative, technology-based learning and assessment tools including: embedded automated scoring, skill weighting, and response scoring, as well as the use of a range of multimedia elements (e.g., video, PDFs, images).
The focus is on the reliability and validity of the insights, noted Dr. Rosen, “We wanted to make sure that we were controlling as much of our technical ability to gather information as possible in order to provide evidence of skill while still keeping the environment as authentic for the students as possible.”
The chat environment is designed to allow for a range of response types including multiple choice and open response types. “The really great thing about this assessment and learning tool is that it is also designed to provide support and feedback to students who are demonstrating lower levels of skill. Since the game takes place through response selections made by the student in a group chat environment we can provide real-time feedback to students that are choosing responses that reflect a lower level of skill. The feedback can come from peers or the advisor and can involve a range of media that might correct a misconception or support a better understanding of the skill in that scenario.” noted Kristin Stoeffler, Lead Designer for the project and Sr. Learning Solutions Designer at ACTNext. “The insights provided by the team at Citizen Schools allowed us to really hone in on the actual design team scenarios that were challenging for students and explore the demonstration of the skills in those scenarios. We also measure those skills across other scenarios to better understand their demonstration overall.”
The reporting features for the environment include the option of real-time automated scoring for students and teachers, as well as the incorporation of multimedia elements designed to enhance the understanding of those scores. Insights gleaned from these interactions are intended to support learners in their understanding of the range of skills required for these interactions as well as teachers’ insights about the performance of students and classes in these skill areas.
Dr. Rosen explained that the partnership and development are intended to move the field closer to the larger goals of cultivating students’ higher order skills, “Ideally, we hope to get to a point where we can provide actionable insights to both students and teachers about these skills and the value of the programs that are being implemented to cultivate them in the classroom. Ultimately, learning is a journey and this partnership is a great way for us to support learners and the teachers who guide them on their unique pathways to success.”
