Future Skills — The Superpowers Needed to Save the World. In my last blog, I shared the 21st century skills framework that we developed for the World Bank Evoke project and summarized the research that led to this framework as detailed in the World Bank technical paper — Evoke: Developing Skills in Youth to Solve the World’s Most Complex Problems: The Social Innovators’ Framework which argues that complex problem solving — specifically of the world’s global grand challenges (hunger, clean water, illiteracy, peace, etc.) and the associated skills involved in social innovation provide an essential skill set for youth around the world.
How does Evoke develop these skills? Evoke is a project-based learning model that uses storytelling, game mechanics, and global social networks, to imbue young people with the skills they need to develop social innovations that address grand challenges (e.g., displacement, hunger, poverty) in their communities.
The Evoke project is designed to support young people as they develop an understanding of these complex challenges, acquire 21st century skills (e.g., creativity, collaboration, critical reflection), socio-emotional skills (e.g., curiosity, empathy, generosity), and gain the confidence to experiment, collaborate, and create innovative solutions.
In total, Evoke aims to cultivate 16 strengths and 48 specific skills. The 48 skills are each translated into engaging and interesting activities that capture the essence of the skills and that can be demonstrated by students through their actions. In the Evoke project, a specific activity is designed to represent (operationalize) a specific skill or competency, though many activities require a mix of multiple skills to complete. The activities vary in nature; some are intended to be conducted in the real-world through field work, others can be conducted online and through research.
Evaluating Impact in Soacha, Colombia. This blog is a summary of the World Bank technical paper “Evoke: Developing Skills in Youth to Solve the World’s Most Complex Problems: Randomized Impact Evaluation Findings”. Evoke was implemented and evaluated over one semester with 297 students at the Soacha campus of Uniminuto University on the outskirts of Bogota. The theme of this Evoke module was peace and displaced persons. Soacha is a community of thousands of people violently displaced by the conflict in Colombia. A randomized control trial with a pre-test post-test control group design was implemented with random assignment to treatment and control at the classroom level (students however selected their class at random largely based on their schedules). Two control groups were assigned — control 1 is a Social Responsibility class and the main control group while control 2 is a Contemporary Social Development class which is infused with social responsibility concepts but does not engage in community level field work. Below is a summary of the student participants:
Control 1 is the Social Responsibility condition and Control 2 is the Contemporary Social Development condition.
** Blanco is Caucasian or White participants
** Includes Negro/Afro Colombian, Indigenous, Not stated
Three Questions. The randomized evaluation addressed three key research questions:
Research Question 1: To what extent does Evoke improve participants’ social innovator 21st century and socio-emotional skills as compared to the control groups?
Research Question 2: In what ways does Evoke shift participants’ perceptions toward believing they can help bring about a more hopeful and peaceful future through social innovation as compared to the control groups?
Research Question 3: What are the relationships between engagement with the Evoke project and participants’ learning outcomes, perceptions and personal characteristics (gender, ethnicity, age, and major area of study)?
To what extent does Evoke improve participants’ social innovator 21st century and socio-emotional skills as compared to the control groups?
During the semester, Evoke students demonstrated statistically significant greater learning outcomes in 21st century and socio-emotional skills which enabled them to be more innovative, empathetic, collaborative, and better problem solvers. The effect size on skills acquisition was medium, which is strong for a brand new program when compared with two well-established programs, and especially strong when considering that this was not a small supplemental intervention but a whole program that replaced five existing university classes for an entire semester.
In what ways does Evoke shift participants’ perceptions toward believing they can help bring about a more hopeful and peaceful future through social innovation as compared to the control groups?
After participating in the experiment, the Evoke treatment group had a stronger perception of their capacity to bring about a more hopeful and peaceful future for people who have been violently displaced by conflict than did the control groups, although this finding was not statistically significant. In the Evoke program, students have the opportunity to apply the skills they are learning in the community, and the findings suggest that this may have been enough to start the process of changing perceptions toward students believing that their voices and actions matter. Note, however that students in the Control 1 group also did fieldwork and while their perceptions also improved, it was not to the same extent.
What are the relationships between engagement with the Evoke project and participants’ learning outcomes, perceptions and personal characteristics (gender, ethnicity, age, and major)?
The Evoke groups skills and perceptions of their capacity to bring about a more hopeful and peaceful future for people who have been displaced improved equally regardless of gender, ethnicity, academic field of study or age. This suggests that Evoke has the potential to improve students 21st century and socio emotional skills and empower young people from diverse backgrounds to become agents of change.
Another finding is that the analysis showed a statistically significant relationship between students’ belief in their capacity to bring about a more hopeful and peaceful future and engagement with particular Evoke activities, as a creative visionary (imagination, ideation, vision, courage) and as an empathetic activist (leadership, empathy, transformation, curiosity). This is an interesting finding because it suggests a connection between engagement with instructional activities and change in perceptions.
Moreover, the findings from the correlational analysis lend credence to the idea that requiring students to operationalize their skills through real-world activities may be the beginning of the process that leads to students’ increased belief in their abilities to effect change
More Questions. As with any intervention and evaluation, there are always more questions and additional areas of explore and understand. Additional questions and areas of research that we hope to explore include:
- What is the impact of Evoke on student retention and graduation?
- What is the impact of Evoke on students’ ability to get a job and are employers satisfied with their skill set?
- What is impact of Evoke student projects in the communities?
- What is the link between learning activities and skill acquisition — i.e. are certain activities more effective in developing creativity for instance?
- What is the impact of behavioral triggers such as future thinking, provision of resources, and social capital on students choices and outlook on their future prospects?
More data will be available in the coming months. The World Bank and World Vision (All Children Reading) recently signed an MOU to design and pilot an Evoke campaign focused on Literacy — see trailer. The pilot was launched in the Fall in Limpopo, South Africa. The evaluation is expected to be available in the coming months and Ill share it once available.