Reflections on GenAI in Education so far: from navigating prompt craft to sector-wide disruption
By Daniel Emmerson, Academic Affairs Lead at Goodnotes
For the last four months, Goodnotes has been conducting independent research concerning the impact of generative artificial intelligence (GenAI) on education, with a particular focus on assessment, homework and coursework in schools around the world. There are over 50 participants from more than 20 countries across the globe who are taking part in this work, which has inevitably given us access to a broad range of educational areas, from best practice for teachers to helpful tips for students. This article will highlight some of the key findings from one of our working groups, which brings together students from ten countries. These young people are either in their final two years of high school or their first two years of university, and they all have different access to resources in their learning communities.
This group is of particular interest to us because they represent a unique demographic of those who will be able to recall teaching and learning before GenAI becomes mainstream, but who are also well versed enough in the technology that they are able to understand how it works and why it has the potential to create such a positive impact from a learner perspective. They understand the immediate impact of changes to how teaching and learning takes place at schools and they are able to guide our way of thinking about the future, while also providing insights relating to emerging trends and patterns. These initial reflections are by no means comprehensive, but they provide pertinent insight as to how GenAI is already impacting the education landscape, while they also offer ideas relating to the challenges that we are seeing in the media on an almost daily basis.
1. “The prevalence of ChatGPT has forced through the changes that we needed to see in assessment and this can only be a good thing — sometimes it takes utter disruption in order to implement the changes that we need.”
Whenever we speak with schools and students about the changes that GenAI is bringing to education, it is generally agreed that assessment needs to change and that current methods of addressing student knowledge and capability are antiquated, unless we need to be assessing the capability of a student to sit down for two hours straight and recite facts and figures from a textbook. An argument in support of traditional methods is that focus is a key attribute that will be required in the workforce of tomorrow because of how much of a distraction-centric world we live in. Everywhere we look there are apps trying to help us maximise productivity, just as much as there are games, adverts, promotions and notifications that have the potential to utterly leach our attention while we attempt to concentrate. Students feel the same way about their working environment and although there are varying degrees of control during regular school hours, exams should be happening in exam conditions to offer fairness across the board and to enable students to concentrate on a specific subject, set of problems or themes for a 60–120 minute period. Due to the interruption-centric world that we currently live in, that level of concentration has become an increasingly tall order, and if a student is able to do that, then this demonstrates a skill that is going to be more highly regarded in the future. Indeed, our students are often thankful for the few opportunities they have to demonstrate their knowledge in this way. Not only because this is done with a level playing field for everyone, but because it demands that they focus on one area of work without all of the distractions that continue to bombard them in day to day life. This won’t be the case for everyone of course, but if or when the powers that be decide to make changes to standardised assessment, this needs to be a consideration.
Having said that, assessment can take on many forms, and several of our students are fully embracing the swift alterations that text generating AI has already accelerated. When GenAI started to become more widely used in student communities at the beginning of 2023, there were some teachers who started to take notice and alter the way that they were doing things. Teachers in the southern hemisphere had an advantage as this happened during their summer holiday and they could experiment with new ideas and approaches throughout the break. By the time some students returned to classes in late February, they had already noticed some of the changes to how lessons were conducted. In some instances, project based assessments and homework had been reconsidered so that open GenAI was integrated into what teachers were setting and also what they were grading — more on that later. In addition, students were given the freedom to go multidisciplinary, with new rubrics for assessment that involved concentration, reading, researching and understanding subjects from a fresh perspective. This meant moving away from writing long form answers to questions that required a test in memory and a vague interpretation of a subject, to using multiple forms of creative media and conveying insight, reflection and knowledge through a versatile number of artefacts. An apparent consequence of this shift in approach has seen students truly engage with their work, not because they want to get the best grade, but because they have an increased degree of autonomy and they want to make something that they are proud of.
2. “Guidance and support from human teachers with a cultural awareness of the local environment is crucial in things like history and geography or even English language, where AI gives very standard responses that are typically grounded in a specific cultural narrative that is not universal”
Students have talked through the variations of using GenAI for various subjects, and are already coming across problems that they envisage have the potential to be rectified as the technology develops. For now, these tools need to be given more consideration by the people responsible for them. Although there are well documented examples of how GenAI has not been able to solve certain equations or recall formulas, it is generally believed to be more beneficial in tasks that require ethically neutral standpoints. Examples of where this has gone wrong for students relate directly to humanities subjects in particular, or when using GenAI to plan for a presentation or a debate. Regardless of what types of prompt might be used (such as, write this from the perspective of Emperor Cheng of Han), the cultural assumptions made by the most mainstream text and image generating AI requires the need for more in-class discussion and debate relating to biases and cultural prejudice.
Although this has brought a fresh perspective on how debates in class might work and how students are able to analyse global phenomena from nuanced and diverse perspectives, the focus has often pivoted to how GenAI might not be considering the subject matter from the point of view specified, but from the western interpretation, which leads to interesting conversations about how we know what we know and why. When thinking about potential solutions from the perspective of users and school communities in the future, having more input from industry when it comes to media literacy and teacher training will be more crucial than ever, particularly if understanding multiple perspectives. Suggested solutions for the tech world developing the GenAI responsible for these biases are to reevaluate data sources and to invest in the diversification of personnel responsible for both input and output.
3. “Teachers are still setting homework where students have to use a text generating AI. The difference is that students are being marked on their prompt craft in order to achieve a set result”
In many cases, teachers are still setting homework, but with an open awareness concerning the tools that are available to their learners. The fact that this appears to be happening more in Southern Hemisphere countries (Latin America in the case of our research) is an indication that having those extra couple of months to play with GenAI at the beginning of 2023 truly helped teachers understand how it might be implemented in their classrooms before the start of the new academic year.
Changes to homework and assessment are already impacting teaching and learning in a number of ways. When GenAI becomes part of a homework task, for example, students are able to refine their skills to new and important areas of expertise, such as prompt craft. Not every student has access to an internet connection outside of school, so being mindful of this is the first step in utilising the benefits of the technology. When schools keep their computer labs open to provide more equal access to GenAI, it seems as though this is having more of a positive impact than any when similar initiatives have been introduced in the past, at least as far as active engagement is concerned. Early reports suggest, however, that once the novelty of this new technology has worn off, high school students are less inclined to want to spend additional hours on the school premises. More research needs to be done in this area.
The fact that assessing the quality of prompts and for students to then be evaluating the relevance and accuracy of output has been creating more engagement across a variety of subject areas so far, can only be seen as a positive impact. With an array of opportunities to embrace GenAI in the implementation of new EdTech, schools have an immense number of decisions to make about how they choose to embrace the changes that are set to come. The next step is for the teachers who have immersed themselves into subject areas such as prompt craft and output analysis might be able to share their knowledge and experience with those who have not.
This is where Goodnotes aims to come in: through providing recommendations to schools on the basis of the research we have conducted up to this point. Although we remain immensely grateful for those who have embarked on this journey with us so far, there is a huge amount of learning and discovery that still needs to happen. With an active monthly user base of 21.5 million around the world, and some exciting announcements coming about what can be achieved through our Apple Award Winning app, we will be at the forefront of providing sound and reliable guidance for schools as they embark on their own journeys with GenAI. We hope that you will also join us for the ride.
