Commentary on Assessment as Reality TV
Super Brain and University Challenge are examples of popular variety shows that pit university students against one another. What if in the future, it is also a format for universities to assess and showcase the capabilities of students in a competitive AI augmented world.
Furthermore, the assessment is not just about conceptual knowledge or application of knowledge but the synthesis of knowledge from both the human being and their AI co-pilot. It is also not only focusing on authentic assessment but future-oriented authentic assessment.
This assessment setting is built upon predictions that expectations of graduates would only become greater with generative and other forms of AI. Everyone is expected to be more productive, more creative and more critical thinkers. But what does efficiency, creativity and critical thinking mean when we have generative AI that can write poems, analyze images and code from a diagram on a piece of napkin? Perhaps, it is showcasing how effective we are in working with AI to reach a specific goal.
Hence, the idea of a competition where people display how they are better at using their AI copilot for a given problem. In other words, we can say that John had won the competition not because of JABIS II (which stands for Just Another Better Intelligent System II) but how he was able to use it to solve a problem better than others whose co-pilot is based on the same system. It is almost a Pokémon reference. John’s LITTLE JOHNNY is akin to Ash’s greninja. Ash’s bond with his greninja makes it stronger than any other trainers’ greninja. BIG JOHNNY is not just another JABIS II because of John and how well he is able to synchronize with it.
Not everyone has access to the latest and greatest. The main character Sok Srey was depicted as having access only to JARIS which is at least two generations older. Perhaps, the competition would be fairer if we had given every participant the same co-pilot. I perfectly agree. However, in today’s world, not every student has a laptop or the same laptop. In professional competitions such as tennis and e-sports, individual players have different preferences in terms of their equipment. Furthermore, it makes great television when someone overcomes the odds to win with a poorer piece of technology. An even more compelling and convincing story about how the human brain can triumph over machines and algorithms.
The story also paints the role of professors and experts as designers and commentators in the assessment process. The assessment is left to their AI co-pilot YAMA (who is the God of Death and Justice in Hinduism). It is plausible that even the professors themselves have used YAMA or other AI to design the task. However, it is equally possible that the criteria YAMA used to determine the value of the participants’ answers are in fact also co-designed by the professors. Would we as a society come to accept assessments co-designed with AI, co-completed with AI and co-determined with AI as reliable and valid?
The ending scene of the story is a cheeky dig at how assessment may still be competitive and individualistic even though we know that collaborative learning is more effective and that the workplace more likely than not would continue to value people who can work with others to accomplish the company’s goals. It would be rather fascinating if collaboration is more prevalent between the AI agents themselves, between AI and humans rather among humans ourselves.
