How to read a con man — Why we need to critically reflect upon the design of conversational AIs
Over the past months I’ve been following the rapid advancements of AI as expressed in ChatGPT, Midjourney, and numerous variations of these. It’s a wild time to be alive, to say the least, and I have to admit that I have been both impressed and excited by what is being made possible. However, I have also become increasingly concerned over what this development can lead to, and how this is happening. My main issue? The way that AI is designed to convince or deceive us into believing that what it brings us is true.
Let me explain.
A while ago I asked ChatGPT to summarise a quite long text that I had written about innovation and creativity in large organisations. The response that I got was really structured and very well formulated:
“The article “We Have to Stop Laughing and Start Working Now” discusses the urgent need for action in the face of the current climate crisis. The author argues that while it’s easy to laugh at the absurdity of some climate-denying politicians and their supporters, it’s time to move past this and focus on real solutions.
The article highlights the severity of the crisis, citing recent extreme weather events and the dire warnings of scientists. It also notes that while individual actions are important, they alone will not be enough to mitigate the damage already done.
The author suggests that the only way to truly address the crisis is through collective action, both by governments and by individuals coming together to demand change. They call for a shift in mindset from one of complacency to urgency, and for people to prioritize the future of the planet over short-term personal gain.
Overall, the article serves as a call to action for readers to take the climate crisis seriously and to take steps towards creating a more sustainable future.”
The only, yet rather big, problem is that the answer has absolutely nothing to do with my text. None at all. It’s completely made up. When asked about it, ChatGPT claimed that the link to the article might have been incorrect or that the article had been removed, none of which was true at all.
Let’s have a look at another example. I asked Midjourney to create an image that would depict “People sitting outdoors at a cafe in Sweden”. I was presented with the following four images:
All in all, the environments look great and the overall light feel quite Scandinavian so at first glance it’s pretty impressive. People’s faces are in some cases a bit strange, but that’s probably because I’m using an older version of Midjourney — examples from the latest release show great improvements in that aspect. The big question mark in this example has to do with diversity. Sweden is demographically speaking a diverse country with citizens of different colour, religious beliefs, and cultural affiliations — none of which is depicted in either of the four examples.
The problem of believability
The big problem that these two examples illustrate is about how AI applications such as ChatGPT and Midjourney have been designed to work. The remarkable thing is that you can get these outputs from very simple conversational instructions, which in one sense is quite user friendly. No complex interfaces, just plain language. However, the output comes back in an equally simplistic, and deliberately human like, way. The analysis of my article is very well written, totally believable. However, it was also completely wrong, something that I only could know if I knew the content of my article and could compare the two. Imagine that I had been asking about a topic or domain that I wouldn’t know that much about, a use case often brought up when AI is being discussed — that AI is suitable for addressing complex problems or contexts — how would I be able to assess and evaluate the response? The same reasoning applies to the image generation example. There is something magical in writing something very simple and get such a ultra realistic back in seconds. Apart from the bias towards racial stereotyping of a country that is, but how many will really notice?
The interfaces to and interaction with these applications are designed to deliver a result that is understandable and believable for me as a human, almost as if I’m having a dialogue with another human being. This is hugely concerning, because by trying to become as human like as possible it becomes very difficult for a real human to critically reflect upon the information presented. In normal dialogues we evaluate many different parameters when interpreting and making sense of information, especially when we are discussing with other human beings. If I ask someone for advice I mostly have an idea of the person’s competence, experience, and expertise, but I have also a hunch about other contexts relating to politics, values, interests, agendas, etc.
Technology in disguise
In terms of conversational AIs this is very difficult. The technology behind is often black boxed, the data being used unclear, and to even suggest to look into aspects such as bias, politics, values etc feels far away. It can be argued that in theory one can ask the AI about these aspects, or include them in the prompts, but even then there is no guarantee that the answers are coming back true. AIs don’t have degrees, job histories, or references to call, they are simply code that someone once wrote. To engage with technology of this capacity using human language in the way that we are currently envisioning it is hugely problematic. For more in depth reflections about this, read Yuval Noah Harari’s recent article in the Economist in which he argues that AI has hacked the operating system of human civilization.
A call for new design approaches
In the light of the above it is not an overreaction to say that we have to start thinking about how AIs applications as mentioned in this text are to be interacted with in order to become useful and a source of good. This is to a large extent a design issue. We have to start looking at these new technologies from more perspectives than making them believable, preferably by addressing them as completely new domains for interactions to take place and that have to be investigated, developed, and designed accordingly. This should have started quite a while ago, but better late then never.
