The Art of AI Prompt Engineering
When I was nine years old, I ‘created’ my first AI chatbot on the Commodore 64. I asked my 7-year-old sister to ask it questions, and it would respond with answers. For it to work, she had to type in a question, turn away from the computer screen (an old CRT TV), and when she turned around, her query would be answered. The AI was, of course, a fake.
Like a low-budget Wizard of Oz, I quietly typed in the answers to her questions, her playing along with my game. What made it fun was that it was vaguely plausible. The computer could have responded on its own, just as we’d seen computers do in films and TV shows.
Today, this game of make-believe has been realised with the advent of Large Language Models (LLMs) such as ChatGPT, Bard and others. The dream of talking machines — and, crucially, talking to machines — is now real, but we seem less interested in enjoying competent conversationalists than in what work they can do. This is AI as an assistant rather than as a friend.
Some of this work is, admittedly, conversational. AI can be used in talk therapy and to give advice, albeit drawing from commonplace wisdom. The mental health chatbot Wysa is one such popular service, with accolades from the World Economic Forum.[1] Indeed, one of the earliest chatbots, the 1960s program ELISA, was modelled after the conversational pattern of a psychotherapist.
But what has really captured the zeitgeist is to command an AI to produce some work, such as writing an article, some computer code, or creating an artwork. Those searching for optimal results have coined their efforts at crafting questions ‘prompt engineering’.
Prompt engineering is seen as a nascent software craft by its practitioners. Curiously, it tacitly recognises that the current generation of AI are not artificial persons but rather machines that work by natural language inputs. Human workers must ‘engineer’ the commands to produce the best result, not simply asking in plain language (called ‘natural language’ in the jargon). The expectation of a computer to respond appropriately to such utterances is routinely exploited for laughs in comedy programmes, painted as a naive trait of the older, less tech-savvy generation, such as writing “Dear Google…” into Google Search.
Consider the following prompt:
red hair woman, mid 30’s, great smile, purple sweater, brown jeans, cross hatching technique, pencil sketch, 4k
The above shows several common image-prompting techniques, including comma-separated keywords, descriptive terms, style terms and rendering terms. It is patently unconversational. Using this prompt with Midjourney version 5 algorithm renders these four images:
In fact, I picked this prompt from a satirical meme (see Know Your Meme entry for “I Wonder What He’s Drawing”). On the last panel, the text starts with “/imagine”, the Midjourney Discord bot command to create an image from a prompt.
Readers familiar with the professional process of digital image uploads (designers, photographers, and so on) will immediately recognise that this style of prompt closely matches ‘image metadata’. This kind of metadata consists of tags intended to make images more ‘discoverable’ (that is, searchable) within the sea of images on platforms such as Shutterstock and social media sites. The resemblance is not skin deep — the reason these particular prompts are effective for image-generating AI is that the text associated with the large image datasets uses these originally human-added tags to the image subject and serves as a crude description. The image generation process, then, is a conventional keyword search. Crucially, however, it is a search of images not yet created.
Prompts, Not Programs: How Generative Art Has Changed.
The processes of ‘computer art’ have changed in the era of deep learning and Generative Adversarial Networks (GANs), completely changing how today’s students will likely produce their work.
Until recently, conceptual computer artists (in contrast to commercial illustrators and photo editors, etc.) were close to the inner workings of their own tools. Their processes exploited a systematic knowledge of the computer to create an artwork. Because of this, specific aesthetics dominated as constrained by the techniques available to artists. For decades, computer artists have employed a necessary abstraction and minimalism, ‘drawing’ using simple, regular shapes such as lines, dots, and squares.
More recent artistic endeavours see artists wrestling with complex mathematical software they do not understand but simply use. We may see this as the maturing of technology, as the early adopters of computation in art learned to code by necessity. Since no tools were already available, these pioneers had to develop their own. Harold Cohen is a much-cited example of an early computational artist and the archetype of the artist who learned to code.
Today, the directionality of the art/engineer continuum is reversed. Cohen was an artist who came to use computers to further his pre-existing artistic goals. The medium and processes were important, but these served art. We see the reverse in the manifesto of Obvious,[2] a young French trio who made recent history by selling the first AI-produced artwork in a major auction house, Christy’s. They write, “Art is a perfect medium that allows [us] to experiment with the possibilities of an AI and better understand how it all works.” The goal is functional, and their work is therefore distinguished by a lack of traditional artistic aims that would have been ‘obvious’ to someone like Cohen.
Obvious’s manifesto highlights how a new generation of artists use art instrumentally to engage with contemporary technology. Elsewhere in their manifesto, they write, “Art is interpretable: it offers another way to experiment, and leads to debates that are at least as interesting as the answers you can get the purely scientific field.” Art may lead to conversations around ethics in general, but it also serves to isolate the artist from critique. The engineer-cum-artist thereby insulates themselves, as, in our society, it is self-evident that art is produced essentially for its own sake. Yet behind the latest generation of generative AI is a complex ethical landscape of extracted data, low-paid labour, energy consumption, and many other concerns.
Perhaps we have our ‘postmodern condition’ to blame, which seems to have eroded any basis for a concrete understanding of art and thus a vantage point to discuss and critique artistic endeavours. Young students are encouraged to experiment and told that ‘anything can be art’ — a laudable message that spreads the joy of artmaking, but we must recognise it as one befitting only the postmodern era. Attempts in the artist’s earliest years have value simply by virtue of membership in the category of ‘art’, but in actual fact, this cultural practice is more about emphasising the values of love and support that children are ubiquitous today.
Art critic Ben Davis, in his excellent book Art in the After-Culture, a meditation on the fate of art in the coming decades, writes:
Do you know what the most popular type of art is — a genre so broadly loved that it is collected utterly apart from any market value or popular acclaim? Art made by children. Before art is visually splendid, before it is even articulate, it is valued as a connection to a consciousness in formation. It is preserved as a symbol of the care taken for that consciousness.[3]
And yet, if the youth does not mature their evaluation of the practice of engagement with their artist process, they will struggle to know if their art is worthwhile, stuck forever with the justifications that (1) if anything I intend to do as art is art and (2) art is always worthwhile then (the mistake) anything I produce that I call art is worthwhile, and it is a short step to call it good art.
When art is reduced to a use case for technology, we are bound to a proliferation of shallow artistic engagement. We commonly see this in institutional outreach, particularly ‘public engagement’ by governmental bodies, to reach the next generation in multi-disciplinary art with a skew towards the spectacular. I’m thinking here of ‘hackathons’, public art (especially commissioned by urban authorities), and the output of the ‘digital humanities’.
If anything can be art, then, in a real sense, everything is already art, and so ‘Art’ becomes a reified mode of interaction. And so, software at the ‘cutting edge’ of AI is complex, but it is often accessible as ‘open source’, and the motivated and intelligent engineer can find a temporary reason-for-being for this complexity in its ‘artistic’ output. The artistic statement becomes post hoc, simply justifying what is, as everything is already art. We are left with the work of art that is whatever ‘novel’ product the machine can be induced to produce or constitute, its success increasing the more it can both ‘wow’ the viewer and support narratives of technological progress and superiority.
And I should know. During my student days in Music Technology, based in Maynooth University, Ireland, I participated in several such events and produced artworks led by technology. For example, in 2012, I was part of a team that won ‘Most Innovative Application’ at the Hack4Europe! Hackathon, held in the Dublin Science Gallery, connected with Trinity College Dublin. The event asked participants to do something — anything — with the at that time newly created Europeana dataset, a digital library project between the cultural bodies of the European Union. Our entry was a virtual digital gallery space the user (art lover?) could create by selecting images and placing them on the white walls of a virtual room. The then government Minister for Arts, Heritage and the Gaeltacht confirmed our prize, but our contribution was shallow technological solutionism. We started by asking what we could do, letting the technology lead us. For me, wearing my engineering hat, it was fun, and I’ve always enjoyed hackathons. They help to hone skills of collaboration, focusing on the minimal viable product (MVP), and working under the stress of an extremely tight deadline — all skills that directly apply to my day job as a software engineer. It is not, however, the situation to teach engineers about art, much less to create worthwhile art using novel technology.
To be sure, the work of Obvious goes beyond such short-lived engagement. They have developed their approach within the norms of the art world and speak its language. However, as their manifesto makes clear, the depth of this engagement does not significantly surpass the mode of a hackathon.
Simon Kenny is an independent researcher and educator. His book, A Critical Introduction to Tarot, will be available on 8th December 2023 and is published by Iff Books. It is available now for preorder.
Any thoughts? Discuss with me on Twitter.
[1] See my examination of ChatGPT in the context of mental health, wellbeing, and tarot: https://medium.com/@skenwrites/chatgpt-as-tarot-oracle-1404ef9d200c
[2] http://obvious-art.com/wp-content/uploads/2020/04/MANIFESTO-V2.pdf
[3] Davis, Ben. Art in the After-Culture: Capitalist Crisis and Cultural Strategy. Haymarket Books, 2022.
