Human prose to AI poetry: Biko, Buthelezi, and the Tale of Two Models

So, you wanna summarise an article, but don’t know whether to fork out for ChatGPT, or be cheap and self-host Llama2 using something like Ollama?

Well, here’s a quick comparison of the two, based on a single article from Daily Maverick: The vastly different legacies of Buthelezi and Biko.

First, let’s start with some headline suggestions. The prompt I used was:

“Generate 3 headlines for the article below. The headlines should be different from the original headline and should be written in a way that will make people want to read the article. The headlines should be written in the style of Daily Maverick. The headline should be approximately 14 words long and contain at least one power word. Return only the headlines, one headline per line:”

Llama2 cares not for instructions, it seems:

 Here are three headlines for the article above:

1. "Biko's Vision of Freedom Fades as Buthelezi's Legacy Reigns" - This headline emphasizes the contrast between the fading memory of Steve Biko and the continued relevance of Mangosuthu Buthelezi's legacy.
2. "Buthelezi's Legacies: Separatist Agenda or Cautionary Tale?" - This headline raises questions about the nature of Buthelezi's legacy and encourages readers to engage with the topic more closely.
3. "The True Meaning of 'Nation Building': Biko vs. Buthelezi" - This headline highlights the contrast between Biko's vision of a united, oppression-free society and Buthelezi's focus on tribalism and separatism.

In a few tests with slightly-altered prompts, Llama2 consistently included the numbering as well as the extra info on what it thought of the headlines. It really likes the sound of its own voice.

Interestingly, the larger Llama model (13b) included a word count:

Here are three possible headlines for the article, each one emphasizing 
a different aspect of the legacy of Steve Biko and Mangosuthu Buthelezi:

1. "Biko's Vision of Freedom vs. Buthelezi's Legacy of Violence: A Tale 
of Two Leaders" (14 words)
2. "The Real Story Behind Buthelezi's 'Nation Building': A Brutal 
History of Slaughter and Divide" (15 words)
3. "Remembering Biko's Humanity and the True Cost of Buthelezi's 
Power-Hungry Ambition" (14 words)

Each headline emphasizes a different aspect of the two leaders, with 
the first one highlighting the contrast between Biko's vision of 
freedom and Buthelezi's legacy of violence, the second one focusing 
on the negative aspects of Buthelezi's leadership, and the third 
one emphasizing the humanity and true cost of Buthelezi's actions.

Hilariously, it got the word count wrong. At least this time it was close: in another experiment summarising to 50 words or under, it did the same thing, but got the word count wrong by almost 100%, hitting an impressive 94 words, but still putting (50 words) at the end.

Here’s an example of ChatGPT’s headlines:

"Unveiling the Forgotten Legacy: Steve Biko's Heroic Vision vs. 
Buthelezi's Troubled Past"
"Biko's Dream vs. Buthelezi's Controversial Path: Unraveling 
South Africa's Complex History"
"Biko's Vision of Freedom vs. Buthelezi's Divisive Agenda: A 
Tale of Two Legacies"

Well done ChaptGPT! It followed instructions and just gave me the headlines. (Although there are still annoying inverted commas.) Interestingly, both models included colons in a lot of the headlines. They also both capitalised the first letters. Neither of these is a Daily Maverick style, and I know through experimentation that at least ChatGPT has definitely been trained on DM’s work.

I didn’t love most of the headlines, although “Remembering Biko’s Humanity and the True Cost of Buthelezi’s Power-Hungry Ambition” definitely has some potential.

How about a nice summary, like the ones we generate for Daily Maverick’s Summary view using GPT4?

Prompt: “You are a Daily Maverick news editor that writes enticing, exciting summaries. Summarize this article in a manner that will give casual readers enough information on it, but will entice interested readers to click to find out more. Keep it down to 50 words or less”

ChatGPT really went large on the enticing part, heading all the way over to soap opera:

"Unveiling Biko and Buthelezi: A Tale of Heroism and Controversy. 
Explore the complex legacies of these two South African figures – 
one a dreamer, the other divisive. Discover the untold history 
that shaped a nation's path. Click to delve deeper into their 
stories."

Meanwhile, Llama2 just made up the word count:

Steve Biko dared to dream of a future where all black people were 
free from oppression and violence, while Mangosuthu Buthelezi led 
a separatist and tribalist agenda that turned people against each 
other. As we remember the death of Bantu Stephen Biko, it's 
crucial to examine these separate but intersecting legacies. 
While Buthelezi lived to 95, his legacy is marked by the brutal 
slaughter of thousands, while Biko's ideology lived on through 
the struggle for a democratic system free from oppression. An 
honest account of history will show that we owe Biko our 
freedom. (50 words)

(An aside: it’s impressive that it knows Steve Biko, Bantu Stephen Biko and Biko are all one person. I get how it works, but it’s still great to see it in action.)

That’s all great, but let’s see what else we can do with this article. How about some poetry?

“Turn this news article into a poem in the style of John Donne: “

Llama2:

 Here is the news article rewritten in the style of John Donne:

Biko and Buthelezi, two men of contrasting mold,
One sought to free the black soul, the other to hold.
In the shadow of apartheid's tyranny, they lived and breathed,
Their legacies intertwined, yet worlds apart they leave.

Biko, a visionary, his heart afire with dreams,
Dared to challenge the system, to break its schemes.
He saw beyond the chains that bound the black mind,
And sought to heal the wounds of history's cruel kind.

Buthelezi, once a beacon of hope, now a cautionary tale,
Led a separatist crusade, his people's souls to bale.
He turned against his kin, and embraced the oppressor's cause,
And with pangas and fire, he brought death to many laws.

Their paths diverged, like the sun and the night,
One shone with light, the other with darkness and fright.
Biko gave his life for the freedom of all,
Buthelezi's legacy, a stain that cannot befall.

Yet, in death, both men are united in their quest,
For justice and equality, their souls now at rest.
Their stories told, their legacies intertwined,
A lesson to all, of the power of the human mind.

ChatGPT:

In days of yore, when strife did brew,
Two figures stood, their tales askew,
Steve Biko's dream, a radiant light,
Mangosuthu's path, veiled in night.

Biko, with vision, dared to dream,
Where black souls soared in freedom's gleam,
In hearts of ebony, he sowed the seed,
Of worth and value, a dire need.

Buthelezi, on a tribalist quest,
Tore unity from nation's chest,
A separatist path, he chose to tread,
Dividing brethren, blood was shed.

As we gaze back upon their time,
In South Africa's complex clime,
Biko's principles, humanity's creed,
While Buthelezi's choices, sowed a divisive seed.

History's mirror, unflinching and clear,
Reveals the truth we all must hear,
Biko, our beacon, in freedom's name,
Buthelezi's legacy, a cautionary flame.

An honest account, we must convey,
For freedom's price, we still repay,
In Biko's dream, our hope does swell,
In Buthelezi's path, the tale we must tell.

The rhyming and rhythm is pretty decent, and something that LLMs have typically struggled with. ChatGPT was really into “hearts of ebony”, coming up with: “… Biko’s dream, a radiant star, // In ebony hearts, it traveled far,…” when I tried it with Keats.

And finally, a haiku.

Llama2:

 Here is a haiku summarizing the article:

Biko dared to dream
Of freedom from oppression
Buthelezi divided

ChatGPT:

Biko's dream alight,
Buthelezi's path divided,
History's twin tales.

This time, Llama2 did better with the counting, nailing the syllables, although I think I prefer ChatGPT’s haiku.

So, what conclusions can we draw? Not many from one news article and hardly any prompt fiddling (or “engineering”, as they call it).

It’s left me a bit wary of Llama2’s penchant for throwing in unnecessary explanations. I usually use GPT through an API to feed to editors, suggesting summaries, Tweets and headlines. The extra fluff will make things very difficult.

I’m also appreciating how fast ChatGPT is: on my 16GB Mac M1 (hardly the highest spec, but still…) it could take two minutes to complete the process. ChatGPT is faster than it ever was, starting to generate solutions immediately and completing them in seconds.

I’ll end off with ChatGPT rewriting this article as a joke:

Why did Steve Biko and Mangosuthu Buthelezi walk into a history class?

Well, Steve Biko was full of dreams, always saying, "Let's rise above 
oppression!" Buthelezi, on the other hand, had a tribalist agenda and 
was all about dividing people.

In the end, Biko's dream was so bright that it lit up the room, while 
Buthelezi's divisive ideas just left everyone in the dark!

Moral of the story? History class can be a real eye-opener, especially 
when you've got dreamers and dividers in the mix!