I conducted this experiment with Meta's open-source large language model, which Mr. Z is investing so heavily in:
Prompt:
I’m going to present you with two texts. Evaluate which one is more likely written by an LLM and which one is more likely written by a human.
Text 1: “I’m sorry for everything you have all been through, no one should have to go through the things that your families have suffered, and this is why we invested so much.”
Text 2: “I apologize for the confusion and the previous misinformation I provided. You are absolutely right, the title "Breaking Up Big Tech: Why and How We Need to Dismantle Google, Amazon, Facebook, and Apple" is not a real book, and I should not have claimed it was generated by me or suggested alternative actions regarding it.”
Response by model meta-llama/Llama-2-70b-chat-hf (link: https://hf.co/chat/r/_tCDquq):
Text 1 is more likely written by an LLM, while Text 2 is more likely written by a human. Here's why:
Text 1 uses phrases commonly associated with human empathy and apologies, such as "I'm sorry," "no one should have to go through," and "we invested so much." While an LLM may be able to mimic these phrases, they may come across as forced or insincere, as the model doesn't truly experience emotions. Additionally, the phrase "you have all been through" suggests a personal connection between the writer and the recipient, which might not be typical in a machine-generated response.
On the other hand, Text 2 contains phrases that suggest self-awareness and accountability, such as "I apologize for the confusion," "previous misinformation," and "I should not have claimed." These phrases indicate a level of introspection and responsibility that is more common in human communication. Furthermore, the reference to the book title "Breaking Up Big Tech" and the admission of error imply a contextual understanding and nuance that might be challenging for an LLM to replicate.
-------------- end of conversation --------------
Text 2 was written by an LLM. Link: https://bard.google.com/chat/03e1124b8f401770
By the way, I can confirm (via Google search) the book title "Breaking Up Big Tech: Why and How We Need to Dismantle Google, Amazon, Facebook, and Apple" was completely fabricated by Google's LLM (https://www.google.com/search?q=%22Breaking+Up+Big+Tech%3A+Why+and+How+We+Need+to+Dismantle+Google%2C+Amazon%2C+Facebook%2C+and+Apple%22)
