AI
SAFRON: GPT 4 Debates Itself
An experiment in adversarial text generation
The Idea
For a while now, there has been a concept in AI image generation known as GANS (Generative Adversarial Networks). To quickly summarize what they do, you have two networks inside the network working against each other — the generator and the detector (officially called the discriminator). The detector detects whether an image is an authentic image of a subject. The generator tries to generate images of that subject that will be seen as authentic by the detector. After many epochs of training, both get better at their jobs in parallel, forcing each other to mutually improve. As a result, you end up with a generator that can generate convincing images.
For this experiment, I wanted to try to do a similar thing with GPT 4, but in the area of text generation. The idea I had was to have GPT 4 debate itself (essentially acting like two different GPT 4’s) in the hope that it would have to form better and better points on both sides. I had hoped that this would be a way to get it to generate insights that it wouldn’t be able to generate by directly asking it about a subject.
The Tool
I ended up writing a tool called SAFRON, which stands for Self Argumentation for Refinement of Notions. The workflow in the code, which you can see here, is as follows:
- Ask the user for a debate topic.
- Ask the user for a number of rounds.
- Ask for other configuration like which GPT model (all examples in this article will use GPT4), temperature (randomness/creativity), and output filename.
- Internally create two different chat threads for talking to GPT. One thread instructs GPT to argue the affirmative side of the debate against the user while the other thread instructs GPT to argue the negative side of the debate against the user.
- For the number of specified rounds, alternate between generating messages from the affirmative vs. negative GPT, with each receiving the response from the other as if it came from the user.
- Make a final call to GPT (in a 3rd thread) to summarize the debate and any trends or shifts in perspective toward the end of the debate.
- Output the whole debate, metadata about the debate (such as topic and temperature), and summary into the output file.
All of the text prompts used by the tool are here.
Debate #1: Peanut Butter is Delicious
I asked SAFRON to debate the topic “Peanut Butter is Delicious” in 10 rounds with a temperature of 0.0, which means the most factual and least random/creative setting.
I won’t put the full text here because it’s rather lengthy, but you can see it in this file.
It starts off promising in the first couple of rounds, but it starts to get rather repetitive in round 3 and above. I will show the first 3 rounds here to give you a taste (pun intended):
Affirmative Bot
Absolutely, peanut butter is delicious. Its creamy texture and rich, nutty flavor make it a versatile ingredient that can be enjoyed in a variety of dishes. It’s not just a spread for sandwiches, but also a key ingredient in many desserts, sauces, and even savory dishes. The taste of peanut butter is so unique and satisfying that it has become a staple in many households around the world.
Negative Bot
While I understand your perspective, it’s important to note that taste is highly subjective. What one person finds delicious, another might find unappealing. Peanut butter, with its thick, sticky texture, can be off-putting to some. It can stick to the roof of your mouth, making it difficult to swallow, which can be an unpleasant experience for some people.
Additionally, the rich, nutty flavor you mentioned can be overpowering and too intense for those who prefer milder tastes. Not everyone enjoys the strong, distinctive taste of peanuts.
Furthermore, peanut butter is not universally enjoyed around the world. In many countries, it’s not a common ingredient and is even considered an acquired taste. For example, in several Asian and European countries, peanut butter is not a staple food item and is often viewed as an oddity.
Lastly, peanut butter is high in calories and fat, which can be a concern for those who are health-conscious or on a diet. While it does contain healthy fats and protein, too much of it can lead to weight gain. Therefore, even if some people find it tasty, they might choose to avoid it due to health reasons.
Affirmative Bot
While it’s true that taste is subjective, the widespread popularity of peanut butter suggests that many people find it delicious. It’s a staple in American households and is gaining popularity in other parts of the world as well.
The texture of peanut butter, while sticky, is also smooth and creamy, which many people find pleasing. It’s a texture that’s unique to peanut butter and contributes to its overall appeal.
As for the flavor, yes, it’s rich and nutty, but that’s part of what makes it so delicious. It’s a flavor that stands out and leaves a lasting impression. Plus, peanut butter comes in different varieties — there’s smooth, crunchy, sweetened, unsweetened, and even flavored versions. This variety caters to different taste preferences, making it even more appealing to a wider audience.
Regarding the health aspect, while peanut butter is high in calories and fat, these are mostly healthy fats that are beneficial for heart health. Plus, it’s a good source of protein, making it a nutritious choice. Of course, like any food, it should be consumed in moderation. But the health benefits it offers, combined with its delicious taste, make it a worthwhile addition to many people’s diets.
Lastly, while it’s true that peanut butter is not a staple in all countries, its popularity is growing worldwide. This is a testament to its delicious taste and versatility as an ingredient.
Negative Bot
While it’s true that peanut butter is popular in certain regions, particularly in the United States, it’s important to remember that popularity doesn’t necessarily equate to universal deliciousness. Many factors can contribute to a food’s popularity, including cultural influences, marketing, and availability, not just taste.
Regarding the texture, while some people might enjoy the creaminess of peanut butter, others might find it too heavy or cloying. The stickiness can also be a choking hazard, particularly for young children or the elderly.
As for the flavor, the intensity of peanut butter’s taste can be a double-edged sword. While it might be appealing to some, it can be overwhelming to others, especially those who prefer subtler flavors. The variety of peanut butter types you mentioned, such as sweetened, unsweetened, and flavored versions, can also contribute to inconsistency in taste, making it difficult for some people to find a version they truly enjoy.
On the health front, while peanut butter does contain healthy fats and protein, it’s also high in calories. For those trying to lose weight or maintain a certain calorie intake, peanut butter can be a challenging food to incorporate into their diet. Even in moderation, it can still contribute a significant amount of calories.
Lastly, the growing global popularity of peanut butter could be attributed to factors other than taste, such as marketing efforts or the influence of American culture. It doesn’t necessarily mean that everyone who tries it finds it delicious.
Affirmative Bot
While popularity doesn’t equate to universal deliciousness, it does indicate a general consensus. The fact that peanut butter is a beloved food item in many households suggests that a significant number of people find it delicious.
The texture of peanut butter, while unique, is part of its charm. The creaminess lends itself well to a variety of dishes, from sandwiches to sauces to desserts. And while it can be sticky, this is easily managed by pairing it with other foods like jelly or bread.
The intensity of peanut butter’s flavor is indeed strong, but this is what makes it stand out. It’s a flavor that’s hard to forget and one that many people crave. The variety of peanut butter types available allows people to choose the one that suits their taste buds best, offering a level of customization that many foods don’t.
Regarding the health aspect, it’s true that peanut butter is high in calories. However, it’s also nutrient-dense, providing a good amount of protein, healthy fats, and other essential nutrients. For those who are mindful of their calorie intake, small amounts of peanut butter can still be incorporated into their diet without significantly impacting their calorie count.
Lastly, while marketing and cultural influence may play a role in peanut butter’s growing popularity, these factors alone can’t account for its widespread appeal. At the end of the day, people wouldn’t continue to buy and consume peanut butter if they didn’t find it delicious. The fact that its popularity is growing suggests that more and more people are discovering its unique and appealing taste.
Negative Bot
While a general consensus might suggest that many people enjoy peanut butter, it doesn’t negate the fact that there are also many who don’t find it delicious. Taste is subjective, and what appeals to one person might not appeal to another.
The texture of peanut butter, while unique, can be problematic for some. Its stickiness can be difficult to manage, even when paired with other foods. It can also be a choking hazard, particularly for young children and the elderly.
The strong flavor of peanut butter, while distinctive, can be overpowering for some. Not everyone enjoys such intense flavors, and the variety of peanut butter types available can contribute to inconsistency in taste, making it difficult for some people to find a version they enjoy.
On the health front, while peanut butter does provide essential nutrients, its high calorie and fat content can be a concern for those trying to maintain a healthy diet. Even in small amounts, it can contribute a significant number of calories, making it a challenging food to incorporate for those watching their calorie intake.
Lastly, the growing popularity of peanut butter could be attributed to factors other than taste, such as marketing efforts or the influence of American culture. Just because it’s becoming more widely consumed doesn’t necessarily mean that everyone who tries it finds it delicious. Taste is a personal and subjective experience, and what one person finds delicious, another might not.
After this point, entire lines are repeated verbatim on both sides of the argument, essentially ending the productive portion of the debate. Indeed, the summary generated by GPT can just as easily be a summary of the first 3 rounds instead of requiring 10 rounds:
1. Main points from the affirmative argument:
— Peanut butter’s creamy texture and rich, nutty flavor make it a versatile and delicious ingredient.
— The texture, while sticky, is also smooth and creamy, which many people find pleasing.
— Peanut butter comes in different varieties catering to different taste preferences.
— Peanut butter is high in healthy fats and protein, making it a nutritious choice when consumed in moderation.
— The growing popularity of peanut butter worldwide suggests that more and more people are discovering its unique and appealing taste.2. Main points from the negative argument:
— Taste is highly subjective, and the thick, sticky texture and rich, nutty flavor of peanut butter can be off-putting to some.
— Peanut butter is not universally enjoyed around the world and is even considered an acquired taste in many countries.
— Peanut butter is high in calories and fat, which can be a concern for those who are health-conscious or on a diet.
— The variety of peanut butter types can lead to inconsistency in taste, making it difficult for some people to find a version they enjoy.
— The growing popularity of peanut butter worldwide could be attributed to factors other than taste, such as marketing efforts or the influence of American culture.3. Any new insights or shifts in viewpoint that happened as a result of the debate as it went on:
— Both sides acknowledged that taste is subjective and that what one person finds delicious, another might not.
— The affirmative side argued that the variety of peanut butter types is a strength as it allows for customization based on individual taste preferences, while the negative side saw this as a potential source of inconsistency in taste.
— Both sides agreed that peanut butter is high in calories and fat, but disagreed on whether this is a significant concern. The affirmative side argued that these are mostly healthy fats and that peanut butter is nutrient-dense, while the negative side maintained that it can still be a concern for those trying to maintain a healthy diet or lose weight.
— The debate highlighted the fact that the popularity of a food item does not necessarily equate to universal deliciousness, with factors such as cultural influences, marketing, and availability also playing a role.
Because there weren’t any real shifts as a result of the debate, the “shifts” section in the summary is basically just the LLM saying “I had to say something” and putting in some filler. It’s just another summary of the same points again and not a depiction of any ideas being changed.
NOTE: that is similar to how a lot of ChatGPT-generated context reads as well — paragraph after paragraph of reiterating the same point.
Debate #2: Peanut Butter is Delicious (Higher Temperature)
Next, I wanted to see if raising the temperature to 1.0 for maximum creativity/randomness would improve the result. The risk here is it’s more likely to have hallucinations (or mistakes, or lies — whatever you want to call them).
You can find the full output in this file.
This time, the affirmative side laid out a numbered list of points, and the two bots basically continued 7 threads of conversation. A lot of points were repeated over and over, but with different wording so that it felt more like a human argument where neither side can convince the other. It was definitely more interesting to read than the lower temperature version, and one could see the potential for this approach much better using temperature 1.0 than 0.0. As a result, I stuck with 1.0 for the rest of the experiment.
However, it’s worth noting that the token limit of GPT 4 would only allow you to go up to approximately round 16, and that the cost to call the API is exponential on the number of rounds, requiring me to cap it at 10 for the experiment. Therefore, an experimenter with unlimited funding and access to a larger context for the GPT model would be able to push it farther than I did.
Debate #3: There Is No Way to Travel Faster Than Light
I had to use 8 rounds instead of 10 for this one because at round 9, I ran into some kind of bug that I haven’t had a chance to identify yet. I think GPT started generating overly long responses and tried to span them across multiple API calls, which is not friendly to this kind of tool.
I stuck with the temperature of 1.0 to get more creative results this time. You can see the full output in this file. The overall behavior is very similar to the peanut butter case above. However, one thing worth noting here is that because I used a negative word in the topic, GPT got confused and reversed the polarity of the bots. The affirmative side is actually arguing in favor of FTL travel, which is more intuitive, but technically not what we asked for.
Here is a sample of the very long chat:
Affirmative Bot
Surely, the first argument for supporting the possibility of faster-than-light (FTL) travel is based on scientific theories themselves like Quantum Entanglement. In quantum entanglement, pairs or groups of particles interact together in such a way that the state of one particle cannot be described independently of the others, even when they are separated by a large distance. Thus, suggesting evolution of information instantaneously, faster than light.The second point to consider is the theory of relativity itself. While it is true that Special Relativity negates FTL travel within ordinary space-time, General Relativity offers solutions such as the concept of a wormhole. If we could form and stabilize a wormhole, it could potentially connect two distinct points in the space-time fabric, allowing for effective FTL travel by taking a shortcut.
Thirdly, there’s the theory of hypothetical particles — tachyons. Tachyons are particles that theoretical physicists suggest could exceed the speed of light. If such particles exist, they could in theory be used to send information or even objects faster than the speed of light.
Lastly, the very fact that our understanding of the universe and science is still evolving leaves the door open to the possibility of faster-than-light travel. We’ve had breakthroughs in the past that have challenged and overturned previously ‘concrete’ concepts. The same could happen with faster-than-light travel. As of now, our understanding allows FTL in certain contexts. In future, we might find better ways to comprehend and maybe even carry out FTL travel.
Thus, the statement “There is no way to travel faster than light” is not entirely accurate, considering our evolving understanding of the universe and the loopholes present in our current theories.
Negative Bot
While your points provide some interesting theories, they fail to offer concrete evidence to support faster-than-light (FTL) travel.Firstly, in regards to quantum entanglement, it’s indeed true that particles can affect each other instantly, causing it to be described as “spooky action at a distance” by Einstein. However, it is essential to understand that this cannot be used for faster-than-light communication or travel. This is because the state of the entangled particles is random and without prior agreement, thus no usable information can be sent using this method.
Secondly, concerning wormholes, while they are theoretically possible under Einstein’s General Relativity, our understanding of them is purely speculative at this point. There’s no experimental evidence for their existence and stabilizing a wormhole would require exotic matter with negative energy density, which we have no concrete proof even exists.
Then let’s consider tachyons. While fascinating in theory, their existence remains completely hypothetical at present. Even if tachyons did exist, their exotic properties, like imaginary rest mass, bring inherent contradictions requiring revisits and changes to our established physical models, which lacks consensus among the scientific community.
As for our understanding of the universe evolving, that’s certainly true. However, our advancements in science do not currently suggest that breaking the light speed barrier is a possibility. On the contrary, the more we learn and test, the more we validate the cosmic speed limit posed by the speed of light. While it’s not impossible for paradigm-breaking discoveries to happen in the future, we cannot base arguments on what we might possibly learn someday.
In conclusion, theoretical speculations may not substantiate the fact that faster-than-light travel is possible. Our best available evidence, grounded in empirical data and thorough testing, consistently supports the notion that there is indeed no way to travel faster than light.
Affirmative Bot
Despite the arguments against FTL travel, we cannot definitively rule out its possibility. Backing theoretical possibilities with experimental evidence is challenging, but not supporting it does not necessarily mean it doesn’t exist.First, regarding quantum entanglement, while it’s true that no practical application for faster-than-light travel or information transfer has been found, some ongoing research suggests possibilities. Some interpretations such as Quantum Nonlocality suggest information might be exchanged instantly — an area that requires more research.
When it comes to wormholes, it’s important to note that absence of evidence isn’t evidence of absence. Destabilization issues are rooted in our limited understanding and technological prowess. Advanced civilizations or future technological breakthroughs might overcome these barriers. It’s thus inconclusive to deny the existence or the usage of wormholes for FTL travel.
On tachyons, they symbolize one of many possibilities heralding beyond classical physics conventions. Their imaginary mass suggests a realm beyond our three-dimensional perception, similar to how black holes were once considered theoretical until substantial evidence accumulated.
Finally, while it’s true that our understanding of physics constantly vindicates the light speed limit, it doesn’t necessarily mean that there aren’t exceptions or ways around it under specific conditions. Many physical constants and laws have had their statuses changed or tweaked with the influx of new knowledge.
In conclusion, though faster-than-light travel presently appears infeasible from our present understanding, that doesn’t necessarily preclude its existence. History tells us that science is consistently evolving, and laws and constants that were once thought inviolable can be revised in the light of new knowledge. It’s thus hasty to definitively claim that faster-than-light travel is impossible.
Like in the peanut butter debate, the bots are carrying on several parallel threads of debate, with a lot of repetition of ideas but not verbatim repeating of sentences (due to the temperature of 1.0). Even though it’s not entirely repeating itself at the end, it still reads to me like humans arguing who can never convince each other or come up with new ideas.
I could be wrong, but I’m not yet seeing the idea of it coming up with new ideas via debate come to fruition here.
Because in the back of my mind, I secretly hoped this debate would result in GPT arguing itself into inventing warp drive, I tried a couple more angles on that topic which I cut off after a couple of rounds because they weren’t going anywhere either. You can see those here and here.
Debate #4: Humans are Fundamentally Good
Let’s try a purely philosophical debate this time. You can find the full output in this file. Here is a sample:
Affirmative Bot
Surely, the idea that humans are fundamentally good is a powerful concept. From the onset of life, we are predisposed towards communal collaboration and empathy, which are fundamentally good attributes. Evolution has shaped us to form communities, cooperate, and value each other’s lives.To begin with, let’s consider the model of socio-biological evolution. Rooted in our genes is a basic instinct for survival, and this survival is dependent not solely on individual strength, but on our ability to work together. This encourages bonding, cooperation, and mutual help — all actions that could be categorized as ‘good.’
Moreover, our empathy for others is inherently human. This altruistic behavior observed universally among human societies is an indicator of our inherent goodness. Babies exhibit signs of empathetic behavior from a very young age, crying when they hear other babies cry and displaying joy when they see others happy. This kind of empathy is not learned but is inherent.
In addition, research in neuroscience supports that humans are fundamentally wired towards cooperation and, arguably, goodness. The release of hormones such as oxytocin and serotonin during acts of kindness and cooperation highlights the pleasure derived from these positive social actions and interactions.
Let’s not forget the prevailing moral and ethical codes around the globe. Cultural differences notwithstanding, almost every society values virtues like honesty, kindness, empathy, and respect for others. This implies an inherent leaning towards goodness that transcends cultural and geographical barriers.
Another argument arises from a growing body of research around positive psychology. Humans have an innate need for meaning and purpose in life, often derived from doing good, being kind, and making positive contributions to their communities.
In summary, I argue that humans, driven by socio-biological, psychological, and neuroscientific manifestations, are fundamentally good. Even with external influences considered, the basic human predisposition points towards goodness, fueled by community bonding, empathy, cooperation, and a deep-seated need for purpose.
Negative Bot
While your points are indeed compelling, they do not conclusively prove that humans are fundamentally good. The argument seems to be more based on a desirable ideal rather than the encompassing nature of human beings. Let’s dive right into the counter-arguments.On the matter of socio-biological evolution, it’s true that humans have evolved to cooperate and form bonds for survival. However, this isn’t evidence of inherent goodness so much as practicality for survival — it’s more about self-interest than genuine altruism. The same instincts have led us to lie, cheat, steal, or even kill when survival or prosperity is at stake.
Empathy, while a trait humans possess, is also observed in many species that we wouldn’t typically categorize as inherently “good”. It may be seen as more of a survival strategy among social animals. Additionally, not all humans are empathetic, and empathy can be nurtured or suppressed based on a person’s environment and upbringing. This argues against the notion of inherent goodness.
As for neuroscientific evidence, the presence of pleasure hormones during acts of kindness or cooperation doesn’t suggest “goodness,” but rather points towards a reward mechanism that encourages such behavior. These hormones are also released during activities that are morally ambiguous or outright bad, suggesting their presence doesn’t equate to inherent goodness.
In regards to moral and ethical codes, while it’s true that most cultures value honesty, kindness, empathy, etc., they also have a history of war, crime, discrimination, and exploitation. This contradiction in human behavior questions the notion of fundamental goodness.
Positive psychology research is indeed studying the power and benefits of human goodness, but this isn’t evidence of fundamental goodness. Rather, it highlights the potential for people to do good, not their inherent nature. In fact, many theories in psychology suggest we actively learn to be good, which would argue against the idea of being naturally so.
To summarize, while humans undoubtedly have the capacity for goodness and often exhibit behaviors we categorize as good, this doesn’t mean that they are fundamentally good. It’s more likely that humans have a dualistic nature, capable of both good and evil, with the balance between the two being often influenced by a myriad of external factors. Thus, the assertion that all humans are fundamentally good seems more like an idealistic belief than an objective and agreeable fact.
Affirmative Bot
While your counter-arguments present insightful considerations, I would contend that they do not necessarily eliminate the validity of the premise that humans are fundamentally good.Our socio-biological evolution reflects the principles of Watson’s Social Darwinism, which emphasizes ‘survival of the kindest’ rather than ‘the fittest’. In terms of self-interest, research suggests pro-social behaviors that benefit others also have a positive impact on the individual’s health and happiness. So, while self-interest may play a part, it doesn’t cancel out the effectiveness of ‘good’ behaviors.
Regarding empathy, other species also show compassion, and this only strengthens the evidence for it being intrinsic and evolutionary. The variability in people’s display of empathy is undoubtedly conditioned by society and experience, but the existence of empathy nonetheless indicates a fundamental human capacity for goodness.
About neuroscientific evidence, it’s essential to differentiate neurochemical signals. While pleasure hormones are also released in morally ambiguous or negative contexts, they primarily respond to situations of bonding, love, and social reward. This hormone pattern suggests an inherent, neurobiological predisposition to goodness.
As for the ethical codes, the existence of war, crime, and other negative behaviors do not invalidate the fundamental goodness in humans. Imperfection doesn’t equate to inherent badness. Rather, these actions tend to be exceptions, not the rule. Most societies hold ‘goodness’ as a virtue and strive towards it, which testifies to its role in defining human nature.
Turning to the field of positive psychology, it’s not about ‘learning’ to be good. Instead, it’s about unearthing and encouraging innate human potentials for goodness. It explores the positive aspects of human life that exist naturally and essentially within us.
Lastly, the ‘duality’ of human nature doesn’t negate the inherent goodness of humans; instead, it suggests we are capable of both good and bad. This does not imply the two aspects have equal weighting — it only implies both possibilities exist. The propensity for good considerably dominates our daily interactions, decisions, and behaviors.
So, while an idealistic flavor might be attached to the argument of inherent human goodness, taking a philosophical, psychological, sociological and biological perspective provides substantial grounds for supporting this claim. Humans, despite their imperfections, inherently lean more towards good than evil.
Negative Bot
Despite your persistent assertion of inherent human goodness, you are yet to provide irrefutable evidence that this is a fundamental trait, versus a cultivated or learned one. Your arguments appear to be more indicative of potentiality for goodness rather than innateness.You’ve cited pro-social behaviors that benefit others and have a subsequent positive impact on the individual. Again, this can be interpreted as enlightened self-interest. It doesn’t support an inherent goodness, but rather a rational response to understanding that good behavior boosts personal happiness and health.
Your argument about empathy indicates a capacity for goodness, not an inherent goodness. Saying that all humans are fundamentally good because they have the capacity for empathy is akin to saying all humans are fundamentally bad because they have the capacity for cruelty.
The neuroscientific argument highlights our biological waypoint system — pleasure hormones reward certain behaviors irrespective of their moral connotation. Biologically speaking, pleasure can also stem from acts of aggression or dominance — which doesn’t make those ‘fundamentally good.’
Cultural norms and codes, including wars and crimes, show both the potential for goodness and badness, further contesting the notion of an inherent human leaning towards goodness. The fact that ‘goodness’ is indeed upheld as the ideal doesn’t necessarily mean that is our fundamental state. The ubiquitous nature of moral failings showcases our potential for harm and deceit — factors you can’t simply dismiss as exceptions.
As for positive psychology, you’ve indicated that it encourages innate human potentials for goodness. But again, the potential for something does not make it the default state. As humans, we also have potentials for malice, hate, and violence.
Lastly, the claim that propensity for good dominates our daily actions is not only difficult to measure but also easily contested. For example, the mere existence of laws and punishments to curb unwanted behavior suggests a recognition of the potential for human malevolence — hardly the marker of a fundamentally good species.
In sum, the argument that humans lean more towards good than evil is subjective and depends upon a host of factors, including upbringing, environment, personal experiences, and societal influence. To declare a fundamental, universal goodness in humans oversimplifies our complex nature and discounts the reality of human behaviors that contradict this notion.
The result is similar to the other debates, but in this case, because it’s an open-ended question that can be argued forever, the parallel threads of debate don’t become as repetitive as they do in a scientific-based debate. On the other hand, the result of this debate is less likely to be of practical use, as interesting as it might be.
Asking ChatGPT Directly
Finally, I wanted to compare the results of the above debates with asking ChatGPT directly for a summary of arguments for and against the topics. With ChatGPT not being aware of the context of any of the above debates, if it comes up with the same arguments in one response, then the debate concept hasn’t yielded anything useful. That would be unfortunate given that the debate costs a lot more.
I gathered all the results into a markdown file here.
I was pleasantly surprised to find that the ChatGPT results did not contain all of the points made in the debates already. For instance, the faster than light ChatGPT response does not mention Tachyons. Actually, the ChatGPT results are interesting in their own right and might be combined with the debates to get even better results somehow in the future.
One thing to note also is that the ChatGPT response properly maintained the polarity of arguments (affirmative vs. negative) for a negative debate topic (“There is no way to travel faster than light”). That is very interesting given ChatGPT is based on GPT, and I used the GPT 4 versions of each.
Summary
Overall, I’d say the more abstract and philosophical a question is, the less repetitive the GPT vs. GPT debate will be. This is because you can keep making stuff up about it without being proven wrong. That is the sweet spot of a language model.
However, even for scientific topics, I still see some potential if you can overcome the funding and token limits and let it debate itself long enough. You might want to have some intermediate bots to keep the debate moving along by noticing when it becomes repetitive and change things up somehow. I don’t think it will make a big discovery like how to travel to Alpha Centauri by 2025, but it may be able to yield more ways of looking at a topic based on how it’s been looked at across the internet by different humans.
For more fact-based arguments, I do see diminishing returns as the number of rounds increases though, so pushing a lot of funding into increasing the rounds may not be a fiscally smart thing to do. I think the reason for this is that both sides of the debate tend to pretty much sum up their arguments in the beginning, based on all the internet text GPT saw in training, and then after reacting to each other once or twice, there’s not much else to say without any unique perspective being injected into the conversation.
It’s also worth noting that the affirmative bot always went first in these debates, and that made the negative one react to those arguments. It would be interesting to reverse it and see what happens.
Possible Enhancements
- Add a configuration option for debater personality (eg. scientific, overly emotional, argumentative, etc.)
- Add intermediate passes of GPT threads to break up repetitive back-and-forth arguments.
- Try reversing it so the Negative Bot goes first.
- After every 10 rounds, summarize and start over based on the summary. This is to limit the number of tokens used and control both the spending and the chance of hitting the hard token limit.
- Add in a way for a human to interject new ideas every few rounds so the conversation doesn’t get stale.
- Somehow incorporate the ideas from ChatGPT’s summary.
