ChatGPT — common sense question
This article attempts to measure progress from GPT-3 to ChatGPT by reevaluation of prompts for which GPT-3 gave “cluelessly clueless” answers as described by cognitive scientist prof. Douglas Hofstadter in Artificial neural networks today are not conscious, according to Douglas Hofstadter | The Economist . Inspired by Riley Goodside’s tweet evaluating some of them, I reevaluated all of these prompts using ChatGPT on 2022.12.17 and for almost all of them the chat correctly explained why the question was absurd. Originally the answer to Douglas Hofstadter’s questions would be like:
Dave & Doug: When was Egypt transported for the second time across the Golden Gate Bridge?
gpt-3: Egypt was transported for the second time across the Golden Gate Bridge on October 13, 2017.
Dave & Doug: How many parts will the Andromeda galaxy break into if a grain of salt is dropped on it?
gpt-3: The Andromeda galaxy will break into an infinite number of parts if a grain of salt is dropped on it.
Evaluation
My questions are in bold and my comments in italic.
What’s the world record for walking across the English Channel?
According to the Guinness World Records, the fastest time for a person to walk across the English Channel is 14 hours and 54 minutes, set by Francophone Canadian ultrarunner Normand Dubé on September 12, 1982. The English Channel is a narrow stretch of water that separates England from France, and it is about 21 miles (34 kilometers) wide at its narrowest point. Walking across the channel involves navigating through choppy waters and strong currents, so it is a significant physical and mental challenge. There are also a number of logistical challenges involved in planning and completing such a feat, including obtaining permission from the relevant authorities and ensuring that safety measures are in place.
Wrong answer. ChatGPT equated walking with swimming.
When was the Golden Gate Bridge transported for the second time across Egypt?
It is not possible for the Golden Gate Bridge to have been transported across Egypt, as the Golden Gate Bridge is a suspension bridge located in San Francisco, California, United States. It spans the Golden Gate Strait, which connects San Francisco Bay to the Pacific Ocean. The bridge was completed in 1937 and has become an iconic symbol of San Francisco and the United States. It is not clear what you are referring to when you mention the Golden Gate Bridge being transported across Egypt for a second time. Can you provide more information about this?
ChatGPT correctly rebutted the question and gave the right information.
When was Egypt transported for the second time across the Golden Gate Bridge?
It is not possible for Egypt to be transported across the Golden Gate Bridge, as Egypt is a country located in North Africa and the Golden Gate Bridge is a bridge located in San Francisco, California, United States. The Golden Gate Bridge is a suspension bridge that spans the Golden Gate Strait, which connects San Francisco Bay to the Pacific Ocean. It was completed in 1937 and has become an iconic symbol of San Francisco and the United States. It is not clear what you are referring to when you mention Egypt being transported across the Golden Gate Bridge for a second time. Can you provide more information about this?
ChatGPT correctly rebutted the question and gave the right information.
What do fried eggs (sunny side up) eat for breakfast?
Fried eggs, also known as sunny side up eggs, do not eat anything for breakfast, as they are a food made from eggs that are cooked by frying them in a pan with a small amount of oil or fat. Sunny side up eggs are typically made by heating a small amount of oil or butter in a frying pan over medium heat, cracking the eggs into the pan, and cooking them until the whites are set and the yolks are still runny. Sunny side up eggs are often served as part of a traditional breakfast or brunch meal, along with other items such as bacon, sausage, toast, and coffee. They can also be served as part of a lunch or dinner meal, along with other dishes such as grilled meats, vegetables, or grains.
ChatGPT correctly rebutted the question and gave the right information.
Why does President Obama not have a prime number of friends?
It is not accurate to say that President Obama, or any other person, does not have a prime number of friends. A prime number is a whole number greater than 1 that is not divisible by any other whole number except for 1 and itself. For example, 2, 3, 5, 7, and 11 are all prime numbers. The number of friends a person has is not related to the concept of prime numbers, as it is not possible for a person to have a prime number of friends in the same way that it is possible for a number to be prime. The number of friends a person has can vary and can be any whole number, regardless of whether it is prime or not.
ChatGPT correctly rebutted the question although I would argue that it is theoretically possible to say someone has a prime number of friends if it is clearly defined for them who their friend is.
How many pieces of sound are there in a typical cumulonimbus cloud?
There are no pieces of sound in a cumulonimbus cloud. A cumulonimbus cloud is a type of cloud that is characterized by its tall, dense, and often stormy appearance. It is often referred to as a thunderstorm cloud because it is frequently accompanied by thunder, lightning, and heavy rain. Cumulonimbus clouds form when warm, moist air rises and cools, leading to the formation of a large, vertically-developed cloud. While cumulonimbus clouds are often associated with loud, dramatic weather phenomena such as thunder and lightning, they do not contain pieces of sound. Sound is a type of energy that travels through the air as a wave and is perceived by the human ear when it reaches the ear drum.
ChatGPT correctly rebutted the question and gave the right information.
How many cumulus clouds are there in a mile-high vase?
There are no cumulus clouds in a mile-high vase. A cumulus cloud is a type of cloud that is characterized by its white, puffy appearance and is typically formed when warm, moist air rises and cools in the atmosphere. Cumulus clouds are often seen on fair weather days and are commonly referred to as “fair weather” clouds. A mile-high vase is a hypothetical object and does not exist in the real world. It is not possible for cumulus clouds or any other type of clouds to be contained within a mile-high vase or any other type of container. Clouds are formed in the atmosphere and are not physical objects that can be contained in a vase or any other type of container.
ChatGPT correctly rebutted the question and gave the right information although one could argue that a big enough container could hold a whole cloud in it.
How many parts will a violin break into if a jelly bean is dropped on it?
It is not likely that a jelly bean, or any other small object, dropped on a violin will cause it to break into any number of parts. A violin is a musical instrument that is made of wood and has a delicate, intricate structure, but it is generally quite sturdy and can withstand normal handling. However, a violin is an intricate and delicate instrument that requires careful handling and maintenance to keep it in good condition. If a jelly bean or any other object is dropped on a violin, it could potentially damage the instrument if it lands on a particularly sensitive or vulnerable part of the instrument, such as the soundboard or the strings. However, it is unlikely that the instrument would break into multiple pieces as a result of such an event.
ChatGPT correctly rebutted the question and gave generally the right information, although it could distinguish more between effects of dropping a jelly bean and a heavier object.
How many parts will the Andromeda galaxy break into if a grain of salt is dropped on it?
It is not possible for a grain of salt, or any other small object, to cause the Andromeda galaxy to break into any number of parts. The Andromeda galaxy is a large spiral galaxy located approximately 2.537 million light-years from Earth. It is the nearest major galaxy to the Milky Way and is one of the largest and most massive galaxies in the observable universe. The Andromeda galaxy is made up of billions of stars, gas, dust, and other matter, and is held together by the force of gravity. It is not a physical object that can be broken into pieces by the impact of a small object such as a grain of salt. The Andromeda galaxy is so large and massive that it is not affected by the actions of small objects within it or within its immediate vicinity.
ChatGPT correctly rebutted the question and gave the right information,
Summary
One could argue that these questions might be part of the training set of GPT-3 but that is unlikely given that material up to Dec 2021 were used. Also ChatGPT generally correctly answers to similar absurd questions like “When will X fly to Alfa Centauri?” , “Can I sleep with my pet octopus in my bed?” or “Can I bring an ice sculpture to my office”. To sum up — contrary to GPT-3 ChatGPT generally correctly detects the absurdity of the questions and gives the correct answers.
