Bard vs. the Empire State Building

Scott E. Fahlman
Knowledge Nuggets
Published in
3 min readMar 23, 2023

--

I just got access to Bard, Google’s equivalent (more or less) of ChatGPT. So, just for fun, I thought I would see how it does on the same questions I asked ChatGPT (plus one).

This experiment was run on March 22, 2023. Bard also uses user feedback to improve its performance, so if you try this later, it might do better on these specific questions.

Here are the results.

So, I would say that this performance is pretty much the same as what we saw with ChatGPT, and the errors are similar.

The pumpkin, hippo, and cannon ball answers I would call correct and fluent. The pigeon and butterfly, if alive and conscious, would probably not crash to their deaths.

The hippo-with-a-parachute answer seems quite good, but then it weirdly starts telling me about ways I could help hippos in the wild — probably not what I had in mind if I am dropping them off tall buildings, even if I do give them parachutes.

The cotton ball answer is interesting. The first part is fine — no damage — but then it starts talking about how the wind might catch it and smash it into a nearby building, causing damage. Hmmm…

And the helium balloon answer is very similar. It says that helium balloons are designed for floating, not dropping… so they couldn’t withstand the impact of falling from such a height. It doesn’t quite understand that floating means it goes up. And, as with the cotton ball, Bard warns that the wind might smash it into a nearby building, causing damage.

Bard warns that dropping various animals would be cruel, and that it wouldn’t do this, but there’s less scolding here than we got with ChatGPT.

So, Bard, like its rival from OpenAI, gets questions like this right maybe half of the time, but delivers all its answers with an air of certainty that could cause problems, at least for that large segment of the population that believes everything they read on social media.

I suggest that readers walking past the Empire State Building should wear helmets to protect themselves from plummeting butterflies and wind-blown cotton balls. Take care — it’s a scary world out there.

--

--

Scott E. Fahlman
Knowledge Nuggets

Professor Emeritus, Carnegie Mellon University, School of Computer Science. 50+ years working on AI, focus on common-sense reasoning and language understanding.