Discoveries from Search generative AI (SGE) tests
Google’s new generative AI responses are a fascinating development that has the potential to revolutionize the way we search for information. However, as with any new technology, there are some kinks that need to be worked out.
First impression
- Available to only US-based users for now. You can join the waitlist through https://labs.google.com/search/install using your Google account. Once you are off the waitlist, you can connect to a USA VPN and try the experiments.
- I could see generative AI responses so far only on the desktop for the queries that I tried. Mobile for any of them is not seen.
- The top of the desktop search results page is now occupied by generative AI responses. If you are not satisfied with the response, you are encouraged to ask follow-up questions. This takes you to the “Converse” tab, which is outside the search results page. It is very unlikely that you will return to the search results page from there.
4. The thumbnails for website links are visible, but it is unclear how they are ranked in that space.
5. One website that is currently ranking as a featured snippet is not included in these thumbnails.
6. One link that I clicked on from the right side had a good page speed, which indicates that LCP and CLS are important.
7. The generative response is not only generated by AI, but it also takes some information from the websites that appear in the right-hand side links.
8. I could see up to 8 links on the right side next to the generative AI response, which clearly indicates 8 search results.
9. I did not see any ads for any of my queries. Google may have excluded ads from this experiment for now to avoid panic and confusion.
10. For mobile, “perspective” is available as one of the tags shown right above the search results. See below
https://www.awesomescreenshot.com/video/18520708?key=8ae27ee27ed0f3136d2a50e82e7ae52d
11. The GAI response is kind of correct and sufficient to solve the user’s query/intent. Just thinking aloud, as a user, if it solves my query and helps me decide with the next step, why should I click any of the links? For example, a user searching for “best gifting options for an introvert”, AI gives a long list of options to choose from. It is even giving me tips on making the person feel better. See below
12. AI responses are not always comprehensive. In some cases, they only provide a basic overview of a topic, and they may not answer all of the user’s questions. In the following video, for instance, let’s say I am a student contemplating between studying in Canada or the UK. The AI responded to the query, however, it did not provide any specific information about why these countries would be a good choice, and it failed to mention that there are many other great places to study abroad. I want to know a comparison of accommodation. That AI was unable to solve so in this case I might end up clicking one of the results or even changing my query.
https://www.awesomescreenshot.com/video/18521060?key=cce3403425a6b0ab131be66c96f16484
13. In some cases, it contradicts its own responses. See the 2 screenshots below for the similar queries.
Question: should we really trust the responses in such scenarios? They seem misleading.
Overall, Google’s new generative AI responses are a promising development, but they are not without their flaws. It is important to use these responses with caution and to be aware of their limitations.
Here are some tips for using Google’s generative AI responses:
- Be aware of the potential for inaccuracies.
- Do not rely on the AI responses as your only source of information.
- Marketers may use AI responses to generate a list of topics to research further.
P.S. I’ll be adding more observations as I continue to do the experiments.
