Using ML to “Understand” Images

We’ve become familiar with using Large Language Models (LLMs) to help us “understand” the contents of text documents or to search for documents or pages that may contain text that is relevant to a question that we ask. This has gone beyond “keyword search” to more of a “semantic search” — searching for content that has the same meaning as what we are asking about, not just content that has the same words as what we ask.

What if we could do the same for images? What if we could take a picture and find other pictures that were “similar” to ours, the same way Google Lens does…