How can I find it? Jennifer Lopez in a Green Versace Dress or How an Image Search Engine Works
This blog forms a part of a series and aims to explore the foundations of the SMK Open. As a community builder interested in how technology transforms societies and culture, I am offering you my perspective on the project and what is happening behind the scenes (…or screens).
The heart of the SMK Open project is creating a solid digital infrastructure that people can build upon. To do that, we are actively cleaning and indexing the museum’s image database, so that everyone can later play around with it through an API. A number of interesting questions come up in the process: how do you index an image? How to make the indexed images accessible and easy to find? That is an interesting and important exercise, so let’s shed some more light on it — shall we?
An Image is Worth More Than a Thousand Words
Behinds the scenes, SMK Open is ordering and indexing thousands of already digitised pieces of art, and creating an Application programming Interface (API). This API will serve as the enabler of all interactions between the users and the museum’s vast online collection. The user experience will be facilitated and shaped by applications, web and mobile, created by the museum and by the community.
Some people like to joke about how the internet for many is just Google. Even if we account for the growing popularity of apps, especially mobile apps, it is still true that whenever we use desktop browsers, we end up using Google. We type in whatever we’d like to use, do or look at, and then an algorithm returns the answers. This is a basic way in which we look for information online. But when it comes to images, things work a little differently.
Translating Words to Numbers, and Numbers to Images
It all started in the year 2000. Back then, there was no image search tool online. Google search was limited to text and links. But then, Jennifer Lopez wore the exotic green dress by Versace… People couldn’t have enough of it and “Jennifer Lopez green dress” quickly became the most popular search query. But there was no way to search for the image online and so, Google developed the image search tool.
This kind of search, a search for images is one of the most important tech features being built for the SMK Open.
The Image search started with a words query being matched with the image index data — the keywords for the image search based on the filename of the image, the link text pointing to the image, and text adjacent to the image. It is changing now. How?
First of all, some programs allow crowdsourced indexing (e.g. you allow the community to tag the image independently to the basic index data and add even more words to the description). Secondly, as we continue to communicate and think in an increasingly visual way, new, more visual ways of search are being introduced. One of them is a popular Shutterstock feature (below), where you can search for images of a certain colour using the spectrum option.
Another one is the Reverse Image Search. It is a content-based image retrieval (CBIR) query technique that involves providing the CBIR system with a sample image that it will then base its search upon. In particular, reverse image search is characterised by a lack of search terms. This effectively removes the need for a user to guess keywords or terms that may or may not return a correct result. Reverse image search also allows users to discover content that is related to a specific sample image, popularity of an image, and discover manipulated versions and derivative works.
SMK Open will provide anyone interested with a clean and well indexed database of over 20 000 digitised, high-res and public domain images, and an API. It means that whether you’d like to just find and access the images, discover new things (like even more images of woods and forests) or use the collection for your own purposes, e.g. in an app — the path for you will be paved.
If it is There, I can Find It
The culture of interacting with images as well as the way we think about them is deeply affected by the technological tools we use. It has thus become increasingly important for the art collections to be available to users in places and in ways they look for them. After all if art is supposed to be relevant, it has to be accessible. And accessibility can mean different things at different times.“If it is not in the internet, it doesn’t exist” is an old phrase now. “If it is there, I can find it” is the new norm. Or, while we are on about dresses, a new black.
This blog post has been inspired by a talk with Nikolaj Erichsen, the SMK’s Senior System Developer who has been working on the SMK Open project for the past 2 months.
