Making Unsplash’s submission system more transparent
A few weeks ago, some Unsplash community members noticed a new notification for their photos letting them know that their recent submission was made searchable. This notification was new, but the underlying system where a photo is made searchable is not new.
What happened next was a discussion around how the submissions system on Unsplash works and why certain photos get a lot of distribution and others do not.
We believe heavily in transparency in all things Unsplash, and after reflecting, we realized that the current system does not live up to our standards for transparency and the standards that our community expects of us.
Before going over the changes we’re making to the system to be more transparent, it helps to understand why the system works the way it does and how it evolved to get there.
From side-project to community 👨👩👧👦
To understand how we got to now, we have to look back at how Unsplash began.
We initially started Unsplash with ten left-over photos from our own photography shoot. There was no search, no editorial feed, no collections — Unsplash was just a little Tumblr blog focusing on beautiful free photos.
Despite this, almost immediately, a community built around Unsplash and photographers began submitting their own photos, competing to be released as one of the 10 photos selected every 10 days. Initially, we handled this process via email but that quickly grew to be unwieldy, and so the first real product features of Unsplash were built: a small uploader and a queue to release the photos in collections of 10.
Each week our team would go through and pick ten of the best photos to be shared with the creative community. This selection process became one of Unsplash’s defining features: unlike previous free photo communities like Flickr and Creative Commons, Unsplash photos were always high-quality and useful.
For the next year, we continued with this format until eventually we hit a new problem: the queue to release photos was growing larger and larger as more great photos were submitted. We wanted to release all of the photos to be freely downloaded, but we also needed to preserve the quality of the photos.
This is a difficult problem to solve though, as photography, like all art, exists on a spectrum of quality. Our community expected great quality photography, but the submissions varied heavily in quality.
After much internal debate, we decided on two new additions to Unsplash: a new feed (creatively called the ‘New’ feed) to showcase the best, curated new photos, and profile pages for every photographer, so that every photo that met the basic submission guidelines could be shared and downloaded.
We saw this as a win-win for the community as the new feed could preserve the quality and utility that made Unsplash popular, while the profile pages could ensure that all photos that met our submission guidelines could be shared openly. While we couldn’t guarantee any distribution for those photos, we could give them the same tools to allow them to be hosted and shared with their own following.
Building search 🔍
Shortly after building these features we released another feature: search.
Initially, search was scoped depending on the feed you were searching. This meant that you could search profiles, the new feed, or just the ‘10 every 10 days’ collections.
Eventually though, we noticed that these scopes weren’t being used and the community were only searching the ‘new’ feed photos.
As we redesigned search and moved it to be a global search, we were faced with the problem of limiting the searches to just the highest-quality photos or all photos uploaded to Unsplash
To help us decide what would be best, we experimented with both and ran two versions of search:
1. search across the entire library
2. search only photos that meet a minimum quality threshold
Immediately, our support inbox flooded with emails from members using version 1 of search, complaining that Unsplash was now full of mixed quality photos and refusing to use the site after many years of loyalty. While we thought this was a little over the top, the data clearly showed that mixed quality results heavily reduced the number of downloads from search, despite offering more results.
Sometimes more isn’t better.
This confirmed a core principle of Unsplash: curation is important. So we moved forward with version 2, searching only photos that were promoted to the ‘New’ feed.
As the number of photos submitted daily to Unsplash continued to grow, we ran into a new problem: the number of great photos being submitted were too numerous to feature in the ‘New’ feed daily.
To get around this, our editorial team discovered a hack in the way photo states had been built and found a way to add a photo to search without promoting it in the new feed. Many of these photos were focused on pure utility: photos that weren’t necessarily ‘beautiful’ but due to the need for the topics, they were heavily downloaded.
After a few months of doing this, our API team patched the ‘hack’ and turned it into a real feature, allowing the editorial team to feature photos separately in search and in the editorial feed.
Around this time, in light of this new state of photos, our API team re-ran the old search experiment, with three versions of results:
1. Everything in the library
2. Photos from the new state
3. Photos from only the ‘promoted’ state
As we expected, version 2, where the photos with the new state appeared, performed the best. From version 1, where all photos were visible regardless of state, we again received a significant number of complaints and a heavy decrease in downloads, as the mix of results quality caused significantly more users to abandon their sessions.
This leads us to the current state of the search system today: photos are reviewed and can be approved, added to search, and promoted in the Editorial feed separately.
Where we went wrong
Amid everything going on in building a company and community, it can be hard to find time to reflect on systems as they evolve. Our submissions and search system weren’t ‘designed’ like some of our other features — they evolved through hundreds of small iterations based on the ‘next most logical step’.
We believe heavily in transparency. We also believe in curation. Squaring those two can be difficult at times. Looking back, it’s clear that the system that developed wasn’t as transparent as it should be.
What we’re changing
We know that search must rely on a minimum quality and utility threshold, as experiments have repeatedly validated that our community doesn’t want to search through thousands of mixed quality results each time they search.
To address this we’ll be splitting search results into two sections: a high confidence section and a less confident section.
In the less confident section, we’ll include results that have a lower relevancy score. This will translate into photos that might match the search query or are less used (indicating that they may not be as high quality). By default, the results will only show the high confidence section. For searchers who want to see all results, they’ll be able to click a button or toggle to show the lower confidence results.
Photos that outperform their expected use for their relative position in the results will increase their relevancy and move out of the ‘less confident’ section. This is very similar to the process used to rank results in other search systems (like Google and Pinterest), as it’s possible to predict the mathematical relationship between the position in a search list and the expected engagement.
Updating our system to do this isn’t something we can do overnight. It’s only even become theoretically possible with some of the recent changes we’ve made to update our search system, which is why we never considered it as an option when originally designing the states system.
Until we build this, we’ve updated the ‘Manage photos’ screen in contributor settings to show the ‘state’ of a photo. This at least makes it clear whether a photo appears in search, approved, or in review.
Once we update search to reflect the new system, we’ll remove the ‘searchable state’ as all photos will be searchable.
Just as curation and quality are an important part of Unsplash, transparency and feedback are also. In retrospect, we recognize that the system we created wasn’t as transparent as it should be and we’re sorry for that.
We’re open to any questions about the system and will do our best to answer them — leave them here or hit me up on Twitter. Similarly, if there are other parts of Unsplash that feel opaque, we’d appreciate knowing so that we can make them more open.
