Zappen Is Open Source Visual Search
Best of all, it actually works.
There’s a scene in the most recent season of Silicon Valley where the characters are holding their breath during a mobile image recognition demo. If you’ve ever worked with this set of technologies, you know the feeling. You also know the wow factor — the oohs and ahs when the app gets it right. Zappen truly is something you have to “see” for yourself.
My company didn’t set out to make “Shazam for pictures.” If we had known how many larger, better-funded companies had tried to implement this functionality and failed, we probably would have left it alone. But we got really frustrated with QR codes. Google’s proprietary solutions were out of our price range, so we thought we’d try a homegrown approach.
You can see the results for yourself. Download the app and take the tour, or upload your own images and visual links. I don’t usually like to self-promote on this forum, but when you build something really cool and you’re giving it away for free, you kind of have to.

You can find the repo on Github to build your own mobile app and Pastec image database. Our “wish list” includes adding machine learning and integration with a decentralized DNS architecture such as EmerDNS to form a global “Visual DNS” system where images take the place of words. We have tested database accuracy and response time to 100,000 images, and are working on scalability testing to hit the 5 million mark before year end. The technology has a 98.3% success rate under lab conditions. In real world situations, I’d say it works at least as well as a QR code.
Much of what we did was simple fine-tuning — improving autofocus control for Android and iOs devices and playing with post-capture processing and filters. Having a strong graphics background was probably helpful there; much of what can be done in Photoshop can be approximated with Open CV. The great thing about free and open source software (FOSS) is that it makes the process of innovation much more accessible, because you are building on the work of others. Without Pastec / ORB, OpenCV, and the visual words algorithm, there would be no Zappen.
Augmented reality and visual search are well known buzzwords, but for me Zappen is a way to bring the virtual world back into alignment with brick and mortar spaces. Scan a logo, a business card, or a sign and connect to the website of your choice. Use it for billboards. Use it for art galleries. Use it for protest signs and organizing in public spaces. We’re not telling you what to do with it. We’re just putting it out there. Fully functional.
Discogs, a champion of open source technology, will integrate Zappen visual search technology into their popular record trading app early next year.
We believe cool tech should not be hidden behind proprietary API’s. Because as many uses as we can imagine for Zappen, we are sure that some of you can imagine better ones.
