When search engines learn to look beyond words
Reconstructing reality in three dimensions is how the search engine of the future works.
The same technologies used to keep self driving cars on the road, detect faces in Facebook pictures, and add AR emoji to a Snapchat can be used to scan an environment and build a knowledge base in real time. With three dimensional information about our environment coupled with machine learning and cloud based AI, search can provide us answers relevant to our needs. Search will use our environment and historical data to make connections and offer us solutions in context to our location and goals.
The technology to accomplish these tasks exists today and need only be connected together into a cohesive whole. Stefen Weitz expands upon this line of thought in his book Search: How the data explosion makes us smarter.
Computer vision and light field imaging are the foundations of a system that can scan and recreate a digital version of reality in near real time.
Digital 3D graphics can be used to fill in the gaps behind the purview of a camera, so a user can lean side to side and still see the three dimensions of an image. A user could use a head mounted display to project their viewpoint into any space or time. Cameras around the world become portals to three dimensional places we can visit. All the sudden we can travel through a projected reality from the comfort of our homes, going anywhere and experiencing anything.
Google street view does this today to a degree by reconstructing rough three dimensional copies of reality from satellite and Google Street View data. Couple this with the VR/AR technologies being developed, as well as real time sensor data and we can see just how close we are to making the future of three dimensional search a reality.
The Japanese anime Accel World explores the technology of recreating and building three dimensional copies of reality using sensor data from nearby devices. When a character in Accel World uses “burst link”, their consciousness is accelerated 1000X in the virtual world. Which in this case is a 3D reconstruction of reality wherever they were standing at the time of using “burst link”, effectively freezing time and allowing the user expanded space to analyze decisions and actions before having to take them. Their 3D avatar stands next to the recreation of their real body.
User profiles are built based on our habits.
Just as today’s text based search engines and two dimensional social media affect public perception, the three dimensional future will too. Today advertisers attempt to profile us based on past purchases and search terms, but what happens when they have not only our historical data, but real time information of our environment? Imagine Amazon’s suggested items feature while you’re walking down the isle of a Walmart super-center.
The search AI will scan the environment and use the objects around it to build up a list of actions it can take to help us on our path to goals we have set(or goals the AI has set for us). Helpful behavioral nudges can steer users towards healthier behavior. Of course this same technology can be used by marketers and governments to nudge user decisions towards profitable or nefarious ends.
Call it escapism, or call it science.
Three dimensional reconstruction of reality will be a means for us to explore dangerous worlds unfit for humans just by sending robots who can scan the environment and reconstruct the worlds for us. We can use this new interface to explore the world we live in today, or infinite worlds built by imagination. We can test assumptions and learn how to make the world better day by day without wasting physical resources, or escape from reality entirely.
The search engine of the future will understand far more than words.
Context, location, three dimensional information, historical trends, personal preferences, behavioral biases, and more… all used cohesively in a single tool built to help humans understand and improve reality. Think of all that has been made possible thanks to the text based search engines of today, and imagine what will be possible when search learns to look beyond words.
