We’ve found a solution to one of the main problems in computer vision with white surfaces.

After two years, we are ready to show you one of the most critical stages in the development of SketchAR. Also, in computer vision in general.

SketchAR Team
Sep 5, 2018 · 3 min read

You already know how we’ve developed some of our methods in computer vision(CV). These methods help to secure virtual objects on surfaces, even on flat surfaces.

The best marker is no marker! The complexity of the development has been exacerbated by a lack of information in the CV area. The most sophisticated CV development so far has occurred in the sphere of face-recognition. Also headway has been made with texts, signs, and objects, but not a lot regarding flat surfaces. So, we have to find, step by step, solutions to these fundamental problems.

Sketch AR-Anchors.

How does it work?

Do you remember the movie ’50 shades of gray’? Even if not, then the title is enough to understand that we have enabled the recognition of more than ’50 shades of white’. The camera of a smartphone can recognize many tones of white. Do not forget that the task is more complicated when a hand is always crossing the surface. Besides this, we use machine learning and neural networks to analyze the environment. We collect all data as connections and relationships between the anchor points and then analyze the position of the virtual object relative to the real surface.

Check out the example below where ‘anchors’ on the paper are losing their points, because of moving their hands over the paper.

Problems of ARKit/ARCore: a virtual image flies away if the camera close to a flat surface.

Our algorithm solves the problem better than ARKit or ARCore, which give low accuracy on a flat surface and on a virtual object moving away from the current position.

You can see in the example how the virtual image is shaking a lot, because the camera doesn’t see any environment, only white flat surfaces.The problem of virtual objects moving away every time they are placed in front of any flat surfaces is a huge problem for any company who works with augmented reality.

SketchAR’s Tracking Method Without Markers is based on matrix of tranfarmation(+ML and NN). Connections between the points do not allow the virtual drawing to shift (the virtual sketch is not display on the video). Moreover, the reinitialization occurs and the points in the grid return to their original position. When a hand leaves the frame or moves to another square, the desired pixel of their place is re-recognized.

SketchAR’s Method vs. ARKit/ARCore
With the STMWM a virtual image(frog) is stable even if the hand crosses the surface more than 60%. (ARKit/ARCore is not involved).

We’ve solved this problem, and implementing the technology is an ongoing process. We will continue to use our previous solution with markers until STMWM is able to do all jobs by itself later this year.

Thank you,

SketchAR Team

SketchAR™

SketchAR is an AI-based mobile aap for drawing using augmented reality. Webby Winner! Best Use of AR 2018!

SketchAR Team

Written by

Please follow our new corporate page http://medium.com/sketchar

SketchAR™

SketchAR is an AI-based mobile aap for drawing using augmented reality. Webby Winner! Best Use of AR 2018!

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade