We’ve found a solution to one of the main problems in computer vision with white surfaces.

After two years, we are ready to show you one of the most critical stages in the development of SketchAR. Also, in computer vision in general.

SketchAR Team
Sep 5, 2018 · 3 min read

You already know how we’ve developed some of our methods in computer vision(CV). These methods help to secure virtual objects on surfaces, even on flat surfaces.

The best marker is no marker! The complexity of the development has been exacerbated by a lack of information in the CV area. The most sophisticated CV development so far has occurred in the sphere of face-recognition. Also headway has been made with texts, signs, and objects, but not a lot regarding flat surfaces. So, we have to find, step by step, solutions to these fundamental problems.

Sketch AR-Anchors.

Image for post

SketchAR Tracking Method Without Markers is able to detect flat surfaces. Moreover, it remains steady even in extreme conditions, such as the camera not seeing the environment clearly when zoomed in and only looking at a white surface.

How does it work?

Do you remember the movie ’50 shades of gray’? Even if not, then the title is enough to understand that we have enabled the recognition of more than ’50 shades of white’. The camera of a smartphone can recognize many tones of white. Do not forget that the task is more complicated when a hand is always crossing the surface. Besides this, we use machine learning and neural networks to analyze the environment. We collect all data as connections and relationships between the anchor points and then analyze the position of the virtual object relative to the real surface.

Check out the example below where ‘anchors’ on the paper are losing their points, because of moving their hands over the paper.

Image for post
Problems of ARKit/ARCore: a virtual image flies away if the camera close to a flat surface.

Our algorithm solves the problem better than ARKit or ARCore, which give low accuracy on a flat surface and on a virtual object moving away from the current position.

You can see in the example how the virtual image is shaking a lot, because the camera doesn’t see any environment, only white flat surfaces.The problem of virtual objects moving away every time they are placed in front of any flat surfaces is a huge problem for any company who works with augmented reality.

SketchAR’s Tracking Method Without Markers is based on matrix of tranfarmation(+ML and NN). Connections between the points do not allow the virtual drawing to shift (the virtual sketch is not display on the video). Moreover, the reinitialization occurs and the points in the grid return to their original position. When a hand leaves the frame or moves to another square, the desired pixel of their place is re-recognized.

SketchAR’s Method vs. ARKit/ARCore
Image for post
With the STMWM a virtual image(frog) is stable even if the hand crosses the surface more than 60%. (ARKit/ARCore is not involved).

We’ve solved this problem, and implementing the technology is an ongoing process. We will continue to use our previous solution with markers until STMWM is able to do all jobs by itself later this year.

Thank you,

SketchAR Team

SketchAR™

Effective new way to enhance creativity using AI+AR

SketchAR Team

Written by

Please follow our new corporate page http://medium.com/sketchar

SketchAR™

SketchAR is an AI-based mobile app and a platform for developing people’s creativity through the unique interactive approach of AR drawing, photo editing, and gamification.

SketchAR Team

Written by

Please follow our new corporate page http://medium.com/sketchar

SketchAR™

SketchAR is an AI-based mobile app and a platform for developing people’s creativity through the unique interactive approach of AR drawing, photo editing, and gamification.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store