Developers’ Diaries. Issue #3

Cappasity mobile app — Easy 3D Scan for mobile devices

Since the intensity with which society switches from stationary to portable devices has been gradually increasing over time, the mobile application development has become a crucial task for our company.

While the Easy 3D Scan desktop application has been handling pre-recorded video files, our mission here is to combine recording and processing in one app and perform all the calculations on one tiny portable device. The latter goal leads us to an idea that this is going to be a whole different story.

The key distinctions between the mobile and desktop application development are as follows:

• the very essence of the application is to loosen the restrictions imposed on the users and their actions. Namely, when the users record videos with their mobile devices, their hands usually tremble, the cameras constantly lose focus, etc. That’s why when developing mobile apps the developers must take into account the specifics of producing videos with the mobile phones and be able to handle it;
• whereas the video is recorded via the mobile camera, one should not expect high quality of the input data in the first place;
• regarding the fact that all computational resources are limited by the user’s mobile device capabilities, the usage of costly algorithms which could compensate the initial difficulties becomes impossible.

Thus, the challenge is to find a solution that balances between the generalization of the input data and the lack of resources.

How does the app work?

The app takes a video recorded by a user via their mobile device as an input. The video should contain at least one complete object rotation. The object must be placed in the center of a white paper sheet with two perpendicular straight lines drawn on it (we will clarify soon why this is necessary).

The application uses the provided data to create a 3D View file format. During the post-processing, the app removes the background (namely, the paper sheet and the exterior), while the object smoothly and stably performs a full 360-degree rotation in the center of each frame, similarly to the Easy 3D Scan desktop app. The final step is converting the processed data to our cloud format.

Here is how the app is programmed:

Firstly, it is necessary to determine the rotation angles for each frame of the input video. To do that, we need the simplest marker — two lines drawn on the paper sheet. Our algorithms detect them on each frame and identify the rotation angles.

Secondly, to make the object position stable, we need to place the first of the final frames so that the object is in the center of the frame and then minimize the distance between the contours of the object for each pair of consecutive frames.

Finally, we need to detect the background. To succeed in this, we use our paper detection algorithm and object contours information .

However, similarly to any other journey, there are always new obstacles to overcome and dragons to vanquish.

Stay tuned!

You can find the current version of Cappasity’s roadmap here — https://trello.com/b/z4OZeLKW/cappasity-roadmap

We know that you can’t wait to test our applications! We are preparing to release an update for Cappasity iOS app.

We invite you all to become the first users of our Cappasity app and help us test it carefully before the official release.

We would be interested to know about how you shoot and what problems you encounter while using the app. To do this, you need to subscribe to our private list, have an iPhone 6 or better and be ready to upload several files via a file sharing service such as Dropbox. Thanks for the help!

We collect applications by April 11 and plan to send out an invitation to TestFlight on April 13th.

Subscribe form — http://eepurl.com/dp1wbr

***

We would be interested in receiving your feedback on the relevant material in our Telegram channel — https://t.me/artoken ! Please voice your opinion about shortcomings and things you would like to know.

We have a suggestion — how about you ask us questions on topics of interest to you in the Telegram channel? We will collect them and later reply to them in the new issue of Developers’ Diaries.