“Rapidly Deployable Live Task Guidance” wins Best Demo Award at ISMAR 2020

Best Demo Award being announced at the ISMAR 2020 closing ceremony held in VR.

The International Symposium for Mixed and Augmented Reality (ISMAR) is the highest ranked academic conference in the fields of AR/VR/MR organized annually since 1998 (including its forerunner events, IWAR/ISAR and ISMR).

The biggest technical advances in the field from PTAM and KinectFusion to all of the works of Dr. Daniel Wagner who founded Vuforia were first presented at this conference.

ISMAR this year was held entirely in VR, from November 9 to 13, and presenters got the opportunity to talk about their research and stream their demos live through virtual booths and take questions from the who’s who in AR/VR/MR.

The winner of the best demonstration award this year, announced at ISMAR’s closing ceremony is the RetroActivity system which automatically builds computational models of a complex physical task, such as a maintenance activity, given only a handful of recorded demonstrations of the task. Once such a model is built, RetroActivity can finely track the job status from live video, to guide a worker through the task, provide independent training, and perform analytics.

Retrocausal, a startup backed by NASA Human Research Program, Incubate Fund US, and Techstars accelerator, based in Redmond, WA, develops the RetroActivity™ platform. According to them, RetroActivity enables ordinary AR developers to build AI-mediated feedback within hours, instead of requiring teams of specialized computer vision engineers who build temporal causation rules on top of 3rd-party visual recognition modules over months.

The key points of setting up RetroActivity are as follows:

1. An industrial engineer records a series of video demonstrations of an expert assembler with each version of the finished product to digitize the bill of process.
2. The video demonstrations are used for the AI software to learn a computational step-by-step model of the activity, which is ready to deploy within a few hours.
3. Then the system is ready to monitor a less experienced worker who is assembling a complex truck panel, a computer system, or any other complex machine.
4. RetroActivity software provides specific feedback in real-time to ensure that the assembly worker is following the correct procedure for the assembly of a specific version of a product.
5. In addition, the analytics dashboard is used by lean engineers to perform root cause analysis and continually improve processes.

ISMAR has made pre-recorded videos and 2-page abstracts of the 19 invited demonstration on their website: https://ismar20.org/demonstrations/