AI-powered mobile AR work instructions

We have been working on a prototype of the AI-powered mobile AR work instructions.

The victim is, again, the glorious Badge Machine.

Used prompt engineering to direct the AI's attention to the specific objects, orientations, and conditions relevant to a machine or process

Wins and gaps:

- needs better AI-context for the rotating table alignment;

- isn't checking the lever-pulling;

+ no custom model training required.

+ detecting almost transparent plastic film (analyzing reflections on the surface) surprisingly good.

+ detecting the Pin Back surprisingly good.

Built with Flutter and Gemini, runs on Android — same stack could work on AR glasses. We are waiting for the upcoming XReal Project Aura AR glasses, which should solve this bottleneck. You can find more information about this project www.vrtik.al/ar-training-simulators. Full video on Youtube:

.