01/02/2017 | Posted in:News
Augmenting awareness by HoloLens: initial exploration
Author/Institution: Jouke Verlinden/TU Delft
This video is a first exploration on how Augmented Reality can be used in production planning and during deployment & operation. It is part of Deliverable 4.4.
- HoloLens: Although we considered projector-based and non-visceral augmentation, the use of a stereoscopic head-mounted display allows to provide better visual (and auditory) feedback. Specifically, if considering a portable solution, being able to be used during the workflow of the Factory in a Day process. In particular, see-through options allow a proper overlap with the environment, and with the emergence of the Microsoft HoloLens and the Meta 2 glasses are enabling these technologies in a proper, ergonomically fitting way. The scenario on motion path and training provides an indication how system state can be overlayed in the physical environment.
- How do you plan to use feature 1 of HoloLens in Factory-in-a-day (scene reconstruction)?
Scene reconstruction is essential in rendering occlusions, which raises the effect of Presence (notion as ‘being-there’). For example, at 2:10 in the video, only a part of the digital objects is rendered as a wall blocks the view. Another important aspect is that this enables a computational integration between spatial context, robot systems, and workers, and can be used to reason on situational awareness of individual workers (in terms of visibility, system status etc.).
- How do you plan to use feature 2 of HoloLens in Factory-in-a-day (object placement)?
Together with feature 1, this option shows how current gesture recognition is done with the HoloLens (so-called “Air tap” and gaze following). Furthermore, these features could be used during maintenance or configuration of robotic systems.
- What are the next steps? This is a first impression based on off-the shelf software. Connectivity with ROS and assessing usability of specific cues will be done during the coming 10 months.