01/02/2017 | Posted in:News


HoloLens used in the Factory-in-a-day project.

Augmenting awareness by HoloLens: initial exploration

Author/Institution: Jouke Verlinden/TU Delft

This video is a first exploration on how Augmented Reality can be used in production planning and during deployment & operation. It is part of Deliverable 4.4.

  1.  HoloLens: Although we considered projector-based and non-visceral augmentation, the use of a stereoscopic head-mounted display allows to provide better visual (and auditory) feedback. Specifically, if considering a portable solution, being able to be used during the workflow of the Factory in a Day process. In particular, see-through options allow a proper overlap with the environment, and with the emergence of the Microsoft HoloLens and the Meta 2 glasses are enabling these technologies in a proper, ergonomically fitting way. The scenario on motion path and training provides an indication how system state can be overlayed in the physical environment.
  2. How do you plan to use feature 1 of HoloLens in Factory-in-a-day (scene reconstruction)?
    Scene reconstruction is essential in rendering occlusions, which raises the effect of Presence (notion as ‘being-there’). For example, at 2:10 in the video, only a part of the digital objects is rendered as a wall blocks the view. Another important aspect is that this enables a computational integration between spatial context, robot systems, and workers, and can be used to reason on situational awareness of individual workers (in terms of visibility, system status etc.).
  3. How do you plan to use feature 2 of HoloLens in Factory-in-a-day (object placement)?
    Together with feature 1, this option shows how current gesture recognition is done with the HoloLens (so-called “Air tap” and gaze following). Furthermore, these features could be used during maintenance or configuration of robotic systems.
  4. What are the next steps? This is a first impression based on off-the shelf software. Connectivity with ROS and assessing usability of specific cues will be done during the coming 10 months.