Distributed Collaboration in Infrastructure Assessment through Mixed and Virtual Reality - a

Authors

  • Zaid Abbas Al-Sabbag University of Waterloo
  • Chul Min Yeum University of Waterloo
  • Sriram Narasimhan University of California, Los Angeles

DOI:

https://doi.org/10.57922/tcrc.642

Keywords:

Visual inspection, Mixed reality, Virtual reality, Remote collaboration, Damage detection

Abstract

In this study, we propose a real-time distributed collaborative system that inspectors equipped with Mixed Reality (MR) and Virtual Reality (VR) headsets can perform synchronous structural inspections on- and off-site. MR headsets are equipped with a holographic display that allows on-site inspectors to visualize the locations of structural defects overlaid on the structure, while VR headsets allow off-site inspectors to visualize and interact with a reconstructed 3D virtual environment. A proposed workflow begins with generating a colorized 3D point cloud map of the site using visual sensors and lidars prior to the inspection. The map is then stored in a centralized server to be accessible to multiple MR/VR users. MR inspectors performing the structural assessment on-site are spatially aligned in the reconstructed map using image-based localization. MR inspectors can then add annotations and capture images of structural defects that are stored on the map and shared with remote VR users. Simultaneously, VR inspectors can perform the inspection using the localized images stored on the map and shared by the MR inspector. VR inspectors can also add annotations that are both visible to other VR and MR inspectors, achieving collaborative inspection. An experimental study is conducted in a lab environment to demonstrate the proposed workflow using Microsoft HoloLens 2 (HL2) as the MR headset and Oculus Quest 2 as the VR headset. A panoramic camera (Ricoh Theta Z1) was attached to the HL2 headset to give the VR user a 360 degrees view of the site from images captured by the MR inspector for inspection and map reconstruction.

Published

2022-08-19

Conference Proceedings Volume

2022: The Digital Reality of Tomorrow

Section

Abstracts

License

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Authors/employers retain all proprietary rights in any process, procedure or article of manufacture described in the Work. Authors/employers may reproduce or authorize others to reproduce the Work, material extracted verbatim from the Work, or derivative works for the author’s personal use or for company use, provided that the source is indicated.