VIRTUAL ELEMENT RETRIEVAL IN MIXED REALITY

Abstract. The application of mixed reality visualisation in construction engineering requires accurate placement and retrieval of virtual models within the real world, which depends on the localisation accuracy. However, it is hard to understand what this means practically from localisation accuracy alone. For example, when we superimpose a Building Information Model (BIM) over the real building, it is unclear how well does a BIM element fit the real one and how small a BIM element are we able to retrieve. In this paper, we evaluate virtual element retrieval by designing an experiment where we attempt to retrieve a set of cubes of different sizes placed in both the real and the virtual world. Furthermore, inspired by existing camera localisation methods for indoor MR being almost exclusively image-based, we use a localisation approach based solely on 3D-3D model registration. The approach is based on the automated registration of a low-density mesh model of the surroundings created by the MR device to the existing point cloud of an indoor environment. We develop a prototype and perform experiments on real-world data which show high localisation accuracy, with average translation and rotation errors of 1.4 cm and 0.24°, respectively. Finally, we show that the success rate of virtual element retrieval is closely related to the localisation accuracy.