The XR4DRAMA Image-Based 3D Reconstruction Pipeline

In yet another look behind the scenes, we explain how we manage to augment the XR4DRAMA maps with more sophisticated 3D models.

Image-based 3D reconstruction is an image processing technique used to create a digital 3D model of a real-world object from a set of 2D images. Several free and commercial software packages and algorithms allow experts to create three-dimensional models. Typically, the reconstruction process extracts geometric information from images via complex algorithms, including feature detection, keypoints matching, structure-from-motion, multi-view stereo matching, model fitting, and texture mapping. This process requires a set of overlapping images taken with hand-held devices, e.g. SLR cameras and mobile phones, or unmanned aerial vehicles (UAVs).

Check out the following infographic to get an even better understanding of the workflow (open image in new tab and zoom in to see all details):

In the XR4DRAMA platform, technical partner up2metric deployed a 3D reconstruction module to allow users – in this case: disaster managers and media production planners –  to build georeferenced 3D models as photo-textured triangular meshes. The images are uploaded to the XR4DRAMA platform via its authoring tool (developed by Nuro) and processed by the 3D reconstruction module. The resulting model is optimized and automatically placed in the 2D/3D desktop and VR environments to offer an even more immersive experience and enhanced situation awareness, respectively: Users can now inspect the updated scene at street level (“walk mode”) or from up above (“fly mode”).

For our pilot in Vicenza, team up2metric visited the city prior to the field test (carried out by AAWA) to capture the necessary drone footage. Local authorities responsible for collecting disaster management data facilitated the UAV operation. The drone images were then uploaded and processed in the way described above.

The screenshots in this post come from the Meta Quest II headset and document a VR inspection of Vicenza (with textured 3D models) from a bird’s eye perspective.