Sowohl im Online-, aber auch im stationären Handel sind schon etliche innovative immersive Anwendungen entstanden, die neue kognitive und affektive Interaktions- und Informationsmöglichkeiten bieten. In den Bereichen Kunst, Immobilien, Architektur, Gaming, Fashion, Stadtplanung und -führungen finden sich ebenfalls mehr und mehr AR/VR Anwendungen. In diesem Beitrag wird nach einer Sichtung ausgewählter immersiver Projekte ein Konzept zur Nutzung von AR bzw. VR für Leerstände in einer ehemals attraktiven Einkaufsmeile in Gelsenkirchen vorgestellt.

In the realm of digital situational awareness during disaster situations, accurate digital representations, like 3D models, play an indispensable role. To ensure the safety of rescue teams, robotic platforms are often deployed to generate these models. In this paper, we introduce an innovative approach that synergizes the capabilities of compact Unmaned Arial Vehicles (UAVs), smaller than 30 cm, equipped with 360° cameras and the advances of Neural Radiance Fields (NeRFs). A NeRF, a specialized neural network, can deduce a 3D representation of any scene using 2D images and then synthesize it from various angles upon request. This method is especially tailored for urban environments which have experienced significant destruction, where the structural integrity of buildings is compromised to the point of barring entry—commonly observed post-earthquakes and after severe fires. We have tested our approach through recent post-fire scenario, underlining the efficacy of NeRFs even in challenging outdoor environments characterized by water, snow, varying light conditions, and reflective surfaces.

Problem: A group of robots, called a swarm, is placed in an unknown environment and is supposed to explore it independently. The goal of the exploration is the creation of a common map. Implementation - Equipping six Kobuki robots with appropriate sensor technology, a large battery, a router and the Jetson board - Setup of the Jetson-Boards with self-made ROS2 nodes and the set up mesh network - Writing of launch files for the common start of all functions - Reinforcement learning is used to train an AI that controls the swarm by selecting points for the robots to approach and navigating to them and navigating them there. - Setting up a responsive website using Angular and the Bootstrap Framework.

The dataset is used for 3D environment modeling, i.e. for the generation of dense 3D point clouds and 3D models with PatchMatch algorithm and neural networks. Difficult for the modeling algorithm are the reflections of rain, water and snow, as well as windows and vehicle surface. In addition, lighting conditions are constantly changing.

At the integration sprint of the E-DRZ consortium in march 2023 we improve the information captured by the human spotter (of the fire brigade) by extending him through a 360° drone. The UAV needs 3 minutes to capture the outdoor scenario and the hall from inside and outside. The hall ist about 70 x 20 meters. When the drone is landed we have all information in 360° degree at 5.7k as you can see it in the video. Furthermore it is a perfect documentation of the deployment scenario. In the next video we will show how to spatial localize the 360° video and how to generate a 3D point cloud from it.