XR Screen Bus Simulator

The XR Screen Bus Simulator project aims to create a sophisticated simulation environment that integrates depth data and 2D RGB imagery to enhance the realism and accuracy of mixed reality (MR) content. This system employs Intel RealSense D435 cameras to capture depth data and 360-degree cameras for comprehensive environmental coverage. By combining these data sources, the simulator can produce a seamless visual representation where distant objects are rendered using 2D RGB imagery, while closer objects are rendered with depth-enhanced 3D data. This approach addresses common issues of pixel voids and ensures high fidelity in visual simulations.

To achieve precise vehicle movement tracking, the project utilizes a fusion of IMU and GPS data processed through a Kalman filter. This fusion compensates for the inherent inaccuracies of individual sensors, providing accurate real-time tracking of the vehicle's trajectory. The simulator's integration within Unity allows for dynamic rendering of MR content based on these data inputs, enabling realistic and interactive experiences. This project not only enhances the accuracy and realism of MR simulations but also provides a scalable framework for future applications in public transportation and interactive display systems.