TAMBRIDGE: Bridging Frame-Centered Tracking and 3D Gaussian Splatting for Enhanced SLAM

arXiv 2024

Peifeng Jiang1, Hong Liu1, Xia Li2, Ti Wang1, Fabian Zhang2, Joachim M. Buhmann2,
1Peking University, China 2ETH Zurich, Switzerland
1
2
Paper arXiv Video Code

Abstract

The limited robustness of 3D Gaussian Splatting (3DGS) to motion blur and camera noise, along with its poor real-time performance, restricts its application in robotic SLAM tasks. Upon analysis, the primary causes of these issues are the density of views with motion blur and the cumulative errors in dense pose estimation from calculating losses based on noisy original images and rendering results, which increase the difficulty of 3DGS rendering convergence. Thus, a cutting-edge 3DGS-based SLAM system is introduced, leveraging the efficiency and flexibility of 3DGS to achieve real-time performance while remaining robust against sensor noise, motion blur, and the challenges posed by long-session SLAM. Central to this approach is the Fusion Bridge module, which seamlessly integrates tracking-centered ORB Visual Odometry with mapping-centered online 3DGS. Precise pose initialization is enabled by this module through joint optimization of re-projection and rendering loss, as well as strategic view selection, enhancing rendering convergence in large-scale scenes. Extensive experiments demonstrate state-of-the-art rendering quality and localization accuracy, positioning this system as a promising solution for real-world robotics applications that require stable, near-real-time performance.}.

Framework Overview

TAMBRIDGE Overview

Framework of the proposed TAMBRIDGE. ORB-based visual odometry analyzes RGB-D data to estimate poses and select keyframes for local mapping. These keyframes feed into the Global Optimization featuring Bundle Adjustment (BA) to refine the path. Simultaneously, the Fusion Bridge module selects reconstruction frames and calculates rendering poses and Border Masks. An online 3DGS backend processes the selected frames to create a globally consistent, high-fidelity scene representation.


Plug and Play Fusion Bridge

Fusion Bridge

The Fusion Bridge module selects reconstruction keyframes from a local map based on viewpoint covisibility. It projects the 3DGS and local map point cloud onto the reconstruction frame, filtering projections through pixel gates. The module then optimizes the pose by jointly minimizing rendering and point cloud reprojection losses, setting the initial rendering pose for the Online 3DGS.


Robotic Long-session SLAM

Fusion Bridge

By seamlessly integrating ORB Visual Odometry with viewpoint selection and re-projection loss, our method significantly improves the robustness towards sensor noise and motion blur especially in long-session robotic tasks.


Results on TUM-RGBD

fr2_360 (5 FPS)

fr2_pioneer_slam2 (9 FPS)

fr1_desk (9 FPS)

fr3_office (9 FPS)



Reconstruction Results

Image 1
Image 2
Image 3
Image 4
Image 5
Image 7

Visual Comparison

Comparison.

FPS-Induced Reconstruction Degradation

Comparison.

BibTeX

@misc{jiang2024tambridge,
      title={TAMBRIDGE: Bridging Frame-Centered Tracking and 3D Gaussian Splatting for Enhanced SLAM}, 
      author={Peifeng Jiang and Hong Liu and Xia Li and Ti Wang and Fabian Zhang and Joachim M. Buhmann},
      year={2024},
      eprint={2405.19614},
      archivePrefix={arXiv},
      primaryClass={cs.RO}
}