icon 3D-UIR: 3D Gaussian for Underwater 3D Scene Reconstruction via Physics-Based Appearance-Medium Decoupling

Jieyu Yuan1, Yujun Li1, Yuanlin Zhang1,2, Chunle Guo1,3, Xiongxin Tang2, Ruixing Wang4, Chongyi Li1,3#
1NanKai University 2Institute of Software, Chinese Academy of Sciences
3NKIARI, Shenzhen Futian 4DJI Co., Ltd
#Corresponding author
arXiv Code BibTeX
Teaser
Optical scattering and absorption in underwater environments present unique challenges for novel view synthesis. The standard volume rendering equation inadequately models participating media with suspended particles, leading to the incorrect reconstruction of volumetric water as floating artifacts in 3D representation (left). Additionally, light source directionality and viewing angles cause attenuation variations that result in inconsistent scene appearance across viewpoints (right). 3DGS lacks proper modeling of scattering media, causing water column effects on scene surfaces and resulting in collapsed depth representations. Existing scattering NVS methods, e.g., SeaThru-NeRF, fail to account for dynamic photometric variations, which introduces water float artifacts (highlighted in yellow boxes). In contrast, our approach effectively models the participating medium to render photorealistic novel views with accurate scene representation, yielding more consistent scene rendering across novel viewpoints and effective elimination of underwater artifacts.

Abstract

Novel view synthesis for underwater scene reconstruction presents unique challenges due to complex light-media interactions. While 3D Gaussian Splatting (3DGS) offers real-time rendering capabilities, it struggles with underwater inhomogeneous environments where scattering media introduce artifacts and inconsistent appearance. In this study, we propose a physics-based framework that disentangles object appearance from water medium effects through specialized Gaussian modeling. Our approach introduces appearance embeddings which are explicit medium representations for backscatter and attenuation for enhancing scene consistency. In addition, we propose a distance-guided optimization for improving geometric fidelity. By integrating these physics-inspired components through an underwater imaging model, our method achieves both high-quality novel view synthesis and physically accurate scene restoration. Experiments demonstrate our significant improvements in rendering quality and restoration accuracy over existing methods. Our code will be made available upon acceptance.

Pipeline

Teaser

Overview of 3D-UIR:

  • Underwater Appearance Modeling Branch (Yellow): This branch incorporates appearance features and embeddings that are crucial for maintaining view consistency, ensuring accurate scene representation across various perspectives.
  • Scatter Medium Modeling Branch (Red): The medium modeling branch separately handles backscatter and attenuation, key factors that influence the appearance of objects in underwater environments.
  • Depth-guided Regularization Optimization (Blue): Pseudo-depth maps are utilized to guide distance optimization, improving the accuracy of parameter estimation and enhancing the overall depth accuracy in the scene reconstruction process.

Physics-Based Integration: All components, including UAM, SMD, and DRO, are integrated into a unified framework through a physics-based underwater image formation model during differentiable rasterization, allowing for smooth, gradient-based optimization.Our method effectively disentangles object appearance from the water medium effects using specialized Gaussian modeling techniques.

Video Comparison

README
• The left side displays the default render video from the Ours method applied to the Japan scene.
• The right side displays the default RGB video from the 3DGS method applied to the Japan scene.
• Switch scenes via Video Selector dropdown
• Drag the slider to compare videos
• Click the Pause Button to stop the video
• RGB Button: Reconstructed Image which presents reconstructed scene geometry from raw sensor data.
• Clear Button: Restoration Image which shows optimized scene representation with descattering restoration.
Comparative Video Analysis: The comparative results validate the effectiveness of our method in achieving high-quality RGB reconstruction and precise depth estimation. Compared to existing baselines, our approach demonstrates improved structural consistency and photometric accuracy, particularly in distant regions where alternative methods tend to produce artifacts or exhibit depth collapse. These findings underscore the robustness of our framework in addressing the challenges of underwater visual degradation. Furthermore, our descattering restoration significantly improves visual clarity, enabling more faithful scene recovery under scattering-dominant conditions.

Quantitative Results Comparison

We evaluate our method on three datasets: SeaThru-NeRF, Underwater in the Wild (U-IW), and our Simulated dataset (U-S). SeaThru-NeRF contains four forward-facing real underwater scenes with diverse aquatic and imaging conditions. U-IW consists of frames sampled from in-the-wild underwater videos collected from the Internet. Both datasets feature unbounded camera-to-scene distances. To further evaluate our method, we simulated underwater scenes using four scenes from the MipNeRF-360 dataset.
Method Seathru-Nerf U-IW U-S Speed
PSNR ↑SSIM ↑LPIPS ↓ PSNR ↑SSIM ↑LPIPS ↓ PSNR ↑SSIM ↑LPIPS ↓ FPS ↑Training ↓
SeaThru-Nerf 27.3940.8600.215 18.9420.6440.383 24.4360.8050.293 0.552 h 39 m
3DGS 26.1880.8590.238 27.3610.8940.158 29.2740.8810.233 149.3617 m
Splatfacto-Wild 25.7500.8320.229 25.1590.8470.209 25.7860.8530.260 42.9821 m
SeaSplat 27.3850.8660.194 27.0230.8890.159 28.5660.8610.253 42.691 h 25 m
RecGS 25.8290.8570.233 22.1860.8380.180 24.6200.8250.259 146.6238 m
Water-Splatting 27.5730.8650.198 25.6730.8820.167 29.9730.8780.235 35.8029 m
icon Ours 28.1160.8760.202 28.1980.9020.150 31.2270.8910.187 48.7248 m
Quantitative evaluation of existing methods: Metrics are averaged across all scenes in various datasets. The top three rankings in each category are highlighted: 1st, 2nd, 3rd. Arrows indicate whether higher (↑) or lower (↓) values are better.

More details (including validation experiments) can be found in our paper.

Related Links

There are several key works in the field of underwater scene reconstruction and novel view synthesis that are closely related to our approach.

The work in 3DGS: 3D Gaussian Splatting for Real-time Rendering provides real-time rendering capabilities, though it lacks the detailed modeling of scattering media needed for underwater scenes.

SeaThru-NeRF: Neural Radiance Fields in Scattering Media focuses on overcoming underwater artifacts caused by scattering media, offering a first step towards scattering-aware novel view synthesis in underwater environments.

DeepSeeColor: Realtime Adaptive Color Correction for Autonomous Underwater Vehicles via Deep Learning Methods introduces deep learning models to restore underwater images, closely related to our work in removing water medium effects.

Contact

Feel free to contact us at jieyuyuan.cn[AT]gmail.com !