DGSI

Dense 3D Gaussian Splatting Initialization

for Sparse Image Data


Eurographics'2024


Simon Seibt1, Thomas Chang1, Bartosz von Rymon Lipinski1 and Marc Erich Latoschik2


1 Game Tech Lab, Nuremberg Institute of Technology
2 Human-Computer Interaction Group, University of Wuerzburgy


Visual comparison for 3D Gaussian Splatting (3DGS) [KKLD23] and 3DGS initialized with DFM4SfM (Ours)
on sparse image data: Scene "Fern" [MSOC∗19] with 16 and 12 input images.


Abstract

This paper presents advancements in novel-view synthesis with 3D Gaussian Splatting (3DGS) using a dense and accurate SfM point cloud initialization approach. We address the challenge of achieving photorealistic renderings from sparse image data, where basic 3DGS training may result in suboptimal convergence, thus leading to visual artifacts. The proposed method enhances precision and density of initially reconstructed point clouds by refining 3D positions and extrapolating additional points, even for difficult image regions, e.g. with repeating patterns and suboptimal visual coverage. Our contributions focus on improving ``Dense Feature Matching for Structure-from-Motion'' (DFM4SfM) based on a homographic decomposition of the image space to support 3DGS training: First, a grid-based feature detection method is introduced for DFM4SfM to ensure a well-distributed 3D Gaussian initialization uniformly over all depth planes. Second, the SfM feature matching is complemented by a geometric plausibility check, priming the homography estimation and thereby improving the initial placement of 3D Gaussians. Experimental results on the NeRF-LLFF dataset demonstrate that this approach achieves superior qualitative and quantitative results, even for fewer views, and the potential for a significantly accelerated 3DGS training with faster convergence.


Paper


Poster


Pipeline Overview

UML-based activity diagram of enhanced DFM4SfM [SVRLCL23] feature matching pipeline
for Structure from Motion.


Visual Results

Detailed visual comparisons on sparse image data from NeRF-LLFF [MSOC∗19]:
3DGS [KKLD23] without (bottom) and with our proposed method (top).


BibTeX

@inproceedings{Seibt2024DGSI,
booktitle = {Eurographics 2024 - Posters},
title = {Dense 3D Gaussian Splatting Initialization for Sparse Image Data},
author = {Seibt, Simon and Chang, Thomas Vincent Siu-Lung and Lipinski, Bartosz von Rymon and Latoschik, Marc Erich},
year = {2024},
publisher = {The Eurographics Association},
}


References

[KKLD23] Kerbl B., Kopanas G., Leimkuehler T., Drettakis G.: 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics. (2023).
[MSOC∗19] Mildenhall B., Srinivasan P. P., Ortiz-Cayon R., Kalantari N. K., Ramamoorthi R., Ng R., Kar A.: Local light field fusion: practical view synthesis with prescriptive lines. ACM Transactions on Graphics. (2019).
[SVRLCL23] Seibt S., Von Rymon Lipinski B., Chang T., Latoschik M. E.: Dfm4sfm - dense feature matching for structure from motion. In IEEE International Conference on Image Processing Workshops. (2023).