DGSI (Poster-Paper@EG2024)
DGSI
Dense 3D Gaussian Splatting Initialization
for Sparse Image Data
Eurographics'2024
Simon Seibt1, Thomas Chang1, Bartosz von Rymon Lipinski1 and Marc Erich Latoschik2
2 Human-Computer Interaction Group, University of Wuerzburgy
Visual comparison for 3D Gaussian Splatting (3DGS) [KKLD23] and 3DGS initialized with DFM4SfM (Ours)
on sparse image data: Scene "Fern" [MSOC∗19] with 16 and 12 input images.
Abstract
This paper presents advancements in novel-view synthesis with 3D Gaussian Splatting (3DGS) using a dense and accurate SfM point cloud initialization approach. We address the challenge of achieving photorealistic renderings from sparse image data, where basic 3DGS training may result in suboptimal convergence, thus leading to visual artifacts. The proposed method enhances precision and density of initially reconstructed point clouds by refining 3D positions and extrapolating additional points, even for difficult image regions, e.g. with repeating patterns and suboptimal visual coverage. Our contributions focus on improving ``Dense Feature Matching for Structure-from-Motion'' (DFM4SfM) based on a homographic decomposition of the image space to support 3DGS training: First, a grid-based feature detection method is introduced for DFM4SfM to ensure a well-distributed 3D Gaussian initialization uniformly over all depth planes. Second, the SfM feature matching is complemented by a geometric plausibility check, priming the homography estimation and thereby improving the initial placement of 3D Gaussians. Experimental results on the NeRF-LLFF dataset demonstrate that this approach achieves superior qualitative and quantitative results, even for fewer views, and the potential for a significantly accelerated 3DGS training with faster convergence.
Paper
Poster
Pipeline Overview
UML-based activity diagram of enhanced DFM4SfM [SVRLCL23] feature matching pipeline
for Structure from Motion.
Visual Results
Detailed visual comparisons on sparse image data from NeRF-LLFF [MSOC∗19]:
3DGS [KKLD23] without (bottom) and with our proposed method (top).
BibTeX
@inproceedings{Seibt2024DGSI,
booktitle = {Eurographics 2024 - Posters},
title = {Dense 3D Gaussian Splatting Initialization for Sparse Image Data},
author = {Seibt, Simon and Chang, Thomas Vincent Siu-Lung and Lipinski, Bartosz von Rymon and Latoschik, Marc Erich},
year = {2024},
publisher = {The Eurographics Association},
}
References
[KKLD23] | Kerbl B., Kopanas G., Leimkuehler T., Drettakis G.: 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics. (2023). |
[MSOC∗19] | Mildenhall B., Srinivasan P. P., Ortiz-Cayon R., Kalantari N. K., Ramamoorthi R., Ng R., Kar A.: Local light field fusion: practical view synthesis with prescriptive lines. ACM Transactions on Graphics. (2019). |
[SVRLCL23] | Seibt S., Von Rymon Lipinski B., Chang T., Latoschik M. E.: Dfm4sfm - dense feature matching for structure from motion. In IEEE International Conference on Image Processing Workshops. (2023). |