Forward Facing 3D Gaussian Splatting as Markov Chain Monte Carlo

In this project, we enhance the performance of 3D Gaussian Splatting as Markov Chain Monte Carlo in forward-facing scenes by introducing Depth Supervision and Near Gaussian Regularizers. Detailed explanations of the project can be found in the downloadable paper on the upper right.

3D Gaussian Splatting as Markov Chain Monte Carlo (3DGS-MCMC) is a modified version of 3D Gaussian Splatting, which reduces the heuristics of the original method. 3DGS-MCMC treats the training and optimization process as a sampling process from a probability distribution. Using the probability approach, 3DGS MCMC treats the complicated Adaptive Density Control step as simple state transitions. This metric does not require cloning or splitting of the Gaussian, reducing the number of hand-tuned parameters throughout the algorithm.

While 3DGS-MCMC performs decently for a number of 360° datasets, we discovered some weaknesses of 3DGS-MCMC in forward-facing scenarios, especially when given random initials. The failure of 3DGS-MCMC in forward-facing scenes is mainly due to floating artifacts, which we suspect is from the model overfitting to closer regions of the camera during the state transition phase.

In order to address the weaknesses mentioned above, we integrated Depth Supervision to the model, while penalizing Gaussians that are placed near the cameras.

As a result, our model, FF-3DGS-MCMC, outperformed the original 3DGS-MCMC algorithm in the LLFF Dataset, a dataset with forward-facing scenes.