Sparse-view 3D Reconstruction using Gaussian Splatting

Variance Sampling Strategy for efficient reconstruction

We compared the performance of NeRF with and without SDS guidance from the diffusion to model to see the impact for different number of input views. (The ZeroNVS paper only compares performance for a single view of a scene)

Our focus is on improving the view selection for SDS so that we can maximize the performance boost for lesser number of input views (<25).

The ZeroNVS algorithm currently randomly samples views around the object (6 views per iteration) and the views are randomly noised with the object masked using depth from NeRF so that foreground is not learnt using SDS.

The variance sampling strategy reduces the number of views sampled by filtering them out based on variance of the CLIP embedding of the RGB samples.

Quantitative Results for View Selection using Variance Sampling Strategy
Baseline Reconstruction using ZeroNVS
Reconstruction using Scored Distillation Sampling
Reconstruction using Variance Sampling Strategy