Reconstruction, Representation, and Rendering of Three-Dimensional Environments

Point clouds (point clouds) and polygonal meshes are among the most widespread three-dimensional representations for storing environments and objects. These approaches have the advantage of clearly separating geometric reconstruction from rendering. This separation, however, makes it necessary to define the geometry and the scene's appearance model separately.

Recently, Neural Radiance Field (NeRF) methods (MST$^+$20), and especially 3D Gaussian Splatting methods, have given this field considerable momentum by reducing the separation between geometric reconstruction and rendering. In both families of methods, the scene is not represented directly by a mesh, but by a representation optimized with respect to the observed images. Explicit geometry can subsequently be extracted, if needed, using surface-reconstruction techniques.