Reconstruction, Representation, and Rendering of Three-Dimensional Environments

Point clouds (point clouds) and polygonal meshes are among the most widely used three-dimensional representations for storing environments and objects. These approaches have the advantage of clearly separating geometric reconstruction from rendering. This separation, however, makes it necessary to define the scene geometry and appearance model separately.

More recently, both Neural Radiance Field (NeRF) methods (MST$^+$20) and, especially, 3D Gaussian Splatting have provided a major impetus in this area by reducing the separation between geometric reconstruction and rendering. In both families of methods, the scene is not represented directly by a mesh but by a representation optimized with respect to the observed images. Explicit geometry can subsequently be extracted, if needed, using surface-reconstruction techniques.