Spherical Harmonics

A key aspect of color representation comes from the use of Spherical Harmonics (Spherical Harmonics, SH). Spherical harmonics are solutions of the Laplace equation in spherical coordinates and form a complete orthogonal basis for functions defined on the surface of a sphere. A function $L(\theta,\phi)$ can therefore be represented as

$$L(\mathbf{d}) \simeq L(\theta,\phi) = \sum_{l=0}^{L} \sum_{m=-l}^{l} k_l^m Y_l^m(\theta,\phi),$$ (10.105)

where $\mathbf{d}$ represents the direction, $l$ is the harmonic degree, $m$ its order, and $k_l^m$ the corresponding coefficient. In a numerical approximation, a finite maximum degree $L$ is generally used.

For real spherical harmonics, which are also commonly used to represent color in Gaussian Splatting, the following definition can be adopted. For $l \geq 0$ and $-l \leq m \leq l$:

$$Y_l^m(\theta,\phi) = \begin{cases} \sqrt{2}\,K_l^m P_l^m(\co... ...m\vert}(\cos\theta)\sin(\vert m\vert\phi), & m < 0, \end{cases}$$ (10.106)

where

$$K_l^m = \sqrt{ \frac{2l+1}{4\pi} \frac{(l-m)!}{(l+m)!} }, \qquad m \geq 0,$$ (10.107)

and $P_l^m$ are the associated Legendre polynomials, defined according to the convention that includes the Condon–Shortley phase factor ^10.1

For $l=0$, there is a single harmonic, constant over the sphere:

$$Y_0^0 = \frac{1}{\sqrt{4\pi}} = \frac{1}{2}\sqrt{\frac{1}{\pi}} \approx 0.2821.$$ (10.108)

For a maximum degree $L$, the total number of harmonics is

$$N_{\mathrm{SH}} = \sum_{l=0}^{L}(2l+1) = (L+1)^2.$$ (10.109)

Consequently, each color component can be represented using $(L+1)^2$ coefficients. For example, with $L=0$, a single coefficient is used; with $L=1$, four coefficients; and with $L=2$, nine coefficients.

In computer graphics, spherical harmonics are used to represent direction-dependent functions compactly. In neural rendering, they can therefore be used to describe the dependence of color on the viewing direction. The coefficients of the representation progressively determine the variations of the function as the direction changes.

The idea is to select a maximum degree $L$ and represent each color component—red, green, and blue—as a linear combination of spherical harmonics, using the viewing direction $(\theta,\phi)$ as the variable; that is, the direction of the optical ray connecting the point to the observer.

For each color component, one therefore obtains, for example, for the red channel:

$$C_R(\mathbf{d}) = \sum_{l=0}^{L} \sum_{m=-l}^{l} k_{l,m}^{R} Y_l^m(\mathbf{d}),$$ (10.110)

The same procedure is used to compute $C_G(\mathbf{d})$ and $C_B(\mathbf{d})$, yielding the RGB color $\mathbf{c}(\mathbf{d}) = [C_R(\mathbf{d}),C_G(\mathbf{d}),C_B(\mathbf{d})]^\top$.

Footnotes

... factor^10.1

Note that different conventions are used in the literature for ordering and signing real spherical harmonics; the same convention must therefore be maintained when generating the functions and interpreting the coefficients.