Optimization on a Manifold

All the optimization methods discussed so far have been designed to operate in a "flat" Euclidean space. When optimizing a state vector that contains one or more variables where the Euclidean space loses its meaning (for example, rotations or matrices), every parametrization results in sub-optimal solutions and is affected by singularities. In recent years, techniques that utilize the overparameterized version of the state vector (Her08) have gained significant traction, allowing for the optimization of the problem directly on the manifold, which is locally homeomorphic to a linear space.

The idea is to transform the classical minimization of $S \in \mathcal{M}$, with $\mathcal{M}$ being an n-dimensional manifold,

$$\boldsymbol\delta \Leftarrow \left. \dfrac{\partial S(\math... ... 0 \qquad \mathbf{x} \Leftarrow \mathbf{x} + \boldsymbol\delta$$ (3.61)

into

$$\boldsymbol\epsilon \Leftarrow \left. \dfrac{\partial S(\ma... ... \mathbf{x} \Leftarrow \mathbf{x} \boxplus \boldsymbol\epsilon$$ (3.62)

with $\boldsymbol\epsilon \in \mathbb{R}^{n}$, assuming that in the neighborhood $\boldsymbol\epsilon = 0$ the function operates in a Euclidean space. The operator $\boxplus$ allows for the addition between elements of the manifold space and elements of the Euclidean space $\mathbb{R}^{n}$.

A very classic example is to consider the optimization of an orientation expressed in 3 dimensions through the use of a quaternion in 4 dimensions.