Orthogonal Distance Fit

In the case where the error is present on both axes (noise as a function of distance), the formulation of the cost function $S$ that maximizes the likelihood is referred to as the Orthogonal least-squares line fit. The error can indeed be expressed using the distance between the point and the line, according to equation (1.31). The regression that utilizes this metric, therefore known as Perpendicular Regression or Total least squares (see section 3.2.2), is meaningful when both coordinates are affected by error, meaning they are both random variables. The amount of noise in the two components is assumed to be equal (for the more general case, see the discussion in section 2.4). The error function $S$ to be minimized is the distance between the point and the line:

$$S = \frac{1}{2n}\sum_{i=1}^{n} { \frac{(a x_i + b y_i + c)^2}{a^2 + b^2} }$$ (3.74)

. The minimum is found at $\nabla S = 0$. It is noteworthy that in the case of perpendicular distance, there exists both a minimum and a maximum as solutions, and therefore there will be two values of lines (orthogonal to each other), both of which are solutions to the system.

From the partial derivative $\frac{ \partial S}{\partial c}=0$, it follows that the regression line passes through the centroid $(\bar{x},\bar{y})$ of the distribution, that is,

$$c = - a \bar{x} - b \bar{y}$$ (3.75)

with $\bar{x}$ and $\bar{y}$ being the means of samples $x_i$ and $y_i$ respectively.

The error function (3.74), using the relationship (3.75), can be expressed as: It seems that you've entered a placeholder or a command for a math block. Please provide the specific content or context you would like to translate, and I'll be happy to assist you! that is, by making appropriate substitutions $S_{xx} = \text{var}(x)$, $S_{yy} = \text{var}(y)$ and $S_{xy} = \text{cov}(x,y)$:

$$S = \frac{a^2 S_{xx} + 2 ab S_{xy} + b^2 S_{yy} }{a^2 + b^2}$$ (3.76)

more easily derivable. The expression (3.76) for the error is not of a general nature, but is valid only for all lines that pass through the centroid of the distribution. Being a homogeneous form, it is known up to a multiplicative factor: therefore, there is not a single solution but a relationship that links the parameters. Excluding the cases $a=0$, $b=0$ (to be treated separately), the constraint for deriving the minimum/maximum has the form of type

$$(a^2 -b^{2}) S_{xy} + a b (S_{yy} - S_{xx}) = 0$$ (3.77)

, which is the solution to the problem.

It is worth noting that the same result can be obtained in a much simpler manner by applying the SVD decomposition to the equation of the lines. In the case of linear regression, the SVD decomposition minimizes both the algebraic and geometric errors (the algebraic and geometric errors coincide when all noise-affected terms remain confined to the known term).