Line Regression

Let

$$y = mx +q + \varepsilon$$ (4.91)

be the equation of a line written in explicit form, with the measurement error entirely along the $y$ axis. With the error along the $y$ axis, the cost function to be minimized is

$$S = \frac{1}{2n} \sum_{i=1}^{n} { \left( m x_i + q - y_i \right)^2 }$$ (4.92)

The solution to the problem is the point at which the gradient of $S$ with respect to $m$ and $q$ vanishes:

$$\begin{array}{rl} \frac{\partial S}{\partial m} & = \frac{... ...\sum y_i \right) = m \bar{x} + q - \bar{y} = 0 \\ \end{array}$$ (4.93)

that is:

$$\begin{array}{l} m = \dfrac{ \bar{(xy)}-\bar{x}\bar{y}}{\b... ...{\text{var}(x)} \\ q = - m \bar{x} + \bar{y} \\ \end{array}$$ (4.94)

where $\bar{x}$ is the mean of the samples $x_i$ (the same notation is used for the other quantities). The line passes through the point $(\bar{x},\bar{y})$, the centroid of the distribution.

This result can easily be modified when one wants to minimize the deviation along $x$ rather than along $y$, or to represent the line equation in implicit form.