Regression to a Line

Let

$$y = mx +q + \varepsilon$$ (4.86)

be the equation of the line written in explicit form, with the measurement error entirely along the $y$ axis. With the error along the $y$ axis, the cost function to be minimized is

$$S = \frac{1}{2n} \sum_{i=1}^{n} { \left( m x_i + q - y_i \right)^2 }$$ (4.87)

The solution to the problem is the point at which the gradient of $S$ at $m$ and $q$ vanishes:

$$\begin{array}{rl} \frac{\partial S}{\partial m} & = \frac{... ...\sum y_i \right) = m \bar{x} + q - \bar{y} = 0 \\ \end{array}$$ (4.88)

that is:

$$\begin{array}{l} m = \dfrac{ \bar{(xy)}-\bar{x}\bar{y}}{\b... ...{\text{var}(x)} \\ q = - m \bar{x} + \bar{y} \\ \end{array}$$ (4.89)

where $\bar{x}$ is the mean of the samples $x_i$ (the other quantities are denoted using the same notation). The line passes through the point $(\bar{x},\bar{y})$, the centroid of the distribution.

This result can easily be modified when one wants to minimize the deviation along the $x$ rather than along the $y$, or represent the equation of the line in implicit form.