Polynomial Function Linear Regression

The method applied to obtain linear regression to a line expressed in explicit form can be generalized to any polynomial function of the type:

$$y = \beta_0 + \beta_1 x + \beta_2 x^2 + \ldots + \beta_m x^m + \varepsilon$$ (3.80)

where $\beta_0 \ldots \beta_m$ are the parameters of the curve to be determined, parameters that are obtained by seeking the minimum of the error function described in (3.6). The derivatives of a polynomial function are noteworthy:

$$\begin{array}{rl} \frac{\partial S}{\partial \beta_j} & = ... ...+ \ldots + \beta_m \sum x_i^{j+m} - \sum y_i x_i^j \end{array}$$ (3.81)

Setting the gradient to zero means solving the associated system:

$$\begin{bmatrix} \sum 1 & \ldots & \sum x_i^{m} \\ \sum x... ... \sum y_i x_i \\ \vdots \\ \sum y_i x_i^m \\ \end{bmatrix}$$ (3.82)

which is a symmetric matrix.

Alternatively, it is possible to leverage the theory of the pseudoinverse (see section ) and directly use equation (3.80) to construct an overdetermined linear system:

$$\begin{bmatrix} 1 & x_1 & \ldots & x_1^{m} \\ 1 & x_2 & ... ...bmatrix} y_1 \\ y_2 \\ \vdots \\ y_n \\ \end{bmatrix}$$ (3.83)

Vandermonde matrix. The solution to this system allows us to obtain the coefficients of the polynomial that minimizes the square of the residuals. When considering the pseudoinverse solved with the method of normal equations, it becomes evident that the resulting system is exactly the same as that of equation (3.82).

As will be seen in other parts of this book, matrices such as the Vandermonde matrix, where the different columns have varying orders of magnitude, are ill-conditioned and require normalization to enhance their numerical stability.