Noise and observation weighting

It is important to clarify from the outset that, in minimizing the quantity (1.2), no assumption has been made about the distribution of noise among the various components of which the matrix is composed: without this information, there is no guarantee that the solution will be statistically optimal.

Without assumptions about the noise distribution, the solution obtained by this minimization is in fact a purely algebraic solution that minimizes an algebraic error (algebraic error).

A statistically more appropriate solution can be obtained when the noise is characterized and, in particular, when its variance across the different observations is known. In this case, different weights can be assigned to each equation of the system by multiplying each row of the system by a suitable weight, thus weighting each acquired datum differently.

A more detailed discussion of this topic can be found in section 4.2 and, in general, chapter 2 will address the general case in which the way that the error in the observed data affects parameter estimation is known.

Further details on the Moore–Penrose pseudoinverse can be found in many books, for example in  (CM09) or in the foundational numerical-analysis text  (GVL96).