Conditioning of linear systems

Section 1.1 examined overdetermined linear systems and the main techniques used to determine their solution. In particular, we saw how the singular value decomposition makes it possible to identify the directions in which the system is more or less constrained.

This section instead addresses the sensitivity of the solution to perturbations in the data. In other words, we seek to determine how much a small variation in the problem data can change the solution. This property is called the conditioning of the problem.

Conditioning must be distinguished from the numerical error introduced by the algorithm used to solve the problem. An ill-conditioned problem is intrinsically sensitive to perturbations in the data, regardless of the algorithm used; a numerically unstable algorithm can instead introduce errors even when solving a well-conditioned problem.