Parameter Estimation

Kalman filtering, in all its variants, is traditionally viewed as a state filter or estimator. However, these techniques are widely used, particularly in machine learning, to estimate the parameters of a model (the meta-model):

$$\mathbf{y}_k = f(\mathbf{x_k}, \boldsymbol\beta)$$ (3.50)

where $\mathbf{y}_k$ are the system outputs, $\mathbf{x_k}$ are the inputs, and $f$ is a function based on the parameters $\boldsymbol\beta$ to be estimated. Model training, or fitting, consists of determining the parameters $\boldsymbol\beta$.

Kalman filtering can determine the model parameters, including time-varying parameters, by using $\boldsymbol\beta$ itself as the state to be estimated, thereby obtaining an iterative system of the form

$$\left\{ \begin{array}{l} \boldsymbol\beta_{k+1} = \boldsymbo... ...{y}_k = f(\mathbf{x}_k, \boldsymbol\beta_k) \end{array}\right.$$ (3.51)

where the optional noise $\mathbf{w}_k$ is used to model possible time variations in the model: the choice of the variance of $\mathbf{w}$ determines the responsiveness to changes in the model parameters.