Transformations of Random Variables

One of the fundamental problems in statistics is understanding how a random variable propagates through a complex system and to what extent it makes the output of that system random.

Let $f(\cdot)$ be a function that transforms the random variable $X$ into the random variable $Y$, that is, $y = f(x)$, with $x$ realizations of the random variable $X$, and suppose that $f$ is invertible, meaning that there exists a function $x = g(y)$ such that $g(f(x))=x$.

Let $\mathcal{I}_x$ be a generic interval in the domain of the values $x$ and $\mathcal{I}_y=\{ y: y=f(x), x \in \mathcal{I}_x \}$ its corresponding image. The probabilities of the events $x$ in $\mathcal{I}_x$ and $y$ in $\mathcal{I}_y$ must clearly be equal, that is,

$$\int_{\mathcal{I}_y} p_Y(y) dy = \int_{\mathcal{I}_x} p_X(x) dx$$ (2.25)

Without loss of generality, the interval $\mathcal{I}_x$ can be taken to be infinitesimal. Under this condition, relation (2.25) reduces to

$$p_Y(y) \vert dy\vert = p_X(x) \vert dx \vert = p_X(g(y)) \vert dx \vert$$ (2.26)

from which

$$p_Y(y) = p_X(g(y)) \frac{\vert dx\vert}{\vert dy \vert } = ... ...= \left. \frac{p_X(x)}{\vert f'(x)\vert} \right\vert _{x=g(y)}$$ (2.27)

This relation can readily be extended to the case of a non-injective function by summing the different intervals, and to the multidimensional case by using the Jacobian instead of the derivative.