Revise Linear combinations of random variables for Cambridge 9709 (6.2)

The rule for finding the expectation (or mean) of a linear combination is wonderfully straightforward. It's known as the 'linearity of expectation'. It works for any random variables, whether they are independent or not. You simply apply the expectation operator to each part of the expression.

For any random variables X and Y, and constants a and b:

$E(aX + bY) = aE(X) + bE(Y)$

This extends to subtraction and simple sums/differences:

$E(X + Y) = E(X) + E(Y)$ $E(X - Y) = E(X) - E(Y)$

Calculating the variance of a linear combination is more subtle and requires a crucial condition: the random variables must be independent. Independence means that the outcome of one variable has no effect on the outcome of the other. When this condition is met, the variances of the individual variables add up. Notice how the constants are squared.

For independent random variables X and Y, and constants a and b:

$\text{Var}(aX + bY) = a^2\text{Var}(X) + b^2\text{Var}(Y)$

A particularly powerful result occurs when we combine independent Normal random variables. Any linear combination of independent Normal variables is itself a Normal variable. This allows us to combine our rules for expectation and variance to define a new Normal distribution, which we can then use to find probabilities.

If $X \sim N(\mu_X, \sigma_X^2)$ and $Y \sim N(\mu_Y, \sigma_Y^2)$ are independent, then:

$aX + bY \sim N(a\mu_X + b\mu_Y, a^2\sigma_X^2 + b^2\sigma_Y^2)$