Linear combinations of random variables

📚The mathematics

One variable, scaled and shifted

Two things to notice. The variance ignores \(b\) entirely — shifting every value by the same amount moves the distribution but does not spread it. And the multiplier is squared, because variance is measured in squared units. Doubling every value quadruples the variance but only doubles the standard deviation.

Two independent variables

The expectation result holds always; the variance result requires independence, and you should say so when you use it.

\( nX \) is not \( X_1 + X_2 + \cdots + X_n \)

This distinction is the one examiners return to. Suppose \( \mathrm{Var}(X) = \sigma^{2} \).

• \( 2X \) means one observation, doubled. Its variance is \( 2^{2}\sigma^{2} = 4\sigma^{2} \).
• \( X_1 + X_2 \) means two independent observations added. Its variance is \( \sigma^{2} + \sigma^{2} = 2\sigma^{2} \).

Both have mean \( 2\mu \), so the mean cannot tell them apart — only the variance can. The sum of two independent observations is less variable than one doubled, because two independent draws partly cancel each other’s deviations, whereas doubling amplifies whatever deviation the single draw had.

Read the context: “the weight of 2 bags” is \( X_1 + X_2 \); “twice the weight of a bag” is \( 2X \).

In general, for \(n\) independent observations,

The sample mean

Since \( \bar{X} = \dfrac{1}{n}\sum X_i \), applying both rules gives

The mean is unchanged but the variance shrinks with \(n\). That is the whole justification for taking larger samples, and it underpins everything in S2 4.

Which distributions survive combination

• Normal: any linear combination of independent normal variables is itself normal. So \( X \sim \mathrm{N}(\mu_1, \sigma_1^{2}) \) and \( Y \sim \mathrm{N}(\mu_2, \sigma_2^{2}) \) independent gives \( X - Y \sim \mathrm{N}\!\left(\mu_1 - \mu_2,\ \sigma_1^{2} + \sigma_2^{2}\right) \). This is what makes probability questions about differences answerable.
• Poisson: the sum of independent Poissons is Poisson, with the parameters added (see S2 1). A difference of Poissons is not Poisson — it can be negative.
• Binomial: not generally preserved. \( \mathrm{B}(n_1, p) + \mathrm{B}(n_2, p) \) is binomial only when \(p\) is the same in both.