From BoykinWiki

EEL6507 Spring 2009, Lecture 18, Monday 2009/02/20 (Notes created by Xiaoyuan Li)

Sum of Two Random Variables
Method of Stages(not discussed in this class)

Sum of Two Continuous Random Variables

$b 1 (x)$ is a probability density function for the random variable $X 1$ . $Prob(X_1 \le x)=\int_{-\infty}^{x}b_1(r)dr$ . Likewise for $b 2$ for $X 2$ . If we define $Y = X 1 + X 2$ , then $b_3(y)=\frac{d}{dy}Prob(Y \le y) = \int_{-\infty}^\infty b_1(x)b_2(y-x) dx$ . Let's see a proof of this:

$\begin{align} Prob(Y \le y)&=&\int_{-\infty}^{\infty} Prob(X_2 = y - x | X_1 = x) b_1(x) dx\\ &=& \int_{-\infty}^{\infty} \left( \int_{-\infty}^{y-x} b_2(x')dx'\right) b_1(x) dx \end{align}$

We can either use the chain rule: $\frac{d}{dy}\int_{a}^{g(y)} f(x) dx = f(g(y))g'(y)$ or we can we change variables so $x' = x'' - x$ to see that:

$\begin{align} \frac{d}{dy} Prob(Y \le y)&=& \frac{d}{dy}\int_{-\infty}^{\infty} \left( \int_{-\infty}^{y-x} b_2(x')dx'\right) b_1(x) dx\\ &=& \int_{-\infty}^{\infty} \left(\frac{d}{dy} \int_{-\infty}^{y-x} b_2(x')dx'\right) b_1(x) dx\\ &=& \int_{-\infty}^{\infty} b_2(y-x) b_1(x) dx\\ \end{align}$

So we see the result is proved above.

Sum of Two Continuous Positive Random Variables

If $X i$ is positive, then $b i (x) = 0$ when $x < 0$ . Thus, $\int_{-\infty}^x b_i(x) dx = \int_0^x b_i(x) dx$ . Similarly, $b i (y - x) = 0$ when $x > y$ . Putting this together, for positive variables, $b_3(y) = \int_{-\infty}^\infty b_1(x)b_2(y-x) dx = \int_{0}^y b_1(x)b_2(y-x) dx$ .

Connection to Laplace Transform

For positive random variables, the Laplace transform can be helpful:

$B_i^*(s) = \int_0^\infty e^{-sx} b_i(x) dx$ .

Looking at the product of two such Laplace transforms. To do this, we will introduct a new variable, $x' = x + y$ :

$\begin{align} B_i^*(s)B_j^*(s) &=& \int_0^\infty\int_0^\infty e^{-sx} b_i(x) e^{-sy} b_j(y) dy dx\\ &=& \int_0^\infty\int_y^\infty e^{-sx'} b_i(x'-y) b_j(y) dx' dy\\ &=& \int_0^\infty e^{-s x'} \int_0^{x'} b_i(x'-y) b_j(y) dy dx'\\ &=& \int_0^\infty e^{-s x'} b_3(x') dx' = B_3^*(s) \end{align}$

Where we defined $b_3(x') = \int_0^{x'} b_i(x'-y) b_j(y) dy$ which is the course the pdf of a random variable which is the sum of two positive random variables, as we saw above. So, what's the lesson: the product of the laplace transforms, is the sum of the random variables (this result is also true for the z-transform for discrete variables). Of course, this means that $\left[ B_i^*(s) \right]^n$ is the LT of the pdf for the sum of n samples. What is $p(B_i^*(s))$ for a z-transform $p(z) = \sum_{i=0}^\infty p_k z^k$ ?