From BoykinWiki

EEL6507 Spring 2009, Lecture 10, Friday 2009/01/30 (Notes created by Vishnu Vijayakumar)

Derivation of Birth-Death Markov Chain
Pure - Birth process
Poisson process → Exponential random variable

Derivation of Birth-Death Markov equation

Let us consider the probability that there will be one birth at time interval $(t+\triangle t)$ given that the population is $\mathrm{K}\,\!$ at time $\mathrm{t}\,\!$

$\begin{alignat}{2} B_1(k) & = Prob[{\text{1 birth in }}(t,t+\triangle t)|K] \\ & = \int\limits_{t}^{t+\triangle t}p_K(\tau)\,d\tau \simeq \triangle t \cdot p_K(t) \end{alignat}$

where, $p_K (t)\,\!$ is the probability density that there is one birth at time t when the population is K.

Since birth is a memoryless process, $p_K (t)\,\!$ is independent of t. Thus we have:

$\begin{alignat}{2} B_{1}(k) & = Prob[{\text{1 birth in}}(t,t+\triangle t)] \simeq \lambda_K \triangle t \end{alignat}$

Similarly, for a pure death process we have:

$\begin{alignat}{2} D_{1}(k) & = Prob[{\text{1 death in}}(t,t+\triangle t)] \simeq \mu_K \triangle t \end{alignat}$

The probability of no births and no deaths in the interval $\triangle t$ are :

$\begin{array}{rcl} B_{0}(k) & = 1 - \triangle t \cdot \lambda_K \\ D_{0}(k) & = 1 - \triangle t \cdot \mu_K \\ \end{array}$

We now wish to derive the probability that the Population is K at a time $t+\triangle t$ . We have the following tools at our disposal:

Marginalization
Conditional Probability
Markov Property

$\begin{alignat}{2} P_k(t+\triangle t) & = \sum_{j=0}^{\infty}Prob[Population\ is\ K\ at\ T = t + \triangle t\ AND\ Population\ is\ j\ at\ T = t ]\\ & = \sum_{j=0}^{\infty}Prob[Pop.\ is\ K\ at\ T=\triangle t | Pop.\ is\ j\ at\ t] \cdot Prob[Pop.\ is\ j\ at\ t]\\ & = \sum_{j=k-1}^{k+1}Prob[Pop.\ is\ K\ at\ T=t+\triangle t | Pop.\ is\ j\ at\ t]\cdot P_j(t)\\ & = P_{k+1}(t)\mu_{k+1}\triangle t + P_k(t)[1 - \triangle t \mu_{k} - \triangle t \lambda_{k} + 2\triangle t^2 \lambda_{k}\mu_{k}]+ P_{k-1}(t)\lambda_{k-1}\triangle t\\ \end{alignat}$

Thus we have:

$\begin{alignat}{2} P_k(t+\triangle t)-P_k(t) = P_{k+1}(t)\mu_{k+1}\triangle t + P_{k}(t)[2\triangle t^2 \lambda_{k} \mu_{k}-\triangle t \mu_{k} - \triangle t \lambda_{k}] + P_{k-1}(t)\lambda_{k-1}\triangle t\\ \end{alignat}$

We can derive the equation for $\frac{dP_k(t)}{dt}$ from the above equation. Dividing both sides of the above equation and taking limits we have:

$\begin{alignat}{2} \lim_{\triangle t \to 0} \frac{P_k(t+\triangle t)-P_k(t)}{\triangle t} & = \lim_{\triangle t \to 0}\lbrace P_{k+1}(t)\mu_{k+1} + P_k(t)[2\triangle t\lambda_{k}\mu_{k}-(\mu_{k}+\lambda_{k})]+ P_{k-1}\lambda_{k-1}\triangle t \rbrace\\ \end{alignat}$

Thus we have:

$\begin{alignat}{2} P'_k(t) = P_{k+1}\mu_{k+1}+P_{k-1}\lambda_{k-1} - P_k(t)[\mu_{k}+\lambda_{k}]\\ \end{alignat}$

This is the equation for the Birth-Death Markov Chain.

Derivation of Birth-Death Markov Chain using the concept of Probability Flow

The Birth-Death equation can be derived using the concept of probability flow. We define Flow as he rate of probability flow out of $P_k(t)\,\!$

$\begin{array}{lll} F_{out} & = (\lambda_k + \mu_k)P_k(t)\\ F_{in} & = \mu_{k+1}P_{k+1}(t) + \lambda_{k-1}P_{k-1}(t)\\ P'_k(t) &= Flow_{in}-Flow_{out} \end{array}$