EEL6507sp09L09

From BoykinWiki

Jump to: navigation, search

Contents

EEL6507 Spring 2009, Lecture 09, Wednesday 2008/01/28 (Notes created by Dexiang Wang)

Move from the discrete-time Markov chain to its continuous-time version

Single Random Variable of Continuous-time Memorylessness (Continuation of last class)

Following the discussion in the last lecture, we already have

P\left[\tau>t+s|\tau>s\right]=h(t)

Here

P\left[\tau>0\right]=1

and hence τ is always positive

P\left[\tau>t|\tau>0\right]=P\left[\tau>t\right]=h(t)

Finally we have

h\left(t+s\right)=h(t)h(s)

So, we need to find a function which can satisfy this feature. Doing the derivative on h\left(t\right), we have:

\begin{align} h'(t)&= \lim_{\Delta t \to 0}\frac {h(t+\Delta t)-h(t)}{\Delta t} \\ &= \lim_{\Delta t \to 0}\frac {h(t)h(\Delta t)-h(t)}{\Delta t} \\ &= h(t)\lim_{\Delta t \to 0}\frac {h(\Delta t)-1}{\Delta t} \\ &= h(t)\lim_{\Delta t \to 0} h'(\Delta t) \\ &= h(t)h'(0) \\ &\equiv h(t)(-\lambda) \end{align}

Now,we turn to solve the first-order derivative equation and solution of it is as follow:

h'\left(t\right)=e^{-\lambda t}

This is the only way to give a memoryless function.

Continuous Time Markov Chain

Now, we can move onto continuous version of Chapman-Kolmogorov equation with substitution of discrete time steps by continuous time points:

P_{ij}\left(s,t\right)=P[X(t)=j|X(s)=i],\ for\ t\ge s

Leveraging marginalization:

P_{ij}\left(s,t\right)=\sum_{k} P_{ik}(s,u)P_{kj}(u,t)

here we stop at middle time point u and at different state k. So, we get continuous-time Chapman-Kolmogorov equation as follow:

H\left(s,t\right)=H(s,u)H(u,t)

Its special form can be derived as follow:

H\left(s,t+\Delta t \right)-H(s,t)=H(s,t)H(t,t+ \Delta t)-H(s,t)

Hence,

\begin{align} \frac{\partial}{\partial t} H(s,t)&=\lim_{\Delta t \to 0}\frac {H(s,t+ \Delta t)-H(s,t)}{\Delta t} \\ &= H(s,t)[\lim_{\Delta t \to 0}\frac {H(t,t+ \Delta t)-I}{\Delta t}] \\ &\equiv H(s,t)Q(t) \end{align}

The solution to this matrix derivative equation is:

H\left(s,t\right)=\exp(\int_{s}^{t}Q(\tau)d\tau)

Birth-Death Process

The birth-death process is a special case of a Markov process in which transitions from state E_{k}\ are permitted only to neighboring states E_{k+1},\ E_k\ and\ E_{k-1}. This restriction permits us to carry the solution much further. Here we have some terminology definitions:

  • Birth rate \left(\lambda_k\right): describes the rate at which births occur when the population is of size k
  • Death rate \left(\mu_k\right): describes the rate at which deaths occur when the population is of size k

Hence we can derive the matrix \Q based on features of birth-death process and stochastic matrices as follow:

Q= \begin{bmatrix} -\lambda_0 & \lambda_0 & 0 & 0 & 0 & \cdots \\ \mu_1 & -(\lambda_1+\mu_1) & \lambda_1 & 0 & 0 & \cdots \\ 0 & \mu_2 & -(\lambda_2+\mu_2) & \lambda_2 & 0 & \cdots \\ 0 & 0 & \mu_3 & -(\lambda_3+\mu_3) & \lambda_3 & \cdots \\ \vdots & \vdots & \vdots & \vdots & \vdots & \ddots \end{bmatrix}

Lecture Voice Recording

lecture_recording

Retrieved from "http://boykin.acis.ufl.edu/wiki/index.php/EEL6507sp09L09"