Advanced statistics
I need help on questions 2,4,6 and 7
STAT 4382: Homework 3
Due by September 28, 2020
Yuly Koshevnik
First Name Last Name
Do Not Write Below This Line, PLEASE!
Problem 1 2 3 4 5 6 7 8 Total
Points
Maximum 15 10 15 10 15 10 15 10 100
Please submit the title page with your homework!
1
I need help on questions 2,4,6 and 7
Problem 1 [15 points = 5 + 5 + 5]
Consider a Markov chain associated with FOUR umbrellas, instead of similar to three, which was consid-
ered in the notes. Assume that rains occur with the same probability, p, during the day, and this occurrence
does not depend on the process, X = {X(n) : n 0}
1. Show transition probabilities for the state space S = {1, 2, 3, 4}.
2. Derive stationary distribution for the number of umbrellas.
3. Evaluate limiting expectation,
lim
n
E [X(n)]
Solution
2
Problem 2 [10 points = 5 + 5]
Proceed with FOUR umbrellas from Problem 1.
In addition, assume that the constant probability of rain is p = 0.5.
1. Suppose that initial distribution of X(0) is stationary, determine E [X(0)]
2. Under the limiting distribution of X(n), find the chance that an umbrella is carried at the time n.
Solution
3
Problem 3 [15 points = 5 + 5 + 5]
Consider a Markov chain with SIX states,
S = {0, 1, 2, 3, 4, 5}
and transition probabilities defined as follows:
P0,0 =
1
3
, P0,1 = 0, P0.2 =
1
3
, P0,3 = P0.4 = 0, = P0.5 =
1
3
P1,0 =
1
2
, P1,1 = P1,2 =
1
4
, P1,3 = P1,4 = P1,5 = 0
P2,0 = P2,1 = P2,2 = P2,3 = 0, P2,4 = 1, P2,5 = 0
P3,0 = P3,1 = P3,2 =
1
4
, P3,3 = P3,4 = 0 P3,5 =
1
4
P4,0 = P4,1 = 0, P4,2 = 1, P4,3 = P4,4 = P4,5 = 0
P5,0 = P5,1 = P5,2 = P5,3 = P5,4 = 0, P5,5 = 1
1. Find all transient states
2. Determine recurrent states
3. Which states are absorbing?
Solution
4
Problem 4 [10 points]
Proceed with the transition probabilities described in Problem 3.
Determine periodic states with period d > 1
Solution
5
Problem 5 [15 points = 5 + 10]
Consider a Markov chain with the state space S = {0, 1, 2, 3} and transition probabilities as follows:
P0,1 = P0,3 =
1
2
and P0.0 = P0.2 = 0
P1,2 = 1 and P1,0 = P1,1 = P1,3 = 0
P2,0 = P2,1 = P2,2 = 0 and P2,3 = 1
P3,0 = P3,3 =
1
2
and P3,1 = P3,2 = 0
1. Which states are recurrent and which of them are transient?
2. Determine the limiting matrix, limn P
n
Solution
6
Problem 6 [10 points = 6 + 4]
Consider a Markov chain, X = {X(n) : n 0}, describing three types of a failure of some device. There
are FOUR states, S = {0, 1, 2, 3}, where 0 means normal functioning, while three other states indicate
the failure type. Transition probability matrix, P, is given as follows:
0 1 2 3
0 a0 a1 a2 a3
1 b1 1 b1 0 0
2 b2 0 1 b2 0
3 b3 0 0 1 b3
Notice that only one type of a failure can occur within a time unit, and values bi determine the probability
of restoration within the next unit. All values ai and bi are such that 0 < ai < 1 and 0 < bi < 1.
1. Find limn P
n
2. In a long run, determine the proportion of time this device will be functioning.
Solution
7
Problem 7 [15 points = 5 + 5 + 5]
Consider a Markov chain with the state space, S = {0, 1} and transition probabilities
P0,0 = 1 a and P0,1 = a
P1,0 = b, and P1,1 = 1 b,
where 0 < a < 1 and 0 < b < 1. Consider a random variable, M, the first return time to the initial state
X(0) = 0.
1. Find P [M = 1|X(0) = 0]
2. For any m 2, derive P [M = m], which is equivalent to
P
km1
[X(k) = 1]
(X(m) = 0)|X(0) = 0
3. Derive E [M|X(0) = 0].
Notice that the straightforward calculation leads to
E [M|X(0) = 0] =
Solution
8
Problem 8 [10 points]
Consider a periodic Markov chain with the state space S = {0, 1, 2, 3} and transition probability matrix
0 1 2 3
0 1 p p 0 0
1 1 p 0 p 0
2 1 p 0 0 p
3 1 0 0 0
Determine the limiting probabilities:
lim
n
P [X(n) = j|X(0) = i]
for all (i, j)
Solution
9
