University of Illinois

Spring 2012

ECE 313: Problem Set 4: Problems and Solutions Geometric, Poisson, and Negative Binomial Distributions, Bernoulli Process, ML estimation, and Markov and Chebychev Inequalities
Due: Reading: Wednesday, February 15 at 4 p.m. ECE 313 Course Notes, Sections 2.52.9.

1. [Buering in Video Streaming] Suppose that you are watching a video clip consisting of thirty frames per second from YouTube. Each frame consists of 1000 packets. When a frame is about to be played out, if ve or more packets of the frame are lost (i.e., if you have received 995 or fewer packets of the frame), you will experience buering during that frame. Each packet is lost with the probability , independently of other packets. (a) Suppose that is unknown. If you have observed that 20 packets of a frame are lost, M L (20) of probability ? what is the maximum likelihood estimate Solution: From the solution to Example 2.8.1 in the lecture note, the maximum likeliM L (20) = 20 = 1 . hood estimate is 1000 50 (b) Suppose = 0.001. Using the Poisson approximation, nd the probability that you experience buering during a given frame. Solution: Let X be the number of lost packets of the frame, which is a binomial random variable with parameters 1000 and 0.001. Let Y be a Poisson random variable with parameter = 1000 103 = 1. By the Poisson approximation, the probability that ve or more packets of the frame are lost can be approximated by P (X 5) P (Y 5) = 1 P (Y < 5) = 1
4 e k

k! k=0 ( ) 1 1 1 1 =1 e1 1 + + + + = 0.00366. 1! 2! 3! 4!

2. [Insurance Company] An insurance company sold life insurance policies to 100,000 people for a premium of $500 each. Assume that the probability of death of each insured person during the contract term is 0.001 and the deaths occur independently. In the case of death, the company pays $200,000 to a designated beneciary. Let pL denote the probability that the company loses money for the contract term. (a) Find the upper bound on pL yielded by Markovs inequality. Solution: The income of the company by selling the insurance is $500 100, 000. Let X be the number of deaths among the insured people. Then, the company has to pay $200, 000 X . For the company to lose money, the net prot must be less than zero as follows: $500 100, 000 $200, 000 X < 0,

which is equivalent to X > 250. Hence, pL = P {X > 250}. Since X is a binomial random variable with parameters 100,000 and 0.001, the mean of X is 100. Applying Markovs inequality, we have P {X > 250} P {X 250} E [X ] 2 = . 250 5

(b) Find the upper bound on pL yielded by Chebychevs inequality. Solution: Since X is a binomial random variable with parameters 100,000 and 0.001, the mean and variance of X are given as below: E [X ] = 100, V ar(X ) = 99.9. Using Chebychevs inequality P (X > 250) P (|X 100| 150) = P (|X 100| aX ) (let a = 150/X ) 1 1 2 < (Since X < 10) a 225 You can see that Chebychevs inequality provides us with a tighter bound. 3. [Hypergeometric Distribution] Suppose there is a box containing m red balls and n m blue balls. When a sample of k balls is taken randomly from among the n balls in the box, let X represent the number of red balls in the sample. (a) Find the probability mass function pX (x) and mean E [X ] of X if the k balls are taken one at a time with replacement. Solution: When k balls are taken sequentially, let Xi = 1 if a red ball is taken at the i-th trial (1 i k ) and Xi = 0 if a blue ball is taken at the trial. Since replacement is X allowed, X1 , , Xk are independent Bernoulli random variables, and thus X = k i=1 i is a binomial random variable with parameters (k, m/n). Letting q = m n , the pmf of X is given by ( ) k i q (1 q )ki , if 0 i k , pX ( i) = i 0, otherwise. The mean of a binomial random variable with parameters (k, m/n) is E [X ] =
mk n .

(b) Suppose there are at least k blue balls and at least k red balls in the box, i.e., m k and n m k . We now assume that the k balls are taken without replacement. Find the probability mass function pX (x) and mean E [X ] of X . The distribution of X is known to be the hypergeometric distribution. To nd E [X ], you may need the following Chu-Vandermonde identity: for s min{w, h w}, ) s ( )( w hw
y =0

y 2

sy

( ) h = . s

Solution: Since there are enough blue and red balls to be chosen for the sample, X must be in {0, 1, , k }. Letting all the balls be distinct (which does not change ( ) the resulting probability), the number of ways to choose k balls among n balls is n k . The number (of)( ways ) to choose i balls among m red balls and k i balls among n m blue nm balls is m x kx . Hence, the pmf of X is ( )( ) m n m i ki (n) , if 0 i k , pX (i) = k 0, otherwise. The mean of X can be found as follows: ( )(nm) k i m i k i (n ) E [X ] =
i=0

1 = (n ) =

= = =

( )( )) m nm i i ki k i=1 ( k ) m! i (n m) 1 (n ) i!(m i)! k i k i=1 ( k ) m (m 1)! (n m) 1 (n ) (i 1)!(m i)! k i k i=1 ( k ( )( )) m m1 nm (n ) i1 ki k i=1 ( k 1 ( )( )) m m1 nm (n ) by letting i = i 1 i k 1 i k i =0 ( ) m n1 (n ) by the Chu-Vandermonde identity k1 k mk . n
k

When applying the Chu-Vandermonde identity, we have used y = x , s = k 1, w = m1, and h = n 1. The mean of X in (b) is identical to the mean of X in (a). 4. [Unreliable Wireless Communications and Opportunistic Scheduling] Suppose that a smart phone attempts to transmit a packet to a base station using wireless communications as shown in Fig. 1. Due to unreliability of the wireless channel, the base station successfully receives the packet with probability q (0, 1), independently of outcomes of prior transmissions. The smart phone knows whether or not the packet is successfully received by the base station right after the transmission. If the packet is not received, then the smart phone immediately retransmits. (a) Let X be the number of transmission attempts until a packet is successfully delivered to the base station. Find the pmf, mean, and variance of X . Solution: Since X is a geometric random variable with parameter q , we have pX (k ) = q (1 q )k1 , for k = 1, 2, 3

Figure 1: Packet transmission from a smart phone to a cell tower E (X ) = 1/q, and V ar(X ) = 1q . q2

(b) Suppose the smart phone has r packets to send, one at a time. Let Y be the number of transmission attempts until all r packets are successfully transmitted. Find the pmf, mean, and variance of Y . Solution: Y is a negative binomial distribution with parameters r and q . Thus, ( ) k1 r pY (k ) = q (1 q )kr , for k = r, r + 1, r + 2, , r1 r E (Y ) = , q and V ar(Y ) = r(1 q ) . q2

(c) Now assume that there are M base stations around the smart phone as shown in Fig. 2. When the smart phone transmits a packet, each base station successfully receives the packet with probability q , independently of the other base stations and prior transmissions. We say that the smart phone successfully delivers a packet if at least one of the M base stations successfully receives the packet. Let Q be the number of transmission attempts until a packet is successfully delivered. Find the pmf, mean, and variance of Q. Solution: For a given transmitted packet, all the base stations will fail to receive a packet with probability (1 q )M . Hence, the probability that the packet is successfully delivered is q = 1 (1 q )M . Since Q is a geometric random variable with parameter q , we have pQ (k ) = q (1 q )k1 , for k = 1, 2, E (Q) = 1/q , and V ar(Q) = 1 q . q 2

Figure 2: Opportunistic Scheduling: Packet transmission from a smart phone to multiple cell towers (d) Suppose the smart phone has r packets to send, one at a time. For each packet, the smart phone keeps transmitting the packet until one of the M base stations successfully receives the packet. Let Z be the number of transmission attempts until r packets are successfully delivered. Find the pmf, mean, and variance of Z . Further, show that when q 0, E [Y ] in (b) can be approximated by M E [Z ], i.e., when the channel condition is pretty bad, the expected delay to deliver r packets as in (d) is M times smaller that the expected delay as in (b). (Hint: Use LHopitals rule: If limtc f (t) = limtc g (t) = and f (t) and g (t) exist near t = c, then f ( t) f (t) = lim . tc g (t) tc g (t) lim Solution: Z is a negative binomial distribution with parameters r and q = 1 (1 q )M . Thus, ( ) k 1 r pZ (k ) = q (1 q )kr , for k = r, r + 1, r + 2, , r1 r E (Z ) = , q and V ar(Z ) = Using LHopitals rule, E [Y ] 1 (1 q )M = lim q 0 E [Z ] q 0 q M (1 q )M 1 = lim = M. q 0 1 lim 5 r(1 q ) . q 2

5. [Matlab Simulation: Unreliable Wireless Communications and Opportunistic Scheduling] Verify your analysis in the previous problem using a Matlab simulation. In this simulation, we will generate the samples of Bernoulli processes, geometric random variables, and negative binomial random variables. The hints for generating these samples are described as below: A length T segment of a Bernoulli process with parameter q can be obtained using B=( rand(1, T)<q) for some large enough T. The indices in B where the corresponding Bernoulli random variable is one can be obtained using S=find(B==1). Here length(S) will be around qT. The intervals between ones can be computed using S(2:length(S))-S(1:(length(S)-1)). The result is a row vector of length length(S)-1 lled with geometric random variables with parameter q . The indices when every r-th one occurs in the Bernoulli process can be obtained by C=S(r:r:length(S)). The length will be around qT /r. The intervals between these timeslots can be obtained using C(2:length(C))-C(1:(length(C)-1)). The result is a row vector lled with negative binomial random variables with parameters r and q. (a) For q=0.3 and T=floor(2000/q), generate the samples of X in 4(a). Display the distributions of X using hist(). (b) For r=300, q=0.3, and T=floor(2000*r/q), generate the samples of Y in 4(b). Display the distributions of Y in 4(b) using hist().