16 EE5139R ProblemSet 1

1
EE5139R Communication Systems, Problem Set 1 Page 1 EE5139R, Problem Set 1 Q1. X and Y are two discrete random variables with joint distribution P (X =0,Y = 0) = P (X = 0,Y = 1) = P (X =1,Y = 0) = 1 3 . Compute H(X), H(Y ), H(X|Y ), H(Y |X) and H(X, Y ). Q2. A useful inequality - Prove that ln(x) x - 1. Also plot ln(x) and x - 1 on the same graph. Q3. The output of a discrete source consists of the possible letters x 1 ,x 2 , ··· ,x n , which occur with probability p 1 ,p 2 , ··· ,p n , respectively. Show that the entropy of the source is at most log(n). Hint: Use the ”useful inequality” you derived above. Q4. Let X be a random variable with entropy H(X) of 8 bits. Let Y (X) be a deterministic function of X and suppose that Y (X) is an invertible function. a. Compute H(Y ),H(Y |X),H(X|Y ), and H(X, Y ). b. If Y (X) is not invertible, what can you say about H(Y ) and H(X|Y )? Q5. X and Y are two discrete random variables with joint distribution P (x, y). Show that I (X; Y ) 0 with equality if and only if X and Y are statistically independent. Hint: Use the ”useful inequality” you derived above.

description

problems linear algebra

Transcript of 16 EE5139R ProblemSet 1

  • EE5139R Communication Systems, Problem Set 1 Page 1

    EE5139R, Problem Set 1

    Q1. X and Y are two discrete random variables with joint distribution P (X = 0, Y = 0) = P (X =0, Y = 1) = P (X = 1, Y = 0) = 13 . Compute H(X), H(Y ), H(X|Y ), H(Y |X) and H(X,Y ).

    Q2. A useful inequality - Prove that ln(x) x 1. Also plot ln(x) and x 1 on the same graph.Q3. The output of a discrete source consists of the possible letters x1, x2, , xn, which occur with

    probability p1, p2, , pn, respectively. Show that the entropy of the source is at most log(n).Hint: Use the useful inequality you derived above.

    Q4. Let X be a random variable with entropy H(X) of 8 bits. Let Y (X) be a deterministic functionof X and suppose that Y (X) is an invertible function.a. Compute H(Y ), H(Y |X), H(X|Y ), and H(X,Y ).b. If Y (X) is not invertible, what can you say about H(Y ) and H(X|Y )?

    Q5. X and Y are two discrete random variables with joint distribution P (x, y). Show that I(X;Y ) 0with equality if and only if X and Y are statistically independent. Hint: Use the useful inequalityyou derived above.