Second assignment 140705010027 Tracy Wu

Source: Internet
Author: User

1. set X is a random variable, and the value range is a set of symbols containing M letters. Proof 0<=h(x)<=log2M.

Because of the entropy of all probability distributions p,
Maximum at equal probabilities and Hmax (X) =log2m
So H (X) <=log2m.
Because X is a random variable, the probability of occurrence is P (x).
The axiomatic definition of probability is as follows: 0<=p (X) <=1
Known h (x) =-e P (x) *LOGP (x) h (x) >=0
So 0<=h (X) <=log2m

2. prove that if the element of a sequence is observed as the IID distribution, the entropy of the sequence is equal to the first order entropy.

Certificate: because H (x) =limn→∞ (1/N*GN), the

Gn=-σi=n ... Σi=1 P (Ai) logp (x1=i1,x2=i2,..., xn=in) * log (X1=i1,x2=i2,..., xn=in)

When the sequence is an iid distribution, the

gn=-nσi=1p (X1=I1) Logp (X1=I1)

so H (x) =-σp (x1) LOGP (x1) is the first order entropy

3. given the symbol set a={a1,a2,a3,a4}, the first-order entropy is obtained under the following conditions:

(a) p(a 1)= P(a2)= P( a3)= P(a4)=1/4

Solution: H (a) =-4X1/4 log (section) =-log (1/4) =-(-2)

H (a) =2 bit/characters

(b) p(a1)=1/2, P(a2)=1/4, P( a3)= P(a4)=1/8

Solution: H=-∑p (AI) logp (AI) =-1/2*LOG1/2-1/4*LOG1/4-2*1/8*LOG1/8

h=7/4=1.75 bit/Characters

(c) P(a1)=0.505, P(a2)=1/4, p ( a3)=1/8, P(a4)=0.12

Solution: H=-∑p (AI) logp (AI) =-0.505*log0.505-1/4*log1/4-1/8*log1/8-0.12*log0.12

h=1.135 bit/Characters

Second assignment 140705010027 Tracy Wu

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.