Authors
Saeid Haghighatshoar, Emmanuel Abbe, I Emre Telatar
Publication date
2014/4/14
Journal
IEEE transactions on information theory
Volume
60
Issue
7
Pages
3787-3796
Publisher
IEEE
Description
The entropy power inequality (EPI) yields lower bounds on the differential entropy of the sum of two independent real-valued random variables in terms of the individual entropies. Versions of the EPI for discrete random variables have been obtained for special families of distributions with the differential entropy replaced by the discrete entropy, but no universal inequality is known (beyond trivial ones). More recently, the sumset theory for the entropy function yields a sharp inequality H(X + X') - H(X) ≥ 1/2 - o(1) when X, X' are independent identically distributed (i.i.d.) with high entropy. This paper provides the inequality H(X + X') - H(X)≥ g(H(X)), where X, X' are arbitrary i.i.d. integer-valued random variables and where g is a universal strictly positive function on R + satisfying g(0) = 0. Extensions to nonidentically distributed random variables and to conditional entropies are also obtained.
Scholar articles
S Haghighatshoar, E Abbe, IE Telatar - IEEE transactions on information theory, 2014