FREE AND CLASSICAL ENTROPY OVER THE CIRCLE

FREE AND CLASSICAL ENTROPY OVER THE CIRCLE

GORDON BLOWER

DEPARTMENT OFMATHEMATICS ANDSTATISTICS

LANCASTERUNIVERSITY

LANCASTERLA1 4FY ENGLANDUK.

g.blower@lancaster.ac.uk

Received 12 September, 2006; accepted 17 February, 2007 Communicated by F. Hansen

ABSTRACT. Relative entropy with respect to normalized arclength on the circle is greater than or equal to the negative logarithmic energy (Voiculescu’s negative free entropy) and is greater than or equal to the modified relative free entropy. This note contains proofs of these inequalities and related consequences of the first Lebedev–Milin inequality.

Key words and phrases: Transportation inequality; Free probability; Random matrices.

2000 Mathematics Subject Classification. 60E15; 46L54.

1. INTRODUCTION ANDDEFINITIONS

In this note we consider inequalities between various notions of relative entropy and related metrics for probability measures on the circle. The introduction contains definitions and brief statements of results which are made precise in subsequent sections.

Definition 1.1. Forµandνprobability measures onTwithνabsolutely continuous with respect to µ, let dν/dµ be the Radon–Nikodym derivative. The (classical) relative entropy ofν with respect toµis

(1.1) Ent(ν |µ) =

Z

T

log dν dµdν;

note that0 ≤Ent(ν | µ) ≤ ∞by Jensen’s inequality; we take Ent(ν | µ) = ∞whenνis not absolutely continuous with respect toµ.

Definition 1.2. Letρbe a probability measure onRthat has no atoms. If the integral

(1.2) Σ(ρ) =

Z Z

R²

log|x−y|ρ(dx)ρ(dy)

converges absolutely, thenρhas free entropyΣ(ρ), that is, the logarithmic energy.

I am grateful to Profs. Hiai and Ueda for helpful communications and to Prof Ledoux for pointing out some references. The research was partially supported by the project ‘Phenomena in High Dimensions MRTN-CT-2004-511953’.

249-06

Voiculescu [14] introduced this along with other concepts of free probability; see also [3], [5], [6], [8], where various notations and constants are employed.

In Theorem 2.1 we compare free with relative entropy with respect to arclength measure dθ/2πonTand show thatρ(dθ) =p(e^iθ)dθ/2πsatisfies

(1.3) −Σ(ρ)≤Ent(ρ|dθ/2π).

The proof involves the sharp Hardy–Littlewood–Sobolev inequality.

Definition 1.3. Suppose that f and g are probability density functions with respect to dθ/2π, and let

(1.4) Σ(f, g) = Z Z

T²

log 1

|e^iθ−e^iφ| f(e^iθ)−g(e^iθ)

f(e^iφ)−g(e^iφ) dθ 2π

dφ 2π be the modified relative free entropy as in [5], [6], [7], [8].

For notational convenience, we identify an absolutely continuous probability measure with its probability density function and writeIfor the constant function1. In Theorem 2.2 we show thatΣ(f,I)≤Ent(f |I).The proof uses the first Lebedev–Milin inequality for functions in the Dirichlet space over unit discD. Letu : D → R be a harmonic function such thatk∇u(z)k² is integrable with respect to area measure, letv be its harmonic conjugate withv(0) = 0and g = (u+iv)/2. Then by [10],usatisfies

(1.5) log

Z

T

exp u(e^iθ)dθ 2π ≤ 1

4π Z Z

D

k∇u(re^iθ)k²rdrdθ+ Z

T

u(e^iθ)dθ 2π;

thusexpg belongs to the Hardy space H²(D). One can interpret this inequality as showing that H²(D)is the symmetric Fock space of Dirichlet space, which is reflected by the reproducing kernels, as in [12].

Definition 1.4. Letµand ν be probability measures on T. Then the Wasserstein pmetric for 1≤p <∞and the cost function|e^iθ−e^iφ|^p/pis

(1.6) W_p(µ, ν) = inf

ω

( 1 p

Z Z

T²

|e^iθ−e^iφ|^pω(dθdφ) _p¹)

,

whereωis a probability measure onT² that has marginalsµandν. See [13].

Letu: T→Rbe a1-Lipschitz function in the sense that|u(e^iθ)−u(e^iφ)| ≤ |e^iθ −e^iφ|for alle^iθ, e^iφ ∈T, and suppose further thatR

Tu(e^iθ)dθ/2π= 0. Then by (1.6), as reformulated in (3.2) below, we have

(1.7)

Z

T

exp tu(e^iθ)dθ

2π ≤exp t²

2

(t∈R).

Bobkov and Götze have shown that the dual form of this concentration inequality is the trans- portation inequalityW₁(ρ, dθ/2π)² ≤ 2Ent(ρ | dθ/2π)for all probability measuresρof finite relative entropy with respect todθ/2π, as in [13], 9.3. In Section 3 we provide a free transporta- tion inequalityW₁(ρ, ν)² ≤2Σ(ρ, ν)which generalizes and strengthens this dual inequality.

2. FREEVERSUSCLASSICAL ENTROPY WITH RESPECT TOARCLENGTH

For completeness, we recall the following result of Beckner and Lieb [2].

Theorem 2.1. Suppose thatf is a probability density function onRsuch that flogf is inte- grable. Thenf has finite free entropy and

(2.1)

Z Z

R²

log 1

|x−y|f(x)f(y)dxdy≤log 2π+ Z

R

f(x) logf(x)dx.

Proof. The sharp form of the Hardy–Littlewood–Sobolev inequality, due to Lieb [2], gives (2.2)

Z Z

R²

f(x)f(y)

|x−y|^λ dxdy ≤π^3/2−2/pΓ(1/p−1/2) Γ(1/p)

Z

R

|f(x)|^pdx ²_p

,

for λ = 2(1−1/p) with 1 ≤ p < 2, and with equality whenp = 1. Hence the derivative at p = 1+of the left-hand side is less than or equal to the derivative of the right-hand side.

By differentiating Legendre’s duplication formula Γ(2x)Γ(1/2) = 2^2x−1Γ(x)Γ(x + 1/2) at x= 1/2, we obtain

(2.3) Γ⁰(1)/Γ(1) = 2 log 2 + Γ⁰ 1

2 Γ

1 2

, and hence we obtain the derivative of the numerical factor in (2.2).

This gives (2.1); to deduce (1.3), we take f(θ) = p(e^iθ)I[0,2π](θ)/2π where ρ(dθ) =

p(e^iθ)dθ/2π.

In [7] the authors assert that the relative and free entropies with respect to arclength are incomparable, contrary to Theorem 2.2 below and (1.3). Whereas the values of the entropies of their attempted counterexample are correct on [7, p. 220] and [5, p. 204], the limit on [7, p.

220, line 7] should be1and not0; so the calculation fails. The calculation on [7, p. 219] does show that (1.3) has no reverse inequality.

Definition 2.1. With realαand Fourier coefficientsf(n) =ˆ R

Tf(e^iθ)e^−inθdθ/2π, letH^α(T)be the subspace ofL²(T)consisting of thosef such that

(2.4) kfk_H^α_(T) = X

n∈Z

(1 +|n|^2α)|fˆ(n)|²

!¹₂

is finite, and letH˙^α(T)be the completion of the subspace {f ∈ H^α(T) : ˆf(0) = 0} for the norm

(2.5) kfkH˙^α(T) =



 X

n∈Z\{0}

|n|^2α|f(n)|ˆ ²





1 2

;

we use the notationkfkH˙^α(T)to indicate the semi-norm defined by this sum for typical elements ofH^α(T).

There is a natural pairing of H˙^α(T) with H˙^−α(T) whereby g(e^iθ) ∼ P

n∈Z\{0}b_ne^inθ in H˙^−α(T)defines a bounded linear functional onH˙^α(T)by

(2.6) X

n∈Z\{0}

a_ne^inθ 7→ X

n∈Z\{0}

a_n¯b_n.

Whenp andq are probability density functions of finite relative free entropy, their difference f =p−qbelongs toH˙^−1/2(T)and is real; so when we take the Taylor expansion of the kernel

in (1.4) we deduce that

(2.7) kp−qk²_H˙−1/2(T) = X

n∈Z\{0}

fˆ(n) ˆf(−n)

|n| = 2

∞

X

n=1

|fˆ(n)|²

n = 2Σ(p, q), as in [8, p. 716].

Theorem 2.2. Letf be a probability density function onTthat has finite relative entropy with respect todθ/2π. Then

(2.8) Σ(f,I)≤Ent(f |I).

Proof. We consider harmonic extensions of L²(T) to the unit disc. Let uφ(e^iθ) = u(e^iθ−iφ) and let u(re^iθ) = R

TP_r(e^iφ)u_φ(e^iθ)dφ/2π be the Poisson extension of u, where P_r(e^iθ) = P

n∈Zr^|n|e^inθ. The dual space of H˙^−1/2(T) under the pairing of (2.6) isH˙^1/2(T), which we identify with the Dirichlet spaceGof harmonic functionsu:D→Rsuch thatR

Tu(e^iθ)dθ/2π = 0and

(2.9)

Z Z

D

k∇uk²dxdy/π <∞.

By the joint convexity of relative entropy [4], any pair of probability density functions of finite relative entropy satisfies

(2.10) Ent(f |u) =

Z

T

P_r(e^iφ)Ent(f_φ|u_φ)dφ

2π ≥Ent(P_rf |P_ru);

so, in particular,

(2.11) Ent(f |I)≥Ent(P_rf |I) (0≤r <1).

Hence it suffices to prove the theorem for P_rf instead of f, and then take limits as r → 1−.

For notational simplicity, we shall assume thatf has a rapidly convergent Fourier series so that various integrals converge absolutely.

Suppose thatuis a real function inH^1/2(T)that hasR

Tu(e^iθ)dθ/2π=−tandkukH˙^1/2(T) = s; by adding a constant touif necessary, we can assume thats²/2 =t. Then by (1.5) we have (2.12)

Z

T

expu(e^iθ)dθ

2π ≤exp s²

2 −t

= 1, and consequently by the dual formula for relative entropy

Z

T

f(e^iθ) logf(e^iθ) dθ

2π = sup Z

T

h(e^iθ)f(e^iθ)dθ 2π :

Z

T

exph(e^iθ)dθ 2π ≤1

(2.13)

≥ Z

T

f(e^iθ)u(e^iθ)dθ 2π. Recalling the dual pairing ofH˙^−1/2(T)withH˙^1/2(T), we write

(2.14) hf, ui=

Z

T

f(e^iθ)u(e^iθ)dθ 2π −

Z

T

f(e^iθ)dθ 2π

Z

T

u(e^iθ)dθ 2π, so that by (2.13)

(2.15) hf, ui ≤t+

Z

T

f(e^iθ) logf(e^iθ)dθ 2π.

We choose theu(n)ˆ forn6= 0to optimize the left-hand side, and deduce that (2.16) kfkH˙^−1/2(T)kukH˙^1/2(T) =skfkH˙^−1/2(T)≤s²/2 +

Z

T

f(e^iθ) logf(e^iθ)dθ 2π,

so by choosingswe can obtain the desired result (2.17) 2Σ(f,I) =kfk²_H˙−1/2(T) ≤2

Z

T

f(e^iθ) logf(e^iθ) dθ 2π.

The quantity Ent(I | w)also appears in free probability, and the appearance of the formula (1.5) likewise becomes unsurprising when we recall the strong Szegö limit theorem. Let w : T → R+ be a probability density with respect todθ/2π such thatu(e^iθ) = logw(e^iθ)belongs toH^1/2(T), letDn= det[ ˆw(j−k)]0≤j,k≤n−1be the determinants of then×nToeplitz matrices associated with the symbolw, and let

(2.18) α_n= exp

(n+ 1) Z

T

u(e^iθ)dθ 2π + 1

4π Z Z

D

k∇u(z)k²dxdy

(n = 0,1, . . .).

Then by (1.5), we haveα₀ ≥1sinceR

w(e^iθ)dθ/2π = 1; further (2.19) D_n^1/n →exp

Z

T

u(e^iθ)dθ 2π

= exp

−Ent(I|w)

(n→ ∞) by [11, p. 169] and by Ibragimov’s Theorem [11, p. 342],

(2.20) D_n/α_n→1 (n→ ∞).

One can refine the proof given in [1] and prove the following result on the asymptotic distribu- tion of linear statistics. Letf be a real function inH^1/2(T)and letX_n : (U(n), µ_U(n))→Rbe the random variable

(2.21) X_n(γ) = trace(f(γ))−n Z

T

f(e^iθ)dθ

2π (γ ∈U(n)),

where µ_U(n) is the Haar measure on the group U(n) of n ×n unitary matrices. Then (Xn) converges in distribution asn→ ∞to a Gaussian random variable with mean zero and variance kfk²_˙

H^1/2(T).

3. A SIMPLEFREE TRANSPORTATION INEQUALITY

Theorem 3.1. Suppose thatpandqare probability density functions with respect todθ/2πsuch that their relative free entropy is finite. Then

(3.1) W₁(p, q)² ≤2Σ(p, q).

Proof. By the Kantorovich–Rubinstein theorem, as in [13, p. 34], (3.2) W₁(p, q) = sup

u

Z

T

u(e^iθ) p(e^iθ)−q(e^iθ)dθ

2π :|u(e^iθ)−u(e^iφ)| ≤ |e^iθ−e^iφ|

.

Any such1–Lipschitz functionubelongs toH^1/2(T), since we have

(3.3) X

n∈Z

|n||ˆu(n)|² = Z Z

T²

u(e^iθ)−u(e^iφ) e^iθ −e^iφ

2 dθ 2π

dφ 2π ≤1,

by [11, 6.1.58]. Hence by the duality betweenH˙^1/2(T)andH˙^−1/2(T), we have W₁(p, q)≤sup

u

Z

T

u(e^iθ) p(e^iθ)−q(e^iθ) dθ

2π :kukH˙^1/2(T) ≤1 (3.4)

=kp−qkH˙^−1/2(T).

In [6] and [7], Hiai, Petz and Ueda prove a transportation inequality for W₂ by means of a difficult matrix approximation argument. Whereas transportation inequalities involving W₂ generally imply transportation inequalities forW₁by the Cauchy–Schwarz inequality, Theorem 3.1 has the merit that it applies to a wide class ofpandq and involves the uniform constant2.

Villani [13, p. 234] compares theW₂ metric with theH⁻¹norm, and Ledoux [9] obtains a free logarithmic Sobolev inequality using a proof based upon the Prékopa–Leindler inequality.

REFERENCES

[1] E.L. BASOR, Toeplitz determinants, Fisher–Hartwig symbols and random matrices, pp. 309–336 in Recent Perspectives in Random Matrix Theory and Number Theory, Eds. F. Mezzadri and N.C.

Snaith, Cambridge University Press, 2005.

[2] W. BECKNER, Sharp Sobolev inequalities on the sphere and the Moser–Trudinger inequality, Ann.

of Math. (2), 138 (1993), 213–242.

[3] P. BIANE AND D. VOICULESCU, A free probability analogue of the Wasserstein metric on the trace-state space, Geom. Funct. Anal., 11 (2001), 1125–1138.

[4] E.A. CARLENANDM.C. CARVALHO, Strict entropy production bounds and stability of the rate of convergence to equilibrium for the Boltzmann equation, J. Statist. Phys., 67 (1992), 575–608.

[5] F. HIAIANDD. PETZ, The Semicircle Law, Free Random Variables and Entropy, American Math- ematical Society, Rhode Island, 2000.

[6] F. HIAIANDD. PETZ, A free analogue of the transportation cost inequality on the circle, in Quan- tum Probability, Edrs. M. Bozejko, W. Mlotkowsky and J. Wysoczansky, Banach Center Publica- tions, Vol. 73, Warsaw, 2006, 199–206.

[7] F. HIAI, D. PETZANDY. UEDA, Free transportation cost inequalities via random matrix approxi- mation, Probab. Theory Relat. Fields, 130 (2004), 199–221.

[8] F. HIAI, M. MIZUNO AND D. PETZ, Free relative entropy for measures and a corresponding perturbation theory, J. Math. Soc. Japan, 54 (2002), 670–718.

[9] M. LEDOUX, A (one-dimensional) free Brunn–Minkowski inequality, C. R. Math. Acad. Sci. Paris, 340 (2005), 301–304.

[10] B. OSGOOD, R. PHILLIPSANDP. SARNAK, Extremals of determinants of Laplacians, J. Funct.

Anal., 80, (1988) 148–211.

[11] B. SIMON, Orthogonal Polynomials on the Unit Circle, Part 1: Classical Theory, American Math- ematical Society, 2005.

[12] V.I. VASYUNINANDN.K. NIKOLSKII, Operator-valued measures and coefficients of univalent functions, St Petersburg Math. J., 3 (1992), 1199–1270.

[13] C. VILLANI, Topics in Optimal Transportation, American Mathematical Society, 2003.

[14] D. VOICULESCU, The analogues of entropy and of Fisher’s information measure in free proba- bility I, Comm. Math. Phys., 115 (1993), 71–92.