(1)http://jipam.vu.edu.au/ Volume 2, Issue 1, Article 5, 2001 FURTHER REVERSE RESULTS FOR JENSEN’S DISCRETE INEQUALITY AND APPLICATIONS IN INFORMATION THEORY I

http://jipam.vu.edu.au/

Volume 2, Issue 1, Article 5, 2001

FURTHER REVERSE RESULTS FOR JENSEN’S DISCRETE INEQUALITY AND APPLICATIONS IN INFORMATION THEORY

I. BUDIMIR, S.S. DRAGOMIR, AND J. PE ˇCARI ´C

DEPARTMENT OFMATHEMATICS, FACULTY OFTEXTILETECHNOLOGY, UNIVERSITY OFZAGREB, CROATIA.

SCHOOL OFCOMMUNICATIONS ANDINFOMATICS, VICTORIAUNIVERSITY OFTECHNOLOGY, P.O. BOX

14428, MELBOURNECITYMC, VICTORIA8001, AUSTRALIA. sever.dragomir@vu.edu.au

URL:http://rgmia.vu.edu.au/SSDragomirWeb.html

DEPARTMENT OFAPPLIEDMATHEMATICS, UNIVERSITY OFADELAIDE, ADELAIDE, 5001.

URL:http://mahazu.hazu.hr/DepMPCS/indexJP.html

Received 12 April, 2000; accepted 6 October, 2000 Communicated by L.-E. Persson

ABSTRACT. Some new inequalities which counterpart Jensen’s discrete inequality and improve the recent results from [4] and [5] are given. A related result for generalized means is estab- lished. Applications in Information Theory are also provided.

Key words and phrases: Convex functions, Jensen’s Inequality, Entropy Mappings.

2000 Mathematics Subject Classification. 26D15, 94Xxx.

1. INTRODUCTION

Let f : X → R be a convex mapping defined on the linear space X and x_i ∈ X, p_i ≥ 0 (i= 1, ..., m)withPm :=Pm

i=1pi >0.

The following inequality is well known in the literature as Jensen’s inequality

(1.1) f 1

P_m

m

X

i=1

pixi

!

≤ 1 P_m

m

X

i=1

pif(xi).

There are many well known inequalities which are particular cases of Jensen’s inequality, such as the weighted arithmetic mean-geometric mean-harmonic mean inequality, the Ky-Fan in- equality, the Hölder inequality, etc. For a comprehensive list of recent results on Jensen’s in- equality, see the book [25] and the papers [9]-[15] where further references are given.

In 1994, Dragomir and Ionescu [18] proved the following inequality which counterparts (1.1) for real mappings of a real variable.

ISSN (electronic): 1443-5756

c 2001 Victoria University. All rights reserved.

007-00

Theorem 1.1. Letf : I ⊆ R→Rbe a differentiable convex mapping on

◦

I (

◦

I is the interior ofI),xi ∈

◦

I, pi ≥0 (i= 1, ..., n)andPn

i=1pi = 1. Then we have the inequality

0 ≤

n

X

i=1

p_if(x_i)−f

n

X

i=1

p_ix_i

! (1.2)

≤

n

X

i=1

p_ix_if⁰(x_i)−

n

X

i=1

p_ix_i

n

X

i=1

p_if⁰(x_i),

wheref⁰ is the derivative off on

◦

I.

Using this result and the discrete version of the Grüss inequality for weighted sums, S.S.

Dragomir obtained the following simple counterpart of Jensen’s inequality [5]:

Theorem 1.2. With the above assumptions forfand ifm, M ∈I^◦andm ≤x_i ≤M(i= 1, ..., n), then we have

(1.3) 0≤

n

X

i=1

p_if(x_i)−f

n

X

i=1

p_ix_i

!

≤ 1

4(M −m) (f⁰(M)−f⁰(m)).

This was subsequently applied in Information Theory for Shannon’s and Rényi’s entropy.

In this paper we point out some other counterparts of Jensen’s inequality that are similar to (1.3), some of which are better than the above inequalities.

2. SOMENEWCOUNTERPARTS FORJENSEN’S DISCRETE INEQUALITY

The following result holds.

Theorem 2.1. Letf : I ⊆ R→R be a differentiable convex mapping on

◦

I and x_i ∈I^◦ with x₁ ≤x₂ ≤ · · · ≤x_nandp_i ≥0 (i= 1, ..., n)withPn

i=1p_i = 1. Then we have

0 ≤

n

X

i=1

pif(xi)−f

n

X

i=1

pixi

! (2.1)

≤ (x_n−x₁) (f⁰(x_n)−f⁰(x₁)) max

1≤k≤n−1

P_kP¯_k+1

≤ 1

4(xn−x1) (f⁰(xn)−f⁰(x1)), whereP_k :=Pk

i=1p_i andP¯_k+1 := 1−P_k.

Proof. We use the following Grüss type inequality due to J. E. Peˇcari´c (see for example [25]):

(2.2)

1 Q_n

n

X

i=1

qiaibi− 1 Q_n

n

X

i=1

qiai· 1 Q_n

n

X

i=1

qibi

≤ |an−a1| |bn−b1| max

1≤k≤n−1

Q_kQ¯_k+1 Q²_n

,

provided thata, bare two monotonicn−tuples,q is a positive one,Q_n :=Pn

i=1q_i > 0, Q_k :=

Pk

i=1q_i andQ¯_k+1 =Q_n−Q_k+1.

If in (2.2) we chooseq_i =p_i,a_i =x_i,b_i =f⁰(x_i)(anda_i, b_iwill be monotonic nondecreasing),

then we may state that (2.3)

n

X

i=1

pixif⁰(xi)−

n

X

i=1

pixi n

X

i=1

pif⁰(xi)

≤(x_n−x₁) (f⁰(x_n)−f⁰(x₁)) max

1≤k≤n−1

P_kP¯_k+1 .

Now, using (1.2) and (2.3) we obtain the first inequality in (2.1).

For the second inequality, we observe that P_kP¯_k+1 =P_k(1−P_k)≤ 1

4(P_k+ 1−P_k)² = 1 4 for allk ∈ {1, ..., n−1}and then

1≤k≤n−1max

P_kP¯_k+1 ≤ 1 4,

which proves the last part of (2.1).

Remark 2.2. It is obvious that the inequality (2.1) is an improvement of (1.3) if we assume that the order forx_i is as in the statement of Theorem 2.1.

Another result is embodied in the following theorem.

Theorem 2.3. Letf :I ⊆R→Rbe a differentiable convex mapping on

◦

I andm, M ∈I^◦ with m ≤ x_i ≤ M (i= 1, ..., n)andp_i ≥ 0 (i= 1, ..., n)withPn

i=1p_i = 1. IfS is a subset of the set{1, ..., n}minimizing the expression

(2.4)

X

i∈S

pi− 1 2 ,

then we have the inequality 0 ≤

n

X

i=1

p_if(x_i)−f

n

X

i=1

p_ix_i

! (2.5)

≤ Q(M −m) (f⁰(M)−f⁰(m))≤ 1

4(M −m) (f⁰(M)−f⁰(m)), where

Q=X

i∈S

p_i 1−X

i∈S

p_i

! .

Proof. We use the following Grüss type inequality due the Andrica and Badea [2]:

(2.6)

Q_n

n

X

i=1

q_ia_ib_i−

n

X

i=1

q_ia_i·

n

X

i=1

q_ib_i

≤(M₁−m₁) (M₂−m₂)X

i∈S

q_i Q_n−X

i∈S

q_i

!

provided thatm₁ ≤ a_i ≤ M₁,m₂ ≤ b_i ≤ M₂ fori = 1, ..., n, andS is the subset of{1, ..., n}

which minimises the expression

X

i∈S

q_i−1 2Q_n

.

Choosingq_i =p_i,a_i =x_i, b_i =f⁰(x_i), then we may state that 0 ≤

n

X

i=1

pixif⁰(xi)−

n

X

i=1

pixi n

X

i=1

pif⁰(xi) (2.7)

≤ (M −m) (f⁰(M)−f⁰(m))X

i∈S

p_i 1−X

i∈S

p_i

! .

Now, using (1.2) and (2.7), we obtain the first inequality in (2.5). For the last part, we observe that

Q≤ 1 4

X

i∈S

p_i+ 1−X

i∈S

p_i

!2

= 1 4

and the theorem is thus proved.

The following inequality is well known in the literature as the arithmetic mean-geometric mean-harmonic-mean inequality:

(2.8) A_n(p, x)≥G_n(p, x)≥H_n(p, x), where

A_n(p, x) : =

n

X

i=1

p_ix_i - the arithmetic mean,

G_n(p, x) : =

n

Y

i=1

x^p_iⁱ - the geometric mean, H_n(p, x) : = 1

n

P

i=1 pi

xi

- the harmonic mean,

andPn

i=1p_i = 1 p_i ≥0,i= 1, n .

Using the above two theorems, we are able to point out the following reverse of the AGH - inequality.

Proposition 2.4. Letx_i >0 (i= 1, ..., n)andp_i ≥0withPn

i=1p_i = 1.

(i) Ifx₁ ≤x₂ ≤ · · · ≤x_n−1 ≤x_n, then we have 1 ≤ A_n(p, x)

G_n(p, x) (2.9)

≤ exp

"

(xn−x1)²

x₁x_n max

1≤k≤n−1

P_kP¯_k+1

#

≤ exp

"

1

4· (x_n−x₁)² x1xn

# .

(ii) If the set S ⊆ {1, ..., n} minimizes the expression (2.4), and0 < m ≤ xi ≤ M < ∞ (i= 1, ..., n), then

(2.10) 1≤ An(p, x)

G_n(p, x) ≤exp

"

Q· (M−m)² mM

#

≤exp

"

1

4· (M −m)² mM

# .

The proof goes by the inequalities (2.1) and (2.5), choosingf(x) = −lnx. A similar result can be stated forG_nandH_n.

Proposition 2.5. Letp≥1andx_i >0,p_i ≥0 (i= 1, ..., n)withPn

i=1p_i = 1.

(i) Ifx1 ≤x2 ≤ · · · ≤xn−1 ≤xn, then we have 0 ≤

n

X

i=1

p_ix^p_i −

n

X

i=1

p_ix_i

!p

(2.11)

≤ p(x_n−x₁) x^p−1_n −x^p−1₁

1≤k≤n−1max

P_kP¯_k+1

≤ p

4(xn−x1) x^p−1_n −x^p−1₁ .

(ii) If the set S ⊆ {1, ..., n} minimizes the expression (2.4), and0 < m ≤ x_i ≤ M < ∞ (i= 1, ..., n), then

0 ≤

n

X

i=1

p_ix^p_i −

n

X

i=1

p_ix_i

!p

(2.12)

≤ pQ(M −m) M^p−1−m^p−1

≤ 1

4p(M −m) M^p−1−m^p−1 .

Remark 2.6. The above results are improvements of the corresponding inequalities obtained in [5].

Remark 2.7. Similar inequalities can be stated if we choose other convex functions such as:

f(x) = xlnx,x >0orf(x) = exp (x),x∈R. We omit the details.

3. A CONVERSEINEQUALITY FOR CONVEX MAPPINGS DEFINED ONRⁿ

In 1996, Dragomir and Goh [15] proved the following converse of Jensen’s inequality for convex mappings onRⁿ.

Theorem 3.1. Letf :Rⁿ→Rbe a differentiable convex mapping onRⁿand (∇f) (x) :=

∂f(x)

∂x¹ , ...,∂f(x)

∂xⁿ

, the vector of the partial derivatives,x= (x¹, ..., xⁿ)∈Rⁿ. Ifxi ∈R^m (i= 1, ..., m),pi ≥0, i= 1, ..., m,withPm :=Pm

i=1pi >0, then

0 ≤ 1

P_m

m

X

i=1

p_if(x_i)−f 1 P_m

m

X

i=1

p_ix_i

! (3.1)

≤ 1 Pm

m

X

i=1

p_ih∇f(x_i), x_ii −

* 1 Pm

m

X

i=1

p_i∇f(x_i), 1 Pm

m

X

i=1

p_ix_i +

.

The result was applied to different problems in Information Theory by providing different coun- terpart inequalities for Shannon’s entropy, conditional entropy, mutual information, conditional mutual information, etc.

For generalizations of (3.1) in Normed Spaces and other applications in Information Theory, see Mati´c’s Ph.D dissertation [23].

Recently, Dragomir [4] provided an upper bound for Jensen’s difference

(3.2) ∆ (f, p, x) := 1

Pm m

X

i=1

p_if(x_i)−f 1 Pm

m

X

i=1

p_ix_i

! ,

which, even though it is not as sharp as (3.1), provides a simpler way, and for applications, a better way, of estimating the Jensen’s differences ∆. His result is embodied in the following theorem.

Theorem 3.2. Letf :Rⁿ →Rbe a differentiable convex mapping andx_i ∈Rⁿ,i = 1, ..., m.

Suppose that there exists the vectorsφ,Φ∈Rⁿsuch that

(3.3) φ≤x_i ≤Φ (the order is considered on the co-ordinates) andm, M ∈Rⁿare such that

(3.4) m≤ ∇f(x_i)≤M

for alli∈ {1, ..., m}. Then for allp_i ≥0 (i= 1, ..., m)withP_m >0, we have the inequality

(3.5) 0≤ 1

P_m

m

X

i=1

p_if(x_i)−f 1 P_m

m

X

i=1

p_ix_i

!

≤ 1

4kΦ−φk kM −mk, wherek·kis the usual Euclidean norm onRⁿ.

He applied this inequality to obtain different upper bounds for Shannon’s and Rényi’s en- tropies.

In this section, we point out another counterpart for Jensen’s difference, assuming that the

∇−operator is of Hölder’s type, as follows.

Theorem 3.3. Let f : Rⁿ → R be a differentiable convex mapping and xi ∈ Rⁿ, pi ≥ 0 (i= 1, ..., m)withPm >0. Suppose that the∇−operator satisfies a condition ofr−H−Hölder type, i.e.,

(3.6) k∇f(x)− ∇f(y)k ≤Hkx−yk^r, for allx, y ∈Rⁿ, whereH >0,r∈(0,1]andk·kis the Euclidean norm.

Then we have the inequality:

0 ≤ 1

P_m

m

X

i=1

pif(xi)−f 1 P_m

m

X

i=1

pixi

! (3.7)

≤ H P_m²

X

1≤i<j≤m

p_ip_jkx_i−x_jk^r+1.

Proof. We recall Korkine’s identity, 1

P_m

m

X

i=1

p_ihy_i, x_ii−

* 1 P_m

m

X

i=1

p_iy_i, 1 P_m

m

X

i=1

p_ix_i +

= 1 2P_m²

n

X

i,j=1

p_ip_jhy_i−y_j, x_i−x_ji, x, y ∈Rⁿ,

and simply write 1

Pm m

X

i=1

p_ih∇f(x_i), x_ii −

* 1 Pm

m

X

i=1

p_i∇f(x_i), 1 Pm

m

X

i=1

p_ix_i +

= 1 2P_m²

n

X

i,j=1

p_ip_jh∇f(x_i)− ∇f(x_j), x_i−x_ji.

Using (3.1) and the properties of the modulus, we have

0 ≤ 1

P_m

m

X

i=1

p_if(x_i)−f 1 P_m

m

X

i=1

p_ix_i

!

≤ 1 2P_m²

m

X

i,j=1

p_ip_j|h∇f(x_i)− ∇f(x_j), x_i−x_ji|

≤ 1 2P_m²

m

X

i,j=1

p_ip_jk∇f(x_i)− ∇f(x_j)k kx_i −x_jk

≤ H P_m²

m

X

i,j=1

p_ip_jkx_i−x_jk^r+1

and the inequality (3.7) is proved.

Corollary 3.4. With the assumptions of Theorem 3.3 and if∆ = max_1≤i<j≤mkx_i−x_jk, then we have the inequality

(3.8) 0≤ 1

Pm m

X

i=1

p_if(x_i)−f 1 Pm

m

X

i=1

p_ix_i

!

≤ H∆^r+1 2P_m² 1−

m

X

i=1

p²_i

! . Proof. Indeed, as

X

1≤i<j≤m

p_ip_jkx_i−x_jk^r+1 ≤∆^r+1 X

1≤i<j≤m

p_ip_j. However,

X

1≤i<j≤m

pipj = 1 2

m

X

i,j=1

pipj−X

i=j

pipj

!

= 1 2 1−

m

X

i=1

p²_i

! ,

and the inequality (3.8) is proved.

The case of Lipschitzian mappings is embodied in the following corollary.

Corollary 3.5. Let f : Rⁿ → R be a differentiable convex mapping and x_i ∈ Rⁿ, p_i ≥ 0 (i= 1, ..., n) with P_m > 0. Suppose that the ∇−operator is Lipschitzian with the constant L >0, i.e.,

(3.9) k∇f(x)− ∇f(y)k ≤Lkx−yk, for allx, y ∈Rⁿ, wherek·kis the Euclidean norm. Then

0 ≤ 1

Pm m

X

i=1

p_if(x_i)−f 1 Pm

m

X

i=1

p_ix_i

! (3.10)

≤ L



 1 Pm

m

X

i=1

p_ikx_ik² −

1 Pm

m

X

i=1

p_ix_i

2

.

Proof. The argument is obvious by Theorem 3.3, taking into account that forr = 1, X

1≤i<j≤m

p_ip_jkx_i−x_jk² =P_m

m

X

i=1

p_ikx_ik²−

m

X

i=1

p_ix_i

2

,

andk·kis the Euclidean norm.

Moreover, if we assume more about the vectors(x_i)_i=1,n, we can obtain a simpler result that is similar to the one in [4].

Corollary 3.6. Assume thatf is as in Corollary 3.5. If

(3.11) φ≤x_i ≤Φ (on the co-ordinates),φ,Φ∈Rⁿ (i= 1, .., m), then we have the inequality

0 ≤ 1

P_m

m

X

i=1

p_if(x_i)−f 1 P_m

m

X

i=1

p_ix_i

! (3.12)

≤ 1

4 ·L· kΦ−φk².

Proof. It follows by the fact that inRⁿ, we have the following Grüss type inequality (as proved in [4])

(3.13) 1

P_m

m

X

i=1

p_ikx_ik²−

1 P_m

m

X

i=1

p_ix_i

2

≤ 1

4kΦ−φk²,

provided that (3.11) holds.

Remark 3.7. For some Grüss type inequalities in Inner Product Spaces, see [7].

4. SOME RELATEDRESULTS

Start with the following definitions from [3].

Definition 4.1. Let−∞ < a < b < ∞. ThenCM[a, b]denotes the set of all functions with domain[a, b]that are continuous and strictly monotonic there.

Definition 4.2. Let −∞ < a < b < ∞, and let f ∈ CM[a, b]. Then, for each positive integern, eachn−tuplex = (x1, ..., xn),wherea ≤xj ≤ b(j = 1,2, ..., n), and eachn-tuple p = (p1, p2, ..., pn),where pj > 0 (j = 1,2, ..., n) andPn

j=1pj = 1, letMf(x, y)denote the (weighted) mean

f⁻¹ ( _n

X

j=1

p_jf(x_j) )

.

We may state now the following result.

Theorem 4.1. LetSbe the subset of{1, ..., n}which minimizes the expression

P

i∈Spi−1/2 . Iff, g ∈CM[a, b], then

sup

x

{|M_f(x, p)−M_g(x, p)|} ≤Q·

f⁻¹⁰ _∞·

f ◦g⁻¹⁰⁰

_∞· |g(b)−g(a)|², provided that the right-hand side of the inequality is finite, where, as above,

Q= X

i∈S

pi

!

1−X

i∈S

pi

! , andk·k_∞is the usual sup-norm.

Proof. Let, as in [3],h=f ◦g⁻¹,n >1,

x= (x₁, x₂, ..., x_n) andp= (p₁, p₂, ..., p_n)

be as in the Definition 4.2, andy_j = g(x_j) (j = 1,2, ..., n). By the mean-value theorem, for someαin the open interval joiningf(a)tof(b), we have

M_f(x, p)−M_g(x, p) = f⁻¹ ( _n

X

j=1

p_jf(x_j) )

−f⁻¹

"

h ( _n

X

j=1

p_jg(x_j) )#

= f⁻¹0

(α)

" _n X

j=1

p_jf(x_j)−h ( _n

X

j=1

p_jg(x_j) )#

= f⁻¹0

(α)

" _n X

j=1

p_jh(y_j)−h ( _n

X

j=1

p_jy_j )#

= f⁻¹0

(α)

" _n X

j=1

p_j (

h(y_j)−h

n

X

k=1

p_ky_k

!)#

.

Using the mean-value theorem a second time, we conclude that there exists pointsz₁, z₂, ..., z_n in the open interval joiningg(a)tog(b), such that

M_f(x, p)−M_g(x, p) = f⁻¹0

(α)

p₁{(1−p₁)y₁−p₂y₂− · · · −p_ny_n}h⁰(z₁) +p₂{−p₁y₁+ (1−p₂)y₂− · · · −p_ny_n}h⁰(z₂)

+· · ·

+pn{−p1y1−p2y2− · · ·+ (1−pn)yn}h⁰(zn)

= f⁻¹0

(α)

p₁{p₂(y₁−y₂) +· · ·+p_n(y₁−y_n)}h⁰(z₁) +p₂{p₁(y₂−y₁) +· · ·+p_n(y₂ −y_n)}h⁰(z₂)

+· · ·

+p_n{p₁(y_n−y₁) +· · ·+pn−1(y_n−yn−1)}h⁰(z_n)

= f⁻¹0

(α) X

1≤i<j≤n

p_ip_j(y_i−y_j){h⁰(z_i)−h⁰(z_j)}.

Using the mean value theorem a third time, we conclude that there exists pointsω_ij(1≤i < j ≤n) in the open interval joiningg(a)tog(b), such that

f⁻¹0

(α) X

1≤i<j≤n

pipj(yi−yj){h⁰(zi)−h⁰(zj)}

= f⁻¹0

(α) X

1≤i<j≤n

p_ip_j(y_i−y_j) (z_i−z_j)h⁰⁰(ω_ij).

Consequently,

|M_f(x, p)−M_g(x, p)| ≤ f⁻¹⁰

(α)

X

1≤i<j≤n

p_ip_j|y_i−y_j| · |z_i−z_j| · |h⁰⁰(ω_ij)|

≤

f⁻¹0

∞· kh⁰⁰k_∞· X

1≤i<j≤n

p_ip_j|y_i−y_j| · |z_i−z_j|

≤ (by the Cauchy-Buniakowski-Schwartz inequality)

≤

f⁻¹0 _∞·

f◦g⁻¹00 _∞·

s X

1≤i<j≤n

p_ip_j|y_i−y_j|²· s

X

1≤i<j≤n

p_ip_j|z_i−z_j|²

≤ (by the Andrica and Badea result)

≤

f⁻¹⁰ _∞·

f◦g⁻¹⁰⁰ _∞·

v u u t

X

i∈S

p_i

!

1−X

i∈S

p_i

!

|g(b)−g(a)|²

· v u u t

X

i∈S

p_i

!

1−X

i∈S

p_i

!

|g(b)−g(a)|²

= Q

f⁻¹⁰ _∞·

f◦g⁻¹⁰⁰

_∞· |g(b)−g(a)|²,

and the theorem is proved.

Corollary 4.2. Iff, g∈CM[a, b], then

sup

x

{|M_f (x, p)−M_g(x, p)|} ≤Q·

1 f⁰

_∞

·

1 g⁰

f⁰ g⁰

⁰ _∞

· |g(b)−g(a)|², provided that the right hand side of the inequality exists.

Proof. This follows at once from the fact that f⁻¹⁰

= 1

f⁰◦f⁻¹ and

f◦g⁻¹00

= (g⁰ ◦g⁻¹) (f⁰⁰◦g⁻¹)−(f⁰◦g⁻¹) (g⁰⁰◦g⁻¹) (g⁰◦g⁻¹)³ =

1 g⁰

f⁰ g⁰

0

◦g⁻¹.

Remark 4.3. This establishes Theorem 4.3 from [3] and replaces the multiplicative factor ¹₄ byQ. In Corollary 4.2, we also replaced the multiplicative factor ¹₄ byQ.

5. APPLICATIONS ININFORMATIONTHEORY

We give some new applications for Shannon’s entropy H_b(X) :=

r

X

i=1

p_ilog_b 1 p_i,

whereX is a random variable with the probability distribution(p_i)_i=1,r.

Theorem 5.1. LetX be as above and assume thatp₁ ≥p₂ ≥ · · · ≥p_r orp₁ ≤ p₂ ≤ · · · ≤p_r. Then we have the inequality

(5.1) 0≤log_br−H_b(X)≤ (p₁−p_r)² p₁p_r max

1≤k≤r

P_kP¯_k+1 . Proof. We choose in Theorem 2.1,f(x) = −log_bx, x > 0, x_i = _p¹

i (i= 1, ..., r). Then we havex₁ ≤x₂ ≤ · · · ≤x_r and by (2.1) we obtain

0≤log_br−H_b(X)≤ 1

pr

− 1 p1

1

−_p¹

r

+ 1

1 p1

!

1≤k≤rmax

P_kP¯_k+1 ,

which is equivalent to (5.1). The same inequality is obtained ifp₁ ≤p₂ ≤ · · · ≤p_r. Theorem 5.2. LetXbe as above and suppose that

p_M : = max{p_i|i= 1, ..., r}, p_m : = min{p_i|i= 1, ..., r}. IfSis a subset of the set{1, ..., r}minimizing the expression

P

i∈Sp_i−1/2

, then we have the estimation

(5.2) 0≤log_br−H_b(X)≤Q· (p_M −p_m)² lnb·pMpm

. Proof. We shall choose in Theorem 2.3,

f(x) = −log_bx, x > 0, xi = 1

p_i i= 1, r . Thenm = _p¹

M,M = _p¹

m,f⁰(x) =−_xln¹_b and the inequality (2.3) becomes:

0 ≤ log_br−

r

X

i=1

pilog_b 1 p_i

≤ Q 1 lnb

1 pm

− 1 pM

− 1

1 pm

+ 1

1 pM

!

= Q· 1

lnb ·(p_M −p_m)² p_Mp_m ,

hence the estimation (5.2) is proved.

Consider the Shannon entropy

(5.3) H(X) := He(X) =

r

X

i=1

piln 1 p_i and Rényi’s entropy of orderα(α∈(0,∞)\ {1})

(5.4) H_[α](X) := 1

1−αln

r

X

i=1

p^α_i

! .

Using the classical Jensen’s discrete inequality for convex mappings, i.e.,

(5.5) f

r

X

i=1

p_ix_i

!

≤

r

X

i=1

p_if(x_i),

where f : I ⊆ R→R is a convex mapping on I, xi ∈ I (i= 1, ..., r) and (pi)_i=1,r is a probability distribution, for the convex mappingf(x) =−lnx,we have

(5.6) ln

r

X

i=1

p_ix_i

!

≥

r

X

i=1

p_ilnx_i.

Choosex_i =p^α−1_i (i= 1, ..., r)in (5.6) to obtain ln

r

X

i=1

p^α_i

!

≥(α−1)

r

X

i=1

p_ilnp_i,

which is equivalent to

(1−α)

H_[α](X)−H(X)

≥0.

Now, ifα ∈ (0,1),then H_[α](X) ≤ H(X), and ifα > 1then H_[α](X) ≥ H(X). Equal- ity holds iff (p_i)_i=1,r is a uniform distribution and this fact follows by the strict convexity of −ln (·). This inequality also follows as a special case of the following well known fact:

H_[α](X)is a nondecreasing function of α. See for example [26] or [22].

Theorem 5.3. Under the above assumptions, given that p_m = min_i=1,rp_i, p_M = max_i=1,rp_i, then we have the inequality

(5.7) 0≤(1−α)

H_[α](X)−H(X)

≤Q· p^α−1_M −p^α−1_m 2

p^α−1_M p^α−1_m , for allα∈(0,1)∪(1,∞).

Proof. Ifα∈(0,1), then

x_i :=p^α−1_i ∈

p^α−1_M , p^α−1_m and ifα∈(1,∞), then

x_i =p^α−1_i ∈

p^α−1_m , p^α−1_M

, fori∈ {1, ..., n}.

Applying Theorem 2.3 forx_i :=p^α−1_i andf(x) =−lnx, and taking into account thatf⁰(x) =

−¹_x, we obtain (1−α)

H_[α](X)−H(X)

≤











Q p^α−1_m −p^α−1_M

− ¹

p^α−1m + ¹

p^α−1_M

if α ∈(0,1), Q p^α−1_M −p^α−1_m

− ¹

p^α−1_M + ¹

p^α−1m

if α ∈(1,∞)

=











Q· (^pα−1m −p^α−1_M )²

p^α−1m p^α−1_M if α∈(0,1), Q· (^pα−1M −p^α−1m )²

p^α−1_M p^α−1m if α∈(1,∞)

=Q· p^α−1_M −p^α−1_m 2

p^α−1_M p^α−1_m

for allα∈(0,1)∪(1,∞)and the theorem is proved.

Using a similar argument to the one in Theorem 5.3, we can state the following direct appli- cation of Theorem 2.3.

Theorem 5.4. Let(p_i)_i=1,r be as in Theorem 5.3. Then we have the inequality (5.8) 0≤(1−α)H_[α](X)−lnr−αlnG_r(p)≤Q· p^α−1_M −p^α−1_m 2

P_M^α−1p^α−1_m , for allα∈(0,1)∪(1,∞).

Remark 5.5. The above results improve the corresponding results from [5] and [4] with the constantQwhich is less than ¹₄.

Acknowledgement 1. The authors would like to thank the anonymous referee for valuable comments and for the references [26] and [22].

REFERENCES

[1] A. RÉNYI, On measures of entropy and information, Proc. Fourth Berkley Symp. Math. Statist.

Prob., 1 (1961), 547–561, Univ. of California Press, Berkley.

[2] D. ANDRICAANDC. BADEA, Grüss’ inequality for positive linear functionals, Periodica Math.

Hung., 19(2) (1988), 155–167.

[3] G.T. CARGOAND O. SHISHA, A metric space connected with general means, J. Approx. Th., 2 (1969), 207–222.

[4] S.S. DRAGOMIR, A converse of the Jensen inequality for convex mappings of several variables and applications. (Electronic Preprint:

http://matilda.vu.edu.au/~rgmia/InfTheory/ConverseJensen.dvi) [5] S.S. DRAGOMIR, A converse result for Jensen’s discrete inequality via Grüss’ inequality and

applications in information theory, Analele Univ. Oradea, Fasc. Math., 7 (1999-2000), 178–189.

[6] S.S. DRAGOMIR, A further improvement of Jensen’s inequality, Tamkang J. Math., 25(1) (1994), 29–36.

[7] S.S. DRAGOMIR, A generalisation of Grüss’s inequality in inner product spaces and applications, J. Math. Anal. Appl., 237 (1999), 74–82.

[8] S.S. DRAGOMIR, A new improvement of Jensen’s inequality, Indian J. Pure Appl. Math., 26(10) (1995), 959–968.

[9] S.S. DRAGOMIR, An improvement of Jensen’s inequality, Bull. Math. Soc. Sci. Math. Romania., 34 (1990), 291–296.

[10] S.S. DRAGOMIR, On some refinements of Jensen’s inequality and applications, Utilitas Math., 43 (1993), 235–243.

[11] S.S. DRAGOMIR, Some refinements of Jensen’s inequality, J. Math. Anal. Appl., 168 (1992), 518–

522.

[12] S.S. DRAGOMIR, Some refinements of Ky Fan’s inequality, J. Math. Anal. Appl., 163 (1992), 317–321.

[13] S.S. DRAGOMIR, Two mappings in connection with Jensen’s inequality, Extracta Math., 8(2-3) (1993), 102–105.

[14] S.S. DRAGOMIRANDS. FITZPARTICK,s- Orlicz convex functions in linear spaces and Jensen’s discrete inequality, J. Math. Anal. Appl., 210 (1997), 419–439.

[15] S.S. DRAGOMIRANDC.J. GOH, A counterpart of Jensen’s discrete inequality for differentiable convex mappings and applications in Information Theory, Math. Comput. Modelling, 24(2) (1996), 1–11.

[16] S.S. DRAGOMIRANDC.J. GOH, Some bounds on entropy measures in information theory, Appl.

Math. Lett., 10(3) (1997), 23–28.

[17] S.S. DRAGOMIRANDC.J. GOH, Some counterpart inequalities for a functional associated with Jensen’s inequality, J. Inequal. Appl., 1 (1997), 311–325.

[18] S.S. DRAGOMIRANDN.M. IONESCU, Some Converse of Jensen’s inequality and applications, Anal. Num. Theor. Approx. (Cluj-Napoca), 23 (1994), 71–78.

[19] S.S. DRAGOMIR, C.E.M. PEARCEANDJ. E. PE ˇCARI ´C, New inequalities for logarithmic map and their applications for entropy and mutual information, Kyungpook Math. J., in press.

[20] S.S. DRAGOMIR, C.E.M. PEARCEANDJ. E. PE ˇCARI ´C, On Jensen’s and related inequalities for isotonic sublinear functionals, Acta Sci. Math., (Szeged), 61 (1995), 373–382.

[21] S.S. DRAGOMIR, J. E. PE ˇCARI ´CANDL.E. PERSSON, Properties of some functionals related to Jensen’s inequality, Acta Math. Hungarica, 20(1-2) (1998), 129–143.

[22] T. KOSKIANDL.E. PERSSON, Some properties of generalized entropy with applications to com- pression of data, J. of Int. Sciences, 62 (1992), 103–132.

[23] M. MATI ´C, Jensen’s Inequality and Applications in Information Theory, (in Croatian), Ph.D. Dis- sertation, Split, Croatia, 1998.

[24] R.J. McELIECE, The Theory of Information and Coding, Addison Wesley Publishing Company, Reading, 1977.

[25] J. E. PE ˇCARI ´C, F. PROSCHANANDY.L. TONG, Convex Functions, Partial Orderings and Sta- tistical Applications, Academic Press, 1992.

[26] J. PEETREANDL.E. PERSSON, A general Beckenbach’s inequality with applications, In: Func- tion Spaces, Differential Operators and Nonlinear Analysis, Pitman Research Notes in Math., 211 (1989), 125–139.