Fractional Laplacian system involving doubly critical nonlinearities in R N

Fractional Laplacian system involving doubly critical nonlinearities in R ^N

Li Wang

, Binlin Zhang

and Haijin Zhang

1College of Science, East China Jiaotong University, Nanchang, 330013, P.R. China

2Department of Mathematics, Heilongjiang Institute of Technology, Harbin, 150050, P.R. China

Received 15 March 2017, appeared 20 July 2017 Communicated by Patrizia Pucci

Abstract. In this article, we are interested in a fractional Laplacian system inR^N, which involves critical Sobolev-type nonlinearities and critical Hardy–Sobolev-type nonlinear- ities. By using variational methods, we investigate the extremals of the corresponding best fractional Hardy–Sobolev constant and establish the existence of solutions. To our best knowledge, our main results are new in the study of the fractional Laplacian system.

Keywords:fractional Laplacian system, doubly critical nonlinearities, variational meth- ods.

2010 Mathematics Subject Classification: 35A15, 35R11, 35J50.

1 Introduction and main result

In this article, we are concerned with the existence of solutions for the following fractional Laplacian system inR^N:











(−_∆)^su−µ u

|x|^2s = (I_a∗ |u|^2]h,a)|u|^2]h,a−2u+|u|^2∗s,b−2u

|x|^b + ^ηα α+β

|u|^α−2u|v|^β

|x|^{b ,} (−_∆)^sv−µ v

|x|^2s = (I_a∗ |v|^2]h,a)|v|^2]h,a−2v+ |v|^2∗s,b−2v

|x|^b + ^{η β} α+β

|u|^α|v|^β−2v

|x|^{b ,}

(1.1)

where I_a(x) = ^Γ(N−22)

2^aπ^N/2Γ(^a₂)|x|^N−a is a Riesz potential, for simplicity, we setI_a(x) = _| ¹

x|^N−a, µ,η ∈ R⁺₀, 0 < a,b < 2s < N, α > 1, β > 1, α+β = 2^∗_s,b = ²⁽_N^N₋⁻_2s^b) and 2^]_h,a = _N^N₋⁺_2s^a are fractional critical exponents for Sobolev-type embeddings. The fractional Laplace operator (−_∆)^s is defined by

(−_∆)^s =F⁻¹(|ξ|^2sFu(ξ)) for allu∈ C₀^∞(_R^N), ξ ∈_R^N,

BCorresponding author. Emails: wangli.423@163.com (L. Wang), zhangbinlin2012@163.com (B. Zhang), 1015330036@qq.com (H. Zhang)

where Fu denotes the Fourier transform of u. Weak solutions of (1.1) will be found in the spaceH = H^˙s(_R^N)×H^˙s(_R^N), where ˙H^s(_R^N)is defined as the completion ofC₀^∞(_R^N)under the norm

kuk²_˙

H^s(_R^N) =

Z

R^N|ξ|^2s|uˆ(ξ)|²dξ. (1.2) Therefore, fors >0, we have

k(−_∆)^s2_uk²_L2(_R^N) =

Z

R^N|ξ|^2s|uˆ(ξ)|²dξ. (1.3) By a (weak) solution(u,v)of problem (1.1), we mean that(u,v)∈ Hsatisfies

Z

R^N

(−_∆)^s2u(−_∆)^2sφ₁+ (−_∆)^2sv(−_∆)^2sφ₂−µ

uφ₁+vφ₂

|x|^2s

dx

=

Z Z

R^2N

|u(y)|^2]h,a|u(x)|^2]h,a−2u(x)φ₁(x) +|v(y)|^2]h,a|v(x)|^2]h,a−2v(x)φ₂(x)

|x−y|^{N−a dxdy}

+

Z

R^N

|u|^2∗s,b−2uφ₁+|v|^2∗s,b−2vφ2+η(α|u|^α−2uφ₁|v|^β+β|u|^α|v|^β−2vφ2)

|x|^{b dx}

for allφ₁,φ2∈ H^˙s(_R^N).

In recent years, much attention has been paid to fractional and non-local operators. More precisely, this type of operators arises in a quite natural way in many different applications, such as, finance, physics, fluid dynamics, population dynamics, image processing, minimal surfaces and game theory, see [4,11] and the references therein. In particular, there are some remarkable mathematical models involving the fractional Laplacian, such as, the fractional Schr ¨odinger equation (see [18,31]), the fractional Kirchhoff equation (see [1,13,24,25]), the fractional porous medium equation (see [5]) and so on.

Problems with one nonlinearity or two nonlinearities involving the Laplacian and the frac- tional Laplacian have been studied by many authors. For example, we refer, in bounded domains to [14,20,21,27,28,30], while in the entire space to [12,16,22]. In [8], Filippucci, Pucci and Robert proved that there exists a positive solution for a p-Laplacian problem with critical Sobolev and Hardy–Sobolev terms. In [15], Fiscella, Pucci and Saldi dealt with the existence of nontrivial nonnegative solutions of Schr ¨odinger–Hardy systems driven by two possibly different fractional ℘-Laplacian operators, also including critical nonlinear terms, where the nonlinearities do not necessarily satisfy the Ambrosetti–Rabinowitz condition. It is natural to consider the concentration–compactness principle for critical problems. However, due to the nonlocal feature of the fractional Laplacian, it is difficult to use the concentration–

compactness principle directly, since one needs to estimate commutators of the fractional Laplacian and smooth functions. A natural strategy, which is named by the s-harmonic ex- tension, is to transform the nonlocal problem into a local problem, as Caffarelli and Silvestre performed in [3]. Since that, many interesting results in the classical elliptic problems have been extended to the setting of the fractional Laplacian. For example, Ghoussoub and Shake- rian in [16] combined thes-harmonic extension with the concentration-compactness principle to investigate the existence of solutions for a doubly critical problem involving the fractional Laplacian. It is worthy pointing out that Yang and Wu in [33] showed the existence of so- lutions for problem (1.1) with η = 0, by using the elementary approach without the use of the concentration-compactness principle or the extension argument of Caffarelli and Silvestre in [3].

In the doubly critical case, two critical nonlinearities interact to each other. There is an asymptotic competition between the energy carried by the two critical nonlinearities. Obvi- ously, the combination of the two critical exponents induces more difficulties. When one crit- ical exponent is only involved, there are solutions to the corresponding equations: in general, these solutions are radially symmetric with respect to the origin of the domain and are ex- plicit, see for instance [23] for the details. However, very few information have been known in our setting, especially for system, here we just refer the reader to an interesting literature [12].

In this paper, we are interested in the existence of solutions for system (1.1) involving doubly critical exponents, by using a refinement of the Sobolev inequality which is related to the Morrey space. A measurable functionu:R^N →_Rbelongs to the Morrey spaceL^p,γ(_R^N) with p ∈[1,+_∞)andγ∈ (0,N], if and only if

kuk^p_Lp,γ(RN)= sup

R>0,x∈_R^N

R^γ−N Z

B_R(x)

|u(y)|^pdy<_∞. (1.4) By the H ¨older inequality, we can verify that L^2∗s(_R^N),→ L^p,N−22sp(_R^N)_{for 1}≤ p<₂^∗_s = _N^2N_−2s, and for 1< q< p<2^∗_s we have

L^p,N−22sp(_R^N),→ L^q,N−22sq(_R^N).

Moreover, here holdsL^p,γ(_R^N),→ L^1,γp(_R^N)provided thatp ∈(1,+_∞)andγ∈(0,N). The following refinement of Hardy–Sobolev inequalities were proved in [19] and [32].

Proposition 1.1. ([32, Theorem 1.1]). For any0 <b< 2s < N,there exists C >₀such that forθ and r satisfyingmax{^N_N⁻₋^2s_b,^2s_N⁻₋^b_b} ≤θ<1≤r <2^∗_s,b,there holds

Z

R^N

|u|^2∗s,b

|x|^{b dx}

!_2∗¹

s,b ≤ Ckuk^θ_H˙s(RN)kuk^1−θ

L^r,r(N−2^2s)(_R^N) (1.5) for any u∈ H^˙s(_R^N).

In the present paper, we work in the product space H= H^˙s(_R^N)×H^˙s(_R^N)be the Carte- sian product of two Hilbert spaces, which is a reflexive Banach space endowed with the norm

k(u,v)k²=k(u,v)k²_H =kuk²_H˙s(RN)+kvk²_H˙s(RN), where

kuk²_˙

H^s(_R^N) =

Z

R^N

|(−_∆)^2su|²−µ|u|²

|x|^2s

dx.

Solutions of (1.1) are equivalent to a nonzero critical points of the functional I(u,v) = ¹

2k(u,v)k²− ¹ 2·2^]_h,a

Z Z

R^2N

|u(x)|^2]h,a|u(y)|^2]h,a+|v(x)|^2]h,a|v(y)|^2]h,a

|x−y|^{N−a dxdy}

− ¹ 2^∗_s,b

Z

R^N

|u|^2∗s,b +|v|^2∗s,b+η|u|^α|v|^β

|x|^{b dx,}

which is defined on H, and I ∈C¹(H,R). We say a pair of functions(u,v)∈ His called to be a solution of (1.1) if

u6=_0, v6=_0, hI⁰(u,v)_,(φ₁,φ₂)i=_0, ∀(φ₁,φ₂)∈ H. (1.6)

If (u,v) = (u, 0) or (u,v) = (0,v), we say that they are the semi-nontrivial solution. In this case, system can be seen as a singular equation, that isη=0, see [33] for the details.

The main result of this paper can be concluded in the following theorem.

Theorem 1.2. If 0≤µ<µ∗ =₄^sΓ2(N+42s)

Γ²(^N−₄^2s), then problem (1.1) possesses at least one nontrivial solution in H.

Remark 1.3. To the best of our knowledge, Theorem1.2 is new in the study of the fractional Laplacian system involving doubly critical nonlinearities in the whole space. We mainly follow the idea of [33] to prove our main result.

This paper is organized as follows. In Section 2, some preliminary results are presented.

In Section 3, the extremals of the corresponding best fractional Hardy–Sobolev constant are achieved. In Section 4, we give the proof of Theorem1.2.

Throughout this paper, we will use the following notations: tz:= t(u,v) = (tu,tv)for all (u,v)∈ Handt∈_R;(u,v)is said to be nonnegative inR^N ifu≥0 andv≥0 inR^N;(u,v)is said to be positive inR^N ifu>0 andv>0 inR^N;B_r(0) ={x∈ _R^N : |x|<r}is a ball inR^N of radiusr > 0 at the origin;o(1)is a generic infinitesimal value. We always denote positive constants asCfor convenience.

2 Preliminaries

In this section, we recall the fractional Sobolev inequality. ForN >2s, the fractional Sobolev embedding ˙H^s(_R^N) ,→ L^N2N−2s(_R^N)was considered in [6,7]. The continuity of this inclusion corresponds to the inequality

kuk²₂∗

s(_R^N) ≤S_µkuk²_˙

H^s(_R^N). (2.1)

The best constantS_µ in (2.1) was computed (see Theorem 1.1 in [7]). Using the moving plane method for integral equations, Chen, Li and Ou in [6] classified the solutions of an integral equation, which is related to the problem

(−_∆)^s_u=|u|^2∗s−2u inR^N. (2.2) The positive regular solutions of (2.2), which verify the equality in (2.1), are precisely given by

U(x) = ¹

(λ²+|x−x₀|²)^{N−22s (2.3)} forλ>0 andx₀∈_R^N.

On the other hand, the Hardy–Littlewood–Sobolev inequality yields Z Z

R^2N

|u(x)|^2]h,a|u(y)|^2]h,a

|x−y|^{N−a dxdy}

! ¹

2]

h,a ≤ kuk²2N N−2s

≤ Ck(−_∆)^s2uk²₂ , (2.4) and the equality in (2.4) holds if and only if u is given by (2.3). Thus, the exponent 2^]_h,a is critical in the sense that it is the limit exponent for the Sobolev-type inequality (2.4). Taking into account Proposition1.1and (2.4), we obtain the following inequality:

Z Z

R^2N

|u(x)|^2]h,a|u(y)|^2]h,a

|x−y|^{N−a dxdy}

! ¹

2]

h,a ≤ Ckuk^2θ_H˙skuk²_L⁽_2,N¹⁻₋^θ_2s⁾ (2.5) foru∈ H^˙s(_R^N)_.

3 Minimizers of S

In this section, we show that the best constantS_s,bin our context can be achieved. Moreover, we investigate the intrinsic relation between S_s,b and the best fractional Hardy–Sobolev constant with single equation.

Forµ>0, we define

µ∗=inf





 R

R^N|(−_∆)^s2u|²dx R

R^N |u|²

|x|^2sdx , u∈ H^˙s(_R^N)\ {0}







. (3.1)

Here we remark thatµ∗ in (3.1) was showed in [34] that µ∗ =4^sΓ²(^N+₄^2s)

Γ²(^N−₄^2s)^.

Evidently, from (3.1) we have the fractional Hardy–Sobolev inequality Z

R^N

|u|²

|x|^2sdx≤ µ⁻_∗¹ Z

R^N|(−_∆)^2su|²dx. (3.2) If 0≤µ<µ∗, by the fractional Sobolev inequality

Z

R^N|u|^2∗sdx ₂²∗

s ≤S⁻¹ Z

R^N|(−_∆)^s2u|²dx (3.3) and (3.2), we have

S

1− ^µ µ∗

Z

R^N|u|^2∗sdx ₂²_∗

s ≤

Z

R^N

|(−_∆)^2su|²−µ|u|²

|x|^2s

dx. (3.4)

Then, for 0≤µ< µ∗, we define the functional

I_s,a(u,v) = R

R^N

|(−_∆)^2su|²−µ^|u|

2

|x|^2s +|(−_∆)^2sv|²−µ^|v|

2

|x|^2s

dx

R R

R^2N |u(x)|²

] h,a|u(y)|²

]

h,a+|v(x)|²

] h,a|v(y)|²

] h,a

|x−y|^N−a dxdy ¹

2] h,a

(3.5)

and

I_s,b(u,v) = R

R^N

|(−_∆)^2su|²−µ^|u|

2

|x|^2s +|(−_∆)^2sv|²−µ^|v|

2

|x|^2s

dx

R

R^N |u(x)|^2∗s,b+|v(x)|^2∗s,b+η|u(x)|^α|v(x)|^β

|x|^b dx

₂²_∗

s,b

. (3.6)

Consider the minimization problem S_s,a =infn

I_s,a(u,v):u,v∈ H^˙s(_R^N)\ {0}^o, (3.7) and

S_s,b=infn

I_s,b(u,v):u,v∈ H^˙s(_R^N)\ {0}^o. (3.8) The following result shows that for 0 ≤µ<µ∗,S_s,a,S_s,b are achieved.

Lemma 3.1. If0≤µ<µ∗,then S_s,a and S_s,bare achieved respectively by a pair of radially symmetric and nonnegative functions.

Proof. Here we only give the proof of process forS_s,a being achieved. With minor changes, we can also get thatS_s,bis achieved by a pair of radially symmetric nonnegative functions.

Let{(un,vn)}_n be a minimizing sequence ofSs,a, that is Z

R^N

|(−_∆)^s2u_n|²−µ|u_n|²

|x|^2s +|(−_∆)^s2v_n|²−µ|v_n|²

|x|^2s

dx→S_s,a asn→_∞and

Z Z

R^2N

|un(x)|^2]h,a|un(y)|^2]h,a+|vn(x)|^2]h,a|vn(y)|^2]h,a

|x−y|^{N−a dxdy}=1.

Inequality in (2.5) enables us to findC>0 such that ku_nk_L2,N−2s(_R^N)≥ C, and the Sobolev embedding ˙H^s(_R^N),→ L^2,N−2s(_R^N)gives

ku_nk²_L2,N−2s(_R^N)≤ C.

So we may findλn >0 andxn∈_R^N such that λ⁻_n^2s

Z

Bλn(xn)

|un|²dy≥ kunk²_L2,N−2s(_R^N)− ^C

2n ≥C₁>0.

Let ˜u_n(x) =λ

N−2s

n2 u_n(λ_nx). Then λ⁻_n^2s

Z

B1(_λn^xn)

|u˜n|²dy≥C₁>0.

Similarly, we can get that

λ⁻_n^2s Z

B₁(^xn

λn)

|v˜_n|²dy≥C₁ >_0, where ˜vn(x) =λ

N−2s

n2 vn(λnx).

By simple computation, we can getI(u_n,v_n) = I(u˜_n(x), ˜v_n(x)), and then{(u˜_n(x), ˜v_n(x))}_n is also a minimizing sequence of S_s,a. We can also show that {(u˜_n(x)_{, ˜}v_n(x))}_{n is bounded} inH. Hence, we may assume

(u˜_n(x), ˜v_n(x))*(u˜(x), ˜v(x)) weakly in ˙H^s(_R^N)×H^˙s(_R^N), (u˜n(x), ˜vn(x))*(u˜(x), ˜v(x)) weakly in

L_loc^p (_R^N)² for all 1≤ p<2^∗_s, (u˜_n(x), ˜v_n(x))→(u˜(x), ˜v(x)) a.e. inR^N× _R^N.

We claim that {^xn

λn}_n is uniformly bounded inn. Indeed, for any 0< κ < 2s, we observe, by the H ¨older inequality, that

0<C₁≤

Z

B1(^xn_λ

n)

|u˜_n|²dy=

Z

B1(^xn_λ

n)

|y|^2κ/2∗s,κ |u˜_n|²

|y|^{2κ/2∗s,κdy}

≤

Z

B₁(^xn

λn)

|y|^{κ(2sN−−κ2s)}dy

!^2s_N⁻₋^κ

κ Z

B₁(^xn

λn)

|u˜_n|^2∗s,κ

|y|^{κ dy}

! ²

2∗ s,κ

.

By the rearrangement inequality, see [17, Theorem 3.4], we have Z

B₁(^xn

λn)

|y|^{κ(2sN−−2sκ)}dy≤

Z

B₁(0)

|y|^{κ(2sN−−κ2s)}dy≤C.

Therefore,

Z

B1(^xn

λn)

|u˜_n(y)|^2∗s,κ

|y|^{κ dy}≥C>0. (3.9)

Now, suppose on the contrary, that |_λ^xn

n| → _∞ as n → ∞. Then, for anyy ∈ B₁(_λ^xn

n), we have

|y| ≥ |^xn

λn| −1 fornlarge. Thus by the H ¨older inequality, it follows that Z

B₁(^xn

λn)

|u˜_n(y)|^2∗s,κ

|y|^{κ dy}≤ ¹ (|^xn

λn| −₁)^κ

Z

B₁(^xn

λn)

|u˜_n(y)|^2∗s,κdy

≤ ¹

(|^xn

λn| −1)^κ

Z

B1(_λ^xn

n)

|u˜_n(y)|^2∗sdy

!^N_N^−κ

≤ ¹

(|^xn

λn| −1)^κku˜_nk^N_˙^N−κ

H^s(_R^N)

≤ ^C

(|^xn

λn| −1)^κ →0 asn→_∞, which contradicts (3.9). Hence,{^xn

λn}_nis uniformly bounded, and there exists R>0 such that Z

BR(0)

|u˜_n(y)|²dy≥

Z

B₁(^xn

λn)

|u˜_n(y)|²dy≥C₁>0.

The compact Sobolev embedding ˙H^s(_R^N),→ L²_loc(_R^N)implies that there exists ˜usatisfing Z

BR(0)

|u˜(y)|²dy≥C₁>0,

which means ˜u 6≡ 0. Similarly we can get ˜v 6≡ 0. By a Br´ezis–Lieb-type lemma, see [19, Lemma 2.4], we obtain

Z

R^N(I_a∗ |u˜_n−u˜|^2]h,a)|u˜_n−u˜|^2]h,adx+

Z

R^N(I_a∗ |u˜|^2]h,a)|u˜|^2]h,adx

=

Z

R^N(I_a∗ |u˜_n|^2]h,a)|u˜_n|^2]h,adx+o(1), and

Z

R^N(I_a∗ |v˜_n−v˜|^2]h,a)|v˜_n−v˜|^2]h,adx+

Z

R^N(I_a∗ |v˜|^2]h,a)|v˜|^2]h,adx

=

Z

R^N(I_a∗ |v˜n|^2]h,a)|v˜n|^2]h,adx+o(1).

Therefore, S_s,a =

Z

R^N

|(−_∆)^s2u˜_n|²−µ|u˜_n|²

|x|^2s +|(−_∆)^s2v˜_n|²−µ|v˜_n|²

|x|^2s

dx+o(₁)

=

Z

R^N

|(−_∆)^s2(u˜_n−u˜)|²−µ|u˜_n−u˜|²

|x|^2s +|(−_∆)^s2|v˜_n−v˜|²−µ|v˜_n−v˜|²

|x|^2s

dx +

Z

R^N

|(−_∆)^2su˜|²−µ|u˜|²

|x|^2s+|(−_∆)^2s|v˜|²−µ|v˜|²

|x|^2s

dx+o(1)

≥S_s,a Z

R^N

I_a∗ |u˜_n−u˜|^2]h,a|u˜_n−u˜|^2]h,adx+

Z

R^N

I_a∗ |v˜_n−v˜|^2]h,a|v˜_n−v˜|^{2]h,a 1}

2] h,a

+S_{s,a Z}

R^N

I_a∗ |u˜|^2]h,a|u˜|^2]h,adx+

Z

R^N

I_a∗ |v˜|^2]h,a|v˜|^2]h,adx ¹

2]

h,a +o(1)

≥S_{s,a Z}

R^N

hI_a∗ |u˜_n−u˜|^2]h,a|u˜_n−u˜|^2]h,a+I_a∗ |v˜_n−v˜|^2]h,a|v˜_n−v˜|^2]h,aidx

+

Z

R^N

hI_a∗ |u˜|^2]h,a|u˜|^2]h,a+I_a∗ |v˜|^2]h,a|v˜|^2]h,aidx ¹

2]

h,a +o(1)

=S_s,a.

Since ˜u, ˜v6≡0, we obtain S_s,a =

Z

R^N

|(−_∆)^s2u˜|²−µ|u˜|²

|x|^2s +|(−_∆)^s2v˜|²−µ|v˜|²

|x|^2s

dx, and

Z

R^N

h(I_a∗ |u˜|^2]h,a)|u˜|^2]h,a+ (I_a∗ |v˜|^2]h,a)|v˜|^2]h,aidx=1.

Hence,Ss,a is achieved.

Let(u, ˜˜ v)be a minimizer. By inequality (A.11) in [26], we get Z

R^N

h|(−_∆)^s2|u˜||²+|(−_∆)^s2|v˜||²ⁱdx≤

Z

R^N

h|(−_∆)^2su˜|²+|(−_∆)^s2v˜|²ⁱdx,

which implies that(|u˜|,|v˜|)is also a minimizer ofS_s,a and hence ˜u ≥ 0, ˜v ≥ 0. All argument of rearrangement (see [11,26]) shows that (u, ˜˜ v)is radially symmetric. The proof is therefore complete.

For anyα,β > 1 withα+β =2^∗_s,b, and 0< µ < µ∗, we define the following best Hardy–

Sobolev-type constant:

Λs,b := inf

u∈H^˙s(_R^N)\{0}

R

R^N

|(−_∆)^2su|²−µ^|u|

2

|x|^2s

dx

R

R^N |u|^2∗s,b

|x|^b dx _2∗²

s,b

. (3.10)

We may prove, as in [32] with minor changes, that Λs,b is achieved by a radially symmetric nonnegative function. From this and the definition ofS_s,b, we can get the following relation betweenΛs,bandS_s,b.

Theorem 3.2. S_s,b= f(τ_min)_Λs,b.Here

f(τ):= ¹+τ² (1+ητ^β+τ^α+β)^α+2β

, τ≥0, (3.11)

f(τ_min):=min

τ≥0 f(τ)>0, (3.12)

whereτ_min ≥0is a minimal point of f(τ)and therefore a root of the equation 2^∗_s,bτ²

∗

s,b−2+η βτ^β−2−ηατ^β−2^∗_s,b=0 , τ≥0. (3.13) Proof. We mimic the proof of Theorem 1.1 in [10]. By the definition of f(τ)defined in (3.11), it follows that

lim

τ→0⁺ f(τ) = lim

τ→+_∞f(τ) =1.

Thus min_τ≥0 f(τ) must be achieved at τ_min ≥ 0. Furthermore, direct calculation shows that there exists a positive constant Csuch that

0<C≤ f(τ_min):=min

τ>0 f(τ)≤1, 0< τ_min <_∞.

From the fact that f⁰(τ_min) =0, we deduce thatτ_min is a root of the following equation 2^∗_s,bτ²

∗

s,b−2+η βτ^β−2−ηατ^β−2^∗_s,b=0, τ≥0.

Suppose that {wn}_n ⊂ H^˙s(_R^N)is a minimizing sequence forΛs,b. Letτ₁,τ₂ > 0 to be chosen later. Takingu_n =τ₁w_nandv_n=τ₂w_nin (3.8), we have

τ₁²+τ₂²

τ²

∗ s,b

1 +τ²

∗ s,b

2 +ητ₁^ατ₂^β ₂∗²

s,b

· kwnk²_˙

H^s

R

R^N

|wn|^2∗s,b

|x|^b dx ²

2∗ s,b

≥ S_s,b. (3.14)

Note that

f τ₂

τ₁

= ^τ

12+τ₂²

τ₁^α+β+ητ₁^α·τ₂^β+τ²

∗ s,b

2

₂∗² s,b

.

Chooseτ₁ andτ₂in (3.14) such that ^τ_τ²

1 = τ_min. Passing to the limit asn→_{∞we have}

f(τ_min)_Λs,b≥S_s,b. (3.15)

On the other hand, let {(u_n,v_n)}_n be a minimizing sequence of S_s,b and define z_n = τ_nv_n, where

τ_n^α+β = R

R^N |un|^α+β

|x|^b dx R

R^N |vn|^α+β

|x|^b dx . Then

Z

R^N

|z_n|^α+β

|x|^{b dx}=

Z

R^N

|u_n|^α+β

|x|^{b dx. (3.16)}

From the Young inequality it follows that Z

R^N

|un|^α· |zn|^β

|x|^{b dx}≤ ^α α+β

Z

R^N

|un|^α+β

|x|^{b dx}+ ^β α+β

Z

R^N

|zn|^α+β

|x|^{b dx.}

Thus by (3.16) we have Z

R^N

|un|^α· |zn|^β

|x|^{b dx} ≤

Z

R^N

|un|^α+β

|x|^{b dx}=

Z

R^N

|zn|^α+β

|x|^{b dx. (3.17)}

Consequently, ku_nk²_˙

H^s+kv_nk²_˙

H^s

R

R^N |un|^2∗s,b+|vn|^2∗s,b+η|un|^α|vn|^β

|x|^b dx

₂∗² s,b

= ku_nk²_˙

H^s +kv_nk²_˙

H^s

1+ητ_n^−β+τ_n^−(α+β) R

R^N |un|^2∗s,b

|x|^b dx ₂∗²

s,b

= ku_nk²_˙

H^s

1+ητ_n^−β+τ_n^−(α+β) R

R^N |un|^2∗s,b

|x|^b dx _2∗²

s,b

+ ^τ

−2 n kz_nk²_˙

H^s

1+ητ_n^−β+τ_n^−(α+β) R

R^N |zn|^2∗s,b

|x|^b dx _2∗²

s,b

≥ f(τ_n⁻¹)_Λs,b≥ f(τ_min)_Λs,b. Asn→_{∞, we have}

S_s,b≥ f(τ_min)_{Λs,b. (3.18)}

From (3.15) and (3.18) it follows that

S_s,b= f(τ_min)_{Λs,b. (3.19)}

Thus, the proof is complete.

4 Proof of Theorem 1.2

In this section, we investigate the existence of solutions for problem (1.1). We first give some technical lemmas so that we can use the mountain pass lemma to seek critical points of prob- lem (1.1).

The Nehari manifold related to I is given by

N =(u,v)⊂H^˙s(_R^N)\{0} ×H^˙s(_R^N)\{0}:hI⁰(u,v),(u,v)i=0 . Then a minimizer of the minimization problem

c₀ = _inf

u∈NI(u,v)

is a solution of problem (1.1). In order to establish the existence of solutions for problem (1.1), we set

cγ = inf

γ∈_Γmax

t∈[0,1]I(γ(t)), whereΓ={γ∈C([0, 1], ˙H^s(_R^N)):γ(0) =0,γ(e)<0}and

c_s= inf

(u,v)∈Hmax

t≥0 I(t(u,v)), and

c^∗ :=min

a+2s 2(N+a)^S

N+a a+2s

s,a , 2s−b 2(N−b)^S

N−b 2s−b

s,b

. Then we have the following result.