Loading [MathJax]/jax/element/mml/optable/MathOperators.js
Research article Special Issues

Linear Rayleigh-Taylor instability for compressible viscoelastic fluids

  • In this paper, we consider the linear Rayleigh-Taylor instability of an equilibrium state of 3D gravity-driven compressible viscoelastic fluid with the elasticity coefficient κ is less than a critical number κc in a moving horizontal periodic domain. We first construct the maximal growing mode solutions to the linearized equations by studying a family of modified variational problems, and then we prove an estimate for arbitrary solutions to the linearized equations.

    Citation: Caifeng Liu. Linear Rayleigh-Taylor instability for compressible viscoelastic fluids[J]. AIMS Mathematics, 2023, 8(7): 14894-14918. doi: 10.3934/math.2023761

    Related Papers:

    [1] Yiping Meng . On the Rayleigh-Taylor instability for the two coupled fluids. AIMS Mathematics, 2024, 9(11): 32849-32871. doi: 10.3934/math.20241572
    [2] Yusuke Ishigaki, Yoshihiro Ueda . Stability of stationary solutions to outflow problem for compressible viscoelastic system in one dimensional half space. AIMS Mathematics, 2024, 9(11): 33215-33253. doi: 10.3934/math.20241585
    [3] Abdelbaki Choucha, Salah Boulaaras, Asma Alharbi . Global existence and asymptotic behavior for a viscoelastic Kirchhoff equation with a logarithmic nonlinearity, distributed delay and Balakrishnan-Taylor damping terms. AIMS Mathematics, 2022, 7(3): 4517-4539. doi: 10.3934/math.2022252
    [4] Yellamma, N. Manjunatha, Umair Khan, Samia Elattar, Sayed M. Eldin, Jasgurpreet Singh Chohan, R. Sumithra, K. Sarada . Onset of triple-diffusive convective stability in the presence of a heat source and temperature gradients: An exact method. AIMS Mathematics, 2023, 8(6): 13432-13453. doi: 10.3934/math.2023681
    [5] Fahad Alsharari, Mohamed M. Mousa . New application of MOL-PACT for simulating buoyancy convection of a copper-water nanofluid in a square enclosure containing an insulated obstacle. AIMS Mathematics, 2022, 7(11): 20292-20312. doi: 10.3934/math.20221111
    [6] Xi Wang, Xueli Ke . Global unique solutions for the 2-D inhomogeneous incompressible viscoelastic rate-type fluids with stress-diffusion. AIMS Mathematics, 2024, 9(11): 29806-29819. doi: 10.3934/math.20241443
    [7] Keltoum Bouhali, Sulima Ahmed Zubair, Wiem Abedelmonem Salah Ben Khalifa, Najla ELzein AbuKaswi Osman, Khaled Zennir . A new strict decay rate for systems of longitudinal $ m $-nonlinear viscoelastic wave equations. AIMS Mathematics, 2023, 8(1): 962-976. doi: 10.3934/math.2023046
    [8] Jonathan D. Evans, Morgan L. Evans . Stress boundary layers for the Giesekus fluid at the static contact line in extrudate swell. AIMS Mathematics, 2024, 9(11): 32921-32944. doi: 10.3934/math.20241575
    [9] Vincent Giovangigli, Yoann Le Calvez, Guillaume Ribert . Multicomponent thermodynamics with instabilities and diffuse interfaces fluids. AIMS Mathematics, 2024, 9(9): 25979-26034. doi: 10.3934/math.20241270
    [10] Bengisen Pekmen Geridonmez . RBF simulation of natural convection in a nanofluid-filled cavity. AIMS Mathematics, 2016, 1(3): 195-207. doi: 10.3934/Math.2016.3.195
  • In this paper, we consider the linear Rayleigh-Taylor instability of an equilibrium state of 3D gravity-driven compressible viscoelastic fluid with the elasticity coefficient κ is less than a critical number κc in a moving horizontal periodic domain. We first construct the maximal growing mode solutions to the linearized equations by studying a family of modified variational problems, and then we prove an estimate for arbitrary solutions to the linearized equations.



    In this paper we study the compressible viscoelastic fluids in a three-dimensional moving horizontal periodic domain Ω(t) with an upper free surface ΣF(t) and a fixed bottom ΣB (see Figure 1)

    {tρ+div(ρv)=0,inΩ(t),ρ(tv+(v)v)+divS=gρe3,inΩ(t),tU+(v)U=vU,inΩ(t),Sn(t)=P atmn(t)σHn(t),onΣF(t),v=0,onΣB,v|t=0=v0,U|t=0=U0,inΩ(0), (1.1)

    where the viscoelastic stress tensor is given by

    S=P(ρ)Iε(ρ)(D(v)23divvI)δ(ρ)(divv)IκρUUT,

    The fluid is described by density, velocity and deformation tensor function, which are given for each t0 by

    ρ(,t):Ω(t)R,v(,t):Ω(t)R3andU(,t):Ω(t)R3×R3

    respectively.

    Figure 1.  The plan view of Ω(t).

    In this expression the superscript T means matrix transposition and I is the 3×3 identity matrix. The scalar function P is the pressure which is a function of density P=P(ρ)>0, and the pressure function is assumed to be smooth, positive and strictly increasing, and P atm stands for the atmospheric pressure, assumed to be constant. We denote V(ΣF(t)) the outer-normal velocity of the free surface ΣF(t), and V(ΣF(t))=vn(t). The elasticity coefficient κ denotes the constant elasticity coefficient of the fluid, and ε,δ denote the shear viscosity, the bulk viscosity, respectively, and we assume ε>0, δ0, ε(ρ),δ(ρ)C((0,)). we denote n(t) the outward-pointing unit normal on ΣF(t), H twice the mean curvature of the surface ΣF(t) and the surface tension to be a constant σ>0, (D(v))ij=(v+vT)ij=jvi+ivj twice the symmetric gradient of the velocity. The constant g>0 stands for the strength of gravity, e3=(0,0,1) is the vertical unit vector, and gρe3 is known as the gravitational force. We write divS for the vector with ith component jSij.

    The equilibrium in a uniform gravitational field, in which a heavy fluid is on top of a light fluid, is unstable. This phenomenon was first studied by Rayleigh [20] and then Taylor [21], and is called therefore the Rayleigh-Taylor instability. In the last decades, this phenomenon has been extensively investigated from both physical and numerical aspects, see [1,10,13,15] for examples. It has been also widely investigated how the Rayleigh-Taylor instability evolves under the effects of other physical factors, internal surface tension [6,24], magnetic fields [2,9,11,12,14], and so on.

    We mention some previous mathematical results concerning the Rayleigh-Taylor instability. For the inviscid Rayleigh-Taylor problem without surface tension, Ebin [3] proved the nonlinear ill-posedness of the problem for incompressible fluids. Guo and Tice [5] showed an analogous result for compressible fluids. Hwang and Guo [8] obtained the nonlinear instability of the incompressible problem with a continuous density distribution. For the viscous Rayleigh-Taylor problem, Prüss and Simonett [19] proved the nonlinear instability for incompressible fluids with surface tension in an Lp setting by using Henry's instability theorem. Wang and Tice [23] established the sharp nonlinear instability criteria for the incompressible surface-internal wave problem with or without surface tension. Jang, Tice and Wang [17] proved the nonlinear instability of the dynamics of two layers of compressible, barotropic, viscous fluid lying atop one another via an argument from Jang and Tice [16], in which the authors utilized the linear growing mode to construct initial data for the nonlinear problem.

    For the viscoelastic Rayleigh-Taylor problem, Huang, Jiang and Wang [7] obtained that the nonhomogeneous incompressible viscoelastic Rayleigh-Taylor equilibrium state is unstable in L2-norm based on a bootstrap instability method. Wang and Zhao [22] proved the instability of compressible viscoelastic Rayleigh-Taylor problem in the sense of Hadamard. There are also a lot of great papers for studying the viscoelastic Rayleigh-Taylor problem, such as [18,25] and their references.

    In this paper, we investigate the linear Rayleigh-Taylor instability for the compressible viscoelastic fluid around a steady-state profile with heavier fluid lying above lighter fluid. We consider the equations with surface tension and the viscosity allowed to depend on the density. The linear instability analysis of this paper comprises the first step in an analysis of the nonlinear instability of the compressible viscoelastic fluids, which will be left for our future work.

    Formulation in Lagrangian Coordinates We use Lagrange transformation to change the free boundary into a fixed boundary. Firstly, we define the fixed Lagrangian domain to be the horizontal periodic slab

    Ω={x=(xh,x3)|b<x3<0,xh=(x1,x2)(2πLT)2}

    with the bottom Σb={x3=b} and the top surface Σ0={x3=0}, where the positive constant b is the depth of the fluid at infinity, and 2πLT stands for the 1D-torus of length 2πL.

    We assume that there exists a mapping

    η0:ΩΩ(0)

    that is invertible, and satisfies ΣF(0)=η0(Σ0), and ΣB=η0(Σb).

    We define the flow map η, as the solution to

    {tη(t,x)=v(t,η(x,t)),t>0,xΩ,η(0,x)=η0(x),xΩ.

    This implies that Ω(t)=η(t,Ω), ΣF(t)=η(Σ0), and ΣB=η(Σb). In order to switch back and forth from Lagrangian to Eulerian coordinates we assume that η(t,x) is invertible.

    We define the Lagrangian unknowns

    u(t,x)def=v(t,η(t,x)),q(t,x)def=ρ(t,η(t,x)),V(t,x)def=U(t,η(t,x)),

    which are defined for (t,x)R+×Ω.

    Define the matrix A via A=(Dη)T, where

    Dη=(1η12η13η11η22η23η21η32η33η3).

    By using the chain rule, we can directly derive

    t(ρ(t,η(t,x)))=(tρ)(t,η(t,x))+(vρ)(t,η(t,x)),t(v(t,η(t,x)))=(tv)(t,η(t,x))+(vv)(t,η(t,x)),t(U(t,η(t,x)))=(tU)(t,η(t,x))+(vU)(t,η(t,x)),(jvi)(t,η(t,x))=(kηj)1k(vi(t,η(t,x)))=Ajkk(vi(t,η(t,x))),(ivi)(t,η(t,x))=(jηi)1j(vi(t,η(t,x)))=Aijj(vi(t,η(t,x))), (1.2)

    where we used the fact that A(Dη)T=I, i.e. Alkkηj=δlj. Using (1.2), we can derive the Lagrangian form of (1.1). Writing j=/xj, the evolution equations for u,q,V,η in Lagrangian coordinates are,

    {tηi=ui,inΩ,tq+qAijjui=0,inΩ,qtui+AjkkTij=gqAijjη3,inΩ,tVkm=AijjukVim,inΩ,Tn=P atmnσHn,onΣ0,u|Σb=0,η|t=0=η0,u|t=0=u0,V|t=0=V0 (1.3)

    where

    Tij=P(q)Iijε(q)(Ajkkui+Aikkuj23(Alkkul)Iij)δ(q)(Alkkul)IijκqVikVjk.

    Here we have employed the Einstein convention of summing over repeated indices and written

    n:=1η×2η|1η×2η||x3=0

    in Lagrangian coordinates and

    H=(|1η|222η2(1η2η)12η+|2η|221η|1η|2|2η|2|1η2η|2)n.

    Steady-state solution We now seek a steady-state equilibrium solution to (1.3) with u=0,η=Id,q(t,x)=ρ0(x3),V=ˉU, where

    ˉUdef=(ˉu000ˉu000ˉu)

    with

    ˉuˉu(x3)def=±F(P(ρ0)ρ0+gρ0)+Cκρ0,

    here F(P(ρ0)ρ0+gρ0) denotes a primitive function of P(ρ0)ρ0+gρ0 and C is a positive constant satisfying

    infx3[b,0]{F(P(ρ0)ρ0+gρ0)+C}>0.

    By a direct calculation, we can find that ˉUˉUT=ˉu2I. Then the system (1.3) reduces to the following ODE with the equilibrium density ρ0=ρ0(x3)

    {d(P(ρ0)κρ0ˉu2)dx3=gρ0x3(b,0),P(ρ0)κρ0ˉu2=P atm,atx3=0. (1.4)

    Noticed that, κρ0ˉu2 can be expressed as a function of variable ρ0 by G(ρ0), i.e., G(ρ0)def=κρ0ˉu2, and we define M(ρ0)def=P(ρ0)κρ0ˉu2. We consider the case that the density satisfies the following conditions:

    ρ0C(ˉΩ),infx3[b,0]ρ0(x3)>0, (1.5)

    and the Rayleigh-Taylor condition

    infx3[b,0]ρ0(x3)>0. (1.6)

    The Rayleigh-Taylor condition assures that the density has larger density with increasing height x3, thus leading to the classical Rayleigh-Taylor instability. From (1.4) and (1.5), we can get M(ρ0)ρ0=gρ0<0, which along with (1.6) implies M(ρ0)<0, ρ0[ˉρ2,ˉρ1], i.e. M(ρ0) is strictly decreasing with respect to ρ0 on [ˉρ2,ˉρ1], where P(ρ0)=P(s)|s=ρ0, M(ρ0)=M(s)|s=ρ0 and ρ0(x3)=dρ0dx3, ˉρ1def=M1(P atm), ˉρ2def=ρ0(b).

    We may claim that the necessary and sufficient for the existence of an equilibrium to (1.4) are as follows:

    (1)P atmM(R+),(2)0<b1gˉρ2ˉρ1M(s)sds. (1.7)

    In fact, since the function M=M(ρ0) is smooth and strictly decreasing with respect to ρ0 on [ˉρ2,ˉρ1], the second equation in (1.4) holds if and only if P atmM(R+), which defines ˉρ1=M1(P atm), that is, the first condition in (1.7) holds. On the other hand, we introduce the function h:(0,+)R given by

    h(z)def=zˉρ1M(s)sds,

    which is smooth, strictly decreasing and positive on [ˉρ2,ˉρ1]. From (1.4), we get

    {dh(ρ0)dx3=g,x3(b,0),ρ0(0)=ˉρ1.

    Solve this ODE to find ρ0(x3)=h1(gx3), which gives a well-defined, smooth and increasing function ρ0:[b,0][ˉρ2,ˉρ1] if and only if

    gbh([ˉρ2,ˉρ1)),

    that is, the second condition in (1.7) holds.

    Linearization around the steady-state We now linearize the Eq (1.3) around the steady-state solution u=0,η=Id,q(t,x)=ρ0(x3),V=ˉU. Then the resulting linearized viscoelastic equations are

    {tη=uinΩ,tq+ρ0divu=0inΩ,ρ0tu+(P(ρ0)q)+gqe3+gρ0η3=div(ε0(D(u)23(divu)I)+δ0(divu)I+κρ0(VˉUT+ˉUVT)+κqˉu2I)inΩ,tV=uˉUinΩ,(P(ρ0)qIε0D(u)(δ023ε0)divuIκρ0(VˉUT+ˉUVT)κqˉu2I)e3=σΔx1,x2η3e3onΣ0,u|Σb=0, (1.8)

    where ε0=ε(ρ0) and δ0=δ(ρ0).

    Before further stating our result, we shall introduce some notations used throughout this paper. We first define the weighted L2 norm and the viscosity seminorm by

    u2=Ωρ0|u|2dxandu2=Ωε02|D(u)23(divu)I|2+δ0|divu|2dx. (1.9)

    And we denote

    H10:={wH1(Ω)|w|Σb=0},Ψ(w)=Ωρ0ˉu2(12|D(w)|2|divw|2)dx,Θ1(w):=Ω(gρ0w3divwgρ0ww3)dxΩP(ρ0)ρ0|divw|2dxΣ0σ|x1,x2w3|2dS0,Θ(w)=Θ1(w)κΨ(w),˜κc:=supwH10Θ(w)Ψ(w),a

    Remark 1.1. We mention that

    \begin{equation} \begin{split} \Psi(w)\geq0, \end{split} \end{equation} (1.10)

    for all w\in H_{0}^{1} . In fact, by a direct calculation,

    \begin{equation*} \begin{split} &\int_{\Omega}\rho_{0}\bar{u}^{2}(\frac{1}{2}|\mathbb{D}(w)|^{2}-|\mbox{div}\, w|^{2})dx \\ & = \int_{\Omega}\rho_{0}\bar{u}^{2}((\partial_{1}w_{2}+\partial_{2}w_{1})^{2} +(\partial_{1}w_{3}+\partial_{3}w_{1})^{2}+(\partial_{2}w_{3}+\partial_{3}w_{2})^{2}\\ &\qquad+|\partial_{1}w_{1}|^{2}+|\partial_{2}w_{2}|^{2}+|\partial_{3}w_{3}|^{2})dx. \end{split} \end{equation*}

    Hence (1.10) holds. Moreover, we can see that, if \Psi(w) = 0 , for some w\in H_{0}^{1} , then w = 0 . This implies that \widetilde{\kappa}_{c} > 0 is equivalent to the condition:

    {there\; exists\; a\; function}\; \widetilde{w}\in H_{0}^{1}\; {such\; that}\; \Theta(\widetilde{w}) > 0,

    and \widetilde{\kappa}_{c} > 0 is also equivalent to the following condition:

    \begin{equation} \begin{split} \kappa < \kappa_{c}: = \sup\limits_{w\in H_{0}^{1}}\frac{\Theta_{1}(w)}{\Psi(w)}. \end{split} \end{equation} (1.11)

    Now we state our main result.

    Theorem 1.1. Let (u, \eta, q, V) be a solution to (1.8). Then

    \begin{equation*} \begin{split} &\|u(t)\|_{\ast}^{2}+\|u(t)\|_{\ast\ast}^{2}+\|\partial_{t}u(t)\|_{\ast}^{2}\leq Ce^{2\Lambda t}I_{0}, \; \; \|\eta(t)\|_{H^{1}}\leq Ce^{\Lambda t}(\|\eta_{0}\|_{H^{1}}+\sqrt{I_{0}}), \\ &\|q(t)\|_{L^{2}}\leq Ce^{\Lambda t}(\|q_{0}\|_{L^{2}}+\sqrt{I_{0}}), \; \; \|V(t)\|_{L^{2}}\leq Ce^{\Lambda t}(\|V_{0}\|_{L^{2}}+\sqrt{I_{0}}), \end{split} \end{equation*}

    for a constant 0 < C = C(\rho_{0}, P, \Lambda, \varepsilon, \delta, \kappa, \sigma, g, m, l) , where \Lambda is defined in (2.37) below and

    I_{0} = \|\partial_{t}u(0)\|_{\ast}^{2}+\|u(0)\|_{\ast}^{2}+\|u(0)\|_{\ast\ast}^{2}+ \sigma\int_{\Sigma_{0}}|\nabla_{x_{1}, x_{2}}u_{3}(0)|^{2}.

    The rest of the paper is organized as follows. In section 2, we construct a growing mode solution to (1.8) by assuming an ansatz, and then we study a family of modified variational problems in order to produce maximal growing modes. In section 3, we firstly take the preliminary estimates, and then prove our main result.

    We want to construct a growing mode solution to (1.8) by assuming an ansatz

    \begin{equation} \begin{split} u(t, x) = w(x)e^{\lambda t}, \; V(t, x) = \widetilde{V}(x)e^{\lambda t}, \; \eta(t, x) = \widetilde{\eta}(x)e^{\lambda t}, \; q(t, x) = \widetilde{q}(x)e^{\lambda t}, \end{split} \end{equation} (2.1)

    for some \lambda > 0 . Substituting this ansatz into (1.8)_{1} , (1.8)_{2} and (1.8)_{4} , respectively, we find that \widetilde{\eta} = \lambda^{-1}w, \; \widetilde{q} = -\lambda^{-1}\rho_{0}\mbox{div}\, w, \; \widetilde{V} = \lambda^{-1}\nabla w\bar{U} . By this fact we can eliminate \widetilde{\eta}, \widetilde{q}, \widetilde{V} from (1.8) and then arrive at the following time-invariant system for w :

    \begin{equation} \begin{split} &\lambda^{2}\rho_{0}w-\nabla(P'(\rho_{0})\rho_{0}\mbox{div}\, w)-g\rho_{0}\mbox{div}\, we_{3}+g\rho_{0}\nabla w_{3}\\ & = \mbox{div}(\lambda\varepsilon_{0}(\mathbb{D}(w)-\frac{2}{3}(\mbox{div}\, w)\mathbb{I}) +\lambda\delta_{0}(\mbox{div}\, w)\mathbb{I}-\kappa\rho_{0}\bar{u}^{2}(\mbox{div}\, w)\mathbb{I} +\kappa\rho_{0}\bar{u}^{2}\mathbb{D}(w)). \end{split} \end{equation} (2.2)

    The boundary conditions linearize to: on \Sigma_{0}

    \begin{equation*} \begin{split} ((\lambda\delta_{0}-2\lambda\varepsilon_{0}/3+&P'(\rho_{0})\rho_{0}-\kappa\rho_{0}\bar{u}^{2}) \mbox{div}\, w\mathbb{I}+(\lambda\varepsilon_{0}+\kappa\rho_{0}\bar{u}^{2}) \mathbb{D}(w))e_{3} = \sigma\Delta_{x_{1}, x_{2}}w_{3}e_{3}, \end{split} \end{equation*}

    and w|_{\Sigma_{b}} = 0 . Since the domain is horizontally flat, we are free to make the further structural assumption that the x' dependence of w is given as a Fourier mode e^{ix'\cdot\xi} , where x'\cdot\xi = x_{1}\xi_{1}+x_{2}\xi_{2} for \xi\in (L^{-1}\mathbb{Z})^{2} , x' = (x_{1}, x_{2}) . Together with the growing mode ansatz (2.1), this constitutes a "normal mode" ansatz (see [1]). We define the new unknowns \varphi, \theta, \psi: (-b, 0)\rightarrow \mathbb{R} according to

    \begin{equation*} \begin{split} w_{1}(x) = -i\varphi(x_{3})e^{ix'\cdot\xi}, \; w_{2}(x) = -i\theta(x_{3})e^{ix'\cdot\xi}, \; \mbox{and} \; w_{3}(x) = \psi(x_{3})e^{ix'\cdot\xi}. \end{split} \end{equation*}

    By a direct calculation, we can get

    \begin{equation*} \begin{split} \mbox{div}\, w = (\xi_{1}\varphi+\xi_{2}\theta+\psi')e^{ix'\cdot\xi}, \end{split} \end{equation*}

    and

    \begin{equation} \begin{split} &\nabla w+\nabla w^{T} = \left( \begin{array}{lll} \; \; \; 2\xi_{1}\varphi & \xi_{1}\theta+\xi_{2}\varphi & i(\xi_{1}\psi-\varphi')\\ \xi_{1}\theta+\xi_{2}\varphi & \; \; \; 2\xi_{2}\theta & i(\xi_{2}\psi-\theta')\\ i(\xi_{1}\psi-\varphi') & i(\xi_{2}\psi-\theta') & \; \; \; \; 2\psi' \\ \end{array}\right)e^{ix'\cdot\xi}. \end{split} \end{equation} (2.3)

    For each fixed nonzero spatial frequency \xi , we deduce from the Eq (2.2) a system of ODEs for \varphi, \theta, \psi, \lambda , denoting ' = d/dx_{3} : in (-b, 0)

    \begin{equation} \begin{split} &-(\lambda\varepsilon_{0}\varphi'+\kappa\rho_{0}\bar{u}^{2}\varphi')'+[\lambda^{2}\rho_{0} +(\lambda\varepsilon_{0}+\kappa\rho_{0}\bar{u}^{2})|\xi|^{2}+\xi_{1}^{2}(\lambda\delta_{0}+\frac{1}{3}\lambda\varepsilon_{0} +P'(\rho_{0})\rho_{0})]\varphi\\ & = -\xi_{1}[(\lambda\delta_{0}+\frac{1}{3}\lambda\varepsilon_{0}+P'(\rho_{0})\rho_{0})\psi' +(\lambda\varepsilon'_{0}+(\kappa\rho_{0}\bar{u}^{2})'-g\rho_{0})\psi]-\xi_{1}\xi_{2} [\lambda\delta_{0}+\frac{1}{3}\lambda\varepsilon_{0}+P'(\rho_{0})\rho_{0}]\theta, \end{split} \end{equation} (2.4)
    \begin{equation} \begin{split} &-(\lambda\varepsilon_{0}\theta'+\kappa\rho_{0}\bar{u}^{2}\theta')'+[\lambda^{2}\rho_{0} +(\lambda\varepsilon_{0}+\kappa\rho_{0}\bar{u}^{2})|\xi|^{2}+\xi_{2}^{2}(\lambda\delta_{0}+\frac{1}{3}\lambda\varepsilon_{0} +P'(\rho_{0})\rho_{0})]\theta\\ & = -\xi_{2}[(\lambda\delta_{0}+\frac{1}{3}\lambda\varepsilon_{0}+P'(\rho_{0})\rho_{0})\psi' +(\lambda\varepsilon'_{0}+(\kappa\rho_{0}\bar{u}^{2})'-g\rho_{0})\psi]-\xi_{1}\xi_{2} [\lambda\delta_{0}+\frac{1}{3}\lambda\varepsilon_{0}+P'(\rho_{0})\rho_{0}]\varphi, \end{split} \end{equation} (2.5)
    \begin{equation} \begin{split} &-[(4\lambda\varepsilon_{0}/3+\lambda\delta_{0}+P'(\rho_{0})\rho_{0}+\kappa\rho_{0}\bar{u}^{2})\psi']' +(\lambda^{2}\rho_{0}+\lambda\varepsilon_{0}|\xi|^{2}+\kappa\rho_{0}\bar{u}^{2}|\xi|^{2})\psi\\ & = [(\lambda\delta_{0}+\frac{1}{3}\lambda\varepsilon_{0}+P'(\rho_{0})\rho_{0})(\xi_{1}\varphi+\xi_{2}\theta)]' +(g\rho_{0}-\lambda\varepsilon'_{0}-(\kappa\rho_{0}\bar{u}^{2})')(\xi_{1}\varphi+\xi_{2}\theta).\qquad\qquad \qquad\quad \end{split} \end{equation} (2.6)

    The upper boundary condition for the new unknowns is:

    \begin{equation*} \begin{split} \bigg((\lambda\delta_{0}-&2\lambda\varepsilon_{0}/3+P'(\rho_{0})\rho_{0}-\kappa\rho_{0}\bar{u}^{2}) (\xi_{1}\varphi+\xi_{2}\theta+\psi')e_{3}\\ &+(\lambda\varepsilon_{0}+\kappa\rho_{0}\bar{u}^{2}) \left( \begin{array}{lll} i(\xi_{1}\psi-\varphi') \\ i(\xi_{2}\psi-\theta')\\ \; \; \; \; \; 2\psi' \\ \end{array}\right) \bigg) = -\sigma|\xi|^{2}\psi e_{3}, \; \; \mbox{at}\; x_{3} = 0, \end{split} \end{equation*}

    which follows that at x_{3} = 0

    \begin{equation*} \begin{split} (\lambda\varepsilon_{0}+\kappa\rho_{0}\bar{u}^{2})(\varphi'-\xi_{1}\psi) = (\lambda\varepsilon_{0}+\kappa\rho_{0}\bar{u}^{2})(\theta'-\xi_{2}\psi) = 0, \end{split} \end{equation*}

    and

    \begin{equation*} \begin{split} (\lambda\delta_{0}+\lambda\varepsilon_{0}/3+P'(\rho_{0})\rho_{0})(\psi'+\xi_{1}\varphi+\xi_{2}\theta) +(\lambda\varepsilon_{0}+\kappa\rho_{0}\bar{u}^{2})(\psi'-\xi_{1}\varphi-\xi_{2}\theta) = -\sigma|\xi|^{2}\psi. \end{split} \end{equation*}

    and the bottom boundary conditions become

    \begin{equation*} \begin{split} \varphi(-b) = \theta(-b) = \psi(-b) = 0. \end{split} \end{equation*}

    Note that if \varphi, \theta, \psi solve the Eqs (2.4)–(2.6) for \xi\in (L^{-1}\mathbb{Z})^{2} and \lambda , then for any rotation operator R\in SO(2) , (\widetilde{\varphi}, \widetilde{\theta}): = R(\varphi, \theta) solve the same equations for \widetilde{\xi}: = R\xi with \psi, \lambda unchanged. Then, we choose a rotation operator R so that R\xi = (|\xi|, 0) to see that \theta solves

    \begin{equation} \begin{cases} -(\lambda\varepsilon_{0}\theta'+\kappa\rho_{0}\bar{u}^{2}\theta')'+(\lambda^{2}\rho_{0} +(\lambda\varepsilon_{0}+\kappa\rho_{0}\bar{u}^{2})|\xi|^{2})\theta = 0, \; \; \mbox{in}\; (-b, 0), \\ (\lambda\varepsilon_{0}+\kappa\rho_{0}\bar{u}^{2})\theta' = 0, \; \; \mbox{at}\; x_{3} = 0, \\ \theta(-b) = 0. \end{cases} \end{equation} (2.7)

    Multiplying (2.7) by \theta , integrating over (-b, 0) , integrating by parts, and using the boundary conditions then yields

    \begin{equation*} \begin{split} \int_{-b}^{0}(\lambda\varepsilon_{0}+\kappa\rho_{0}\bar{u}^{2})|\theta'|^{2}+(\lambda^{2}\rho_{0} +(\lambda\varepsilon_{0}+\kappa\rho_{0}\bar{u}^{2})|\xi|^{2})\theta^{2}dx_{3} = 0, \end{split} \end{equation*}

    which implies that \theta = 0 . Then (2.4)–(2.6) reduce to the pair of equations in (-b, 0) for \varphi, \psi

    \begin{equation} \begin{split} -\lambda^{2}\rho_{0}\varphi& = -(\lambda\varepsilon_{0}\varphi'+\kappa\rho_{0}\bar{u}^{2}\varphi')' +|\xi|^{2}(4\lambda\varepsilon_{0}/3+\lambda\delta_{0}+P'(\rho_{0})\rho_{0} +\kappa\rho_{0}\bar{u}^{2})\varphi\\ &\qquad\quad+|\xi|[(\lambda\delta_{0}+\lambda\varepsilon_{0}/3+P'(\rho_{0})\rho_{0})\psi'+ (\lambda\varepsilon'_{0}+(\kappa\rho_{0}\bar{u}^{2})'-g\rho_{0})\psi], \end{split} \end{equation} (2.8)
    \begin{equation} \begin{split} -\lambda^{2}\rho_{0}\psi = -[(&4\lambda\varepsilon_{0}/3+\lambda\delta_{0}+P'(\rho_{0})\rho_{0} +\kappa\rho_{0}\bar{u}^{2})\psi']'+(\lambda\varepsilon_{0}+\kappa\rho_{0}\bar{u}^{2})|\xi|^{2}\psi\\ &\qquad-|\xi|[((\lambda\delta_{0}+\lambda\varepsilon_{0}/3+P'(\rho_{0})\rho_{0})\varphi)'+ (g\rho_{0}-\lambda\varepsilon'_{0}-(\kappa\rho_{0}\bar{u}^{2})')\varphi], \end{split} \end{equation} (2.9)

    along with the boundary conditions: \mbox{at}\; x_{3} = 0

    \begin{equation} \begin{split} (\lambda\varepsilon_{0}+\kappa\rho_{0}\bar{u}^{2})(\varphi'-|\xi|\psi) = 0, \end{split} \end{equation} (2.10)
    \begin{equation} \begin{split} (\lambda\delta_{0}+\lambda\varepsilon_{0}/3+P'(\rho_{0})\rho_{0})(\psi'+|\xi|\varphi) +(\lambda\varepsilon_{0}+\kappa\rho_{0}\bar{u}^{2})(\psi'-|\xi|\varphi) = -\sigma|\xi|^{2}\psi, \end{split} \end{equation} (2.11)

    and \mbox{at}\; x_{3} = -b

    \begin{equation} \begin{split} \varphi = \psi = 0. \end{split} \end{equation} (2.12)

    Applying a modified variational method to construct a solution of (2.8)–(2.12), we modify (2.8)–(2.12) as follows: in (-b, 0)

    \begin{equation} \begin{split} -\lambda^{2}\rho_{0}\varphi = -(s\varepsilon_{0}\varphi'+&\kappa\rho_{0}\bar{u}^{2}\varphi')' +|\xi|^{2}(\frac{4}{3}s\varepsilon_{0}+s\delta_{0}+P'(\rho_{0})\rho_{0} +\kappa\rho_{0}\bar{u}^{2})\varphi\\ &\qquad+|\xi|[(s\delta_{0}+\frac{1}{3}s\varepsilon_{0}+P'(\rho_{0})\rho_{0})\psi'+ (s\varepsilon_{0}'+(\kappa\rho_{0}\bar{u}^{2})'-g\rho_{0})\psi], \end{split} \end{equation} (2.13)
    \begin{equation} \begin{split} -\lambda^{2}\rho_{0}\psi = -[(&\frac{4}{3}s\varepsilon_{0}+s\delta_{0}+P'(\rho_{0})\rho_{0} +\kappa\rho_{0}\bar{u}^{2})\psi']'+(s\varepsilon_{0}+\kappa\rho_{0}\bar{u}^{2})|\xi|^{2}\psi\\ &\qquad\quad\qquad-|\xi|[((s\delta_{0}+\frac{1}{3}s\varepsilon_{0}+P'(\rho_{0})\rho_{0})\varphi)'+ (g\rho_{0}-s\varepsilon_{0}'-(\kappa\rho_{0}\bar{u}^{2})')\varphi], \end{split} \end{equation} (2.14)

    where s > 0 is an arbitrary parameter, along with the boundary conditions: at x_{3} = 0

    \begin{equation} \begin{split} (s\varepsilon_{0}+\kappa\rho_{0}\bar{u}^{2})(\varphi'-|\xi|\psi) = 0, \end{split} \end{equation} (2.15)
    \begin{equation} \begin{split} (s\delta_{0}+\frac{1}{3}s\varepsilon_{0}+P'(\rho_{0})\rho_{0})(\psi'+|\xi|\varphi) +(s\varepsilon_{0}+\kappa\rho_{0}\bar{u}^{2})(\psi'-|\xi|\varphi) = -\sigma|\xi|^{2}\psi, \end{split} \end{equation} (2.16)

    and at x_{3} = -b

    \begin{equation} \begin{split} \varphi = \psi = 0. \end{split} \end{equation} (2.17)

    Note that for any fixed s > 0 and \xi , (2.13) and (2.14) is a standard eigenvalue problem for -\lambda^{2} , which has a natural variational structure that allows us to use variational methods to construct solutions. For this, we define the energies

    \begin{equation} \begin{split} E(\varphi, \psi;|\xi|, s) = E_{0}(\varphi, \psi;|\xi|)+sE_{1}(\varphi, \psi;|\xi|), \end{split} \end{equation} (2.18)

    and

    \begin{equation*} \begin{split} J(\varphi, \psi) = \frac{1}{2}\int_{-b}^{0}\rho_{0}(\varphi^{2}+\psi^{2})dx_{3}, \end{split} \end{equation*}

    where

    \begin{equation*} \begin{split} E_{0}(\varphi, \psi;|\xi|): = &\frac{\sigma|\xi|^{2}}{2}(\psi(0))^{2}+\frac{1}{2}\int_{-b}^{0} P'(\rho_{0})\rho_{0}(\psi'+|\xi|\varphi)^{2}-2g\rho_{0}|\xi|\varphi\psi dx_{3}\\ &\quad+\frac{1}{2}\int_{-b}^{0}\kappa\rho_{0}\bar{u}^{2} ((\varphi'-|\xi|\psi)^{2}+(\psi'-|\xi|\varphi)^{2})dx_{3}, \end{split} \end{equation*}
    \begin{equation*} \begin{split} E_{1}(\varphi, \psi;|\xi|): = \frac{1}{2}\int_{-b}^{0}(\delta_{0}+\frac{1}{3}\varepsilon_{0}) (\psi'+|\xi|\varphi)^{2} +\varepsilon_{0}((\varphi'-|\xi|\psi)^{2}+(\psi'-|\xi|\varphi)^{2})dx_{3}, \end{split} \end{equation*}

    which are both well-defined on the space H_{0}^{1}((-b, 0))\times H_{0}^{1}((-b, 0)) , where H_{0}^{1}((-b, 0)): = \{f\in H^{1}(-b, 0)|\; f(-b) = 0\} . Consider the set

    \begin{equation*} \begin{split} \mathcal{C} = \{(\varphi, \psi)\in H_{0}^{1}((-b, 0))\times H_{0}^{1}((-b, 0))|J(\varphi, \psi) = 1\}. \end{split} \end{equation*}

    Notice that by employing the identity -2ab = (a-b)^{2}-(a^{2}+b^{2}) and the constraint on J(\varphi, \psi) we may rewrite

    \begin{equation} \begin{split} &E(\varphi, \psi;|\xi|, s) = \frac{\sigma|\xi|^{2}}{2}(\psi(0))^{2}+\frac{1}{2}\int_{-b}^{0} P'(\rho_{0})\rho_{0}(\psi'+|\xi|\varphi)^{2}+g\rho_{0}|\xi|(\varphi-\psi)^{2}dx_{3} \\ &\qquad-\frac{1}{2}\int_{-b}^{0}g\rho_{0}|\xi|(\varphi^{2}+\psi^{2})dx_{3} +\frac{1}{2}\int_{-b}^{0}\kappa\rho_{0}\bar{u}^{2}((\varphi'-|\xi|\psi)^{2}+(\psi'-|\xi|\varphi)^{2})dx_{3}\\ &\quad+\frac{s}{2}\int_{-b}^{0}(\delta_{0}+\frac{1}{3}\varepsilon_{0}) (\psi'+|\xi|\varphi)^{2} +\varepsilon_{0}((\varphi'-|\xi|\psi)^{2}+(\psi'-|\xi|\varphi)^{2})dx_{3}\geq-g|\xi| \end{split} \end{equation} (2.19)

    for any (\varphi, \psi)\in\mathcal{C} . Then we want to find the smallest -\lambda^{2} by minimizing

    \begin{equation} \begin{split} -\lambda^{2}(|\xi|;s) = \alpha(|\xi|;s): = \inf\limits_{(\varphi, \psi)\in\mathcal{C}}E(\varphi, \psi;|\xi|, s). \end{split} \end{equation} (2.20)

    The first Lemma asserts that a minimizer of E(\varphi, \psi; |\xi|, s) in (2.18) over \mathcal{C} exists and the minimizer solves (2.13)–(2.17).

    Lemma 2.1. Let \xi and s > 0 be fixed. Then the following hold:

    (1) E(\varphi, \psi; |\xi|, s) achieves its infimum over \mathcal{C} .

    (2) Let (\widetilde{\varphi}, \widetilde{\psi})\in\mathcal{C} be the minimizers of E constructed in (1). Let \alpha: = E(\widetilde{\varphi}, \widetilde{\psi}; |\xi|, s) . Then (\widetilde{\varphi}, \widetilde{\psi}) are smooth in (-b, 0) and satisfy

    \begin{equation} \begin{split} \alpha\rho_{0}\widetilde{\varphi} = -(s\varepsilon_{0}\widetilde{\varphi}'+&\kappa\rho_{0}\bar{u}^{2}\widetilde{\varphi}')' +|\xi|^{2}(\frac{4}{3}s\varepsilon_{0}+s\delta_{0}+P'(\rho_{0})\rho_{0} +\kappa\rho_{0}\bar{u}^{2})\widetilde{\varphi}\\ &\qquad+|\xi|[(s\delta_{0}+\frac{1}{3}s\varepsilon_{0}+P'(\rho_{0})\rho_{0})\widetilde{\psi}'+ (s\varepsilon_{0}'+(\kappa\rho_{0}\bar{u}^{2})'-g\rho_{0})\widetilde{\psi}], \end{split} \end{equation} (2.21)
    \begin{equation} \begin{split} \alpha\rho_{0}\widetilde{\psi} = -[(&\frac{4}{3}s\varepsilon_{0}+s\delta_{0}+P'(\rho_{0})\rho_{0} +\kappa\rho_{0}\bar{u}^{2})\widetilde{\psi}']'+(s\varepsilon_{0}+\kappa\rho_{0}\bar{u}^{2})|\xi|^{2}\widetilde{\psi}\\ &\qquad\quad\qquad-|\xi|[((s\delta_{0}+\frac{1}{3}s\varepsilon_{0}+P'(\rho_{0})\rho_{0})\widetilde{\varphi})'+ (g\rho_{0}-s\varepsilon_{0}'-(\kappa\rho_{0}\bar{u}^{2})')\widetilde{\varphi}], \end{split} \end{equation} (2.22)

    along with the boundary conditions: at x_{3} = 0

    \begin{equation*} \begin{split} (s\varepsilon_{0}+\kappa\rho_{0}\bar{u}^{2})(\widetilde{\varphi}'-|\xi|\widetilde{\psi}) = 0, \end{split} \end{equation*}
    \begin{equation*} \begin{split} (s\delta_{0}+\frac{1}{3}s\varepsilon_{0}+P'(\rho_{0})\rho_{0})(\widetilde{\psi}'+|\xi|\widetilde{\varphi}) +(s\varepsilon_{0}+\kappa\rho_{0}\bar{u}^{2})(\widetilde{\psi}'-|\xi|\widetilde{\varphi}) = -\sigma|\xi|^{2}\widetilde{\psi}, \end{split} \end{equation*}

    and \widetilde{\varphi}(-b) = \widetilde{\psi}(-b) = 0 .

    Proof. (1) Let (\varphi_{n}, \psi_{n})\in \mathcal{C} be a minimizing sequence. By (2.19), one can see

    \begin{equation*} \begin{split} E(\varphi_{n}, \psi_{n};|\xi|, s)\geq-g|\xi|, \end{split} \end{equation*}

    which shows that E(\varphi_{n}, \psi_{n}; |\xi|, s) is bounded below on \mathcal{C} . Then, there exists a pair (\widetilde{\varphi}, \widetilde{\psi})\in\mathcal{C} and a subsequence (still denoted by (\varphi_{n}, \psi_{n}) for simplicity), such that (\varphi_{n}, \psi_{n})\rightarrow (\widetilde{\varphi}, \widetilde{\psi}) weakly in H_{0}^{1} and strongly in L^{2} . Moreover, by the lower semi-continuity, we find

    \begin{equation*} \begin{split} &\inf\limits_{(\varphi, \psi)\in\mathcal{C}} E(\varphi, \psi;|\xi|, s) = \liminf\limits_{n\rightarrow \infty}E(\varphi_{n}, \psi_{n};|\xi|, s)\\ & = \lim\limits_{n\rightarrow \infty}\bigg(\frac{\sigma|\xi|^{2}}{2}(\psi_{n}(0))^{2} +\frac{1}{2}\int_{-b}^{0}P'(\rho_{0})\rho_{0}(\psi_{n}'+|\xi|\varphi_{n})^{2} +g\rho_{0}|\xi|(\varphi_{n}-\psi_{n})^{2}dx_{3} \\ &\quad+\frac{s}{2}\int_{-b}^{0}(\delta_{0}+\frac{1}{3}\varepsilon_{0}) (\psi_{n}'+|\xi|\varphi_{n})^{2}dx_{3} +\varepsilon_{0}((\varphi_{n}'-|\xi|\psi_{n})^{2} +(\psi_{n}'-|\xi|\varphi_{n})^{2})dx_{3}\\ &\quad+\frac{1}{2}\int_{-b}^{0}\kappa\rho_{0}\bar{u}^{2}((\varphi_{n}' -|\xi|\psi_{n})^{2}+(\psi_{n}'-|\xi|\varphi_{n})^{2})dx_{3}\bigg)\\ &\quad-\frac{1}{2}\limsup\limits_{n\rightarrow \infty}\int_{-b}^{0}g\rho_{0}|\xi|(\varphi_{n}^{2} +\psi_{n}^{2})dx_{3}\geq E(\widetilde{\varphi}, \widetilde{\psi};|\xi|, s)\geq \inf\limits_{(\varphi, \psi)\in\mathcal{C}} E(\varphi, \psi;|\xi|, s). \end{split} \end{equation*}

    This implies that E(\varphi, \psi; |\xi|, s) achieves its infimum over \mathcal{C} .

    (2) We refer to Propositions 3.2 of [6], just only need to replace s\varepsilon_{0} by s\varepsilon_{0}+\kappa\rho_{0}\bar{u}^{2} and s\delta_{0} by s\delta_{0}-\frac{1}{3}\kappa\rho_{0}\bar{u}^{2} in this paper, applying a bootstrap argument to prove the smoothness. Here we omit the specific details.

    Remark 2.1. Compared to the paper [6], the new term \kappa\rho_{0}\bar{u}^{2} in the energy equality does not have a factor s , so we can not directly refer to the construction of \varphi, \psi in [6] to prove that the infimum of E over \mathcal{C} is negative for s sufficiently small. In this paper, we use the definition of \kappa_{c} (see (1.11)) and the condition \kappa < \kappa_{c} to prove the following Lemma.

    Lemma 2.2. If \kappa < \kappa_{c} , there exists a \xi\in(L^{-1}\mathbb{Z})^{2}\backslash\{0\} and a pair of functions (\widetilde{\varphi}, \widetilde{\psi})\in\mathcal{C} , such that E_{0}(\widetilde{\varphi}, \widetilde{\psi}; |\xi|) < 0 .

    Proof. From the definition (1.11) of \kappa_{c} , we find that there exists a function \widetilde{w}(x) = (\widetilde{w}_{1}(x), \widetilde{w}_{2}(x), \widetilde{w}_{3}(x))\in H_{0}^{1}(\Omega) such that \Theta_{1}(\widetilde{w}) > \kappa \Psi(\widetilde{w}) , i.e.,

    \begin{equation} \begin{split} \int_{\Omega}&(g\rho_{0}\widetilde{w}_{3}\mbox{div}\widetilde{w}-g\rho_{0}\widetilde{w}\cdot\nabla \widetilde{w}_{3})dx -\int_{\Omega}P'(\rho_{0})\rho_{0}|\mbox{div}\widetilde{w}|^{2}dx\\ &-\int_{\Sigma_{0}}\sigma|\nabla_{x_{1}, x_{2}}\widetilde{w}_{3}|^{2}dS_{0} > \int_{\Omega}\kappa\rho_{0}\bar{u}^{2}(\frac{1}{2}|\mathbb{D} (\widetilde{w})|^{2}-|\mbox{div}\widetilde{w}|^{2})dx. \end{split} \end{equation} (2.23)

    Let \widehat{w}(\xi, x_{3}) be the horizontal Fourier transform of w(x) , i.e.,

    \begin{equation*} \begin{split} \widehat{w}(\xi, x_{3}) = \int_{(2\pi L\mathbb{T})^{2}}w(x_{h}, x_{3})e^{-ix_{h}\cdot\xi}dx_{h}, \end{split} \end{equation*}

    and the functions \varphi , \theta and \psi are defined by the relations

    \begin{equation*} \begin{split} \widehat{w}_{1}(\xi, x_{3}) = -i\varphi(\xi, x_{3}), \; \widehat{w}_{2}(\xi, x_{3}) = -i\theta(\xi, x_{3}), \; \widehat{w}_{3}(\xi, x_{3}) = \psi(\xi, x_{3}), \end{split} \end{equation*}

    where (\widehat{w}_{1}, \widehat{w}_{2}, \widehat{w}_{3}) = \widehat{w} and \xi\in(L^{-1}\mathbb{Z})^{2} . Then

    \begin{equation} \begin{split} &\widehat{\nabla w} = \left( \begin{array}{lll} \xi_{1}\varphi & \xi_{2}\varphi & -i\varphi'\\ \xi_{1}\theta & \xi_{2}\theta & -i\theta'\\ i\xi_{1}\psi & i\xi_{2}\psi & \; \; \psi' \\ \end{array}\right), \end{split} \end{equation} (2.24)

    and

    \begin{equation} \begin{split} \widehat{\mbox{div}w} = \xi_{1}\varphi+\xi_{2}\theta+\psi'. \end{split} \end{equation} (2.25)

    Recalling the Fubini and Parseval theorems: in the periodic case, if f\in L^{2}(\Omega) , we have that \widehat{f}\in L^{2}(\Omega) , and

    \begin{equation} \begin{split} \int_{\Omega}|f(x)|^{2}dx = \frac{1}{4\pi^{2}L^{2}}\sum\limits_{\xi\in(L^{-1}\mathbb{Z})^{2}}\int_{-b}^{0} |\widehat{f}(\xi, x_{3})|^{2}dx_{3}. \end{split} \end{equation} (2.26)

    Using (2.24)–(2.26) and the identity 2ab = (a^{2}+b^{2})-(a-b)^{2} , we have

    \begin{equation} \begin{split} &\int_{\Omega}(g\rho_{0}\widetilde{w}_{3}\mbox{div}\widetilde{w}-g\rho_{0}\widetilde{w}\cdot\nabla \widetilde{w}_{3})dx = \frac{1}{8\pi^{2}L^{2}}\sum\limits_{\xi\in(L^{-1}\mathbb{Z})^{2}}\int_{-b}^{0}g\rho_{0} \bigg(|\widehat{\widetilde{w}}_{3}|^{2}+|\widehat{\mbox{div}\widetilde{w}}|^{2}\\ &\qquad -|\widehat{\widetilde{w}}_{3}-\widehat{\mbox{div}\widetilde{w}}|^{2}-\sum\limits_{i = 1}^{3} (|\widehat{\widetilde{w}}_{i}|^{2}+|\widehat{\partial_{i}\widetilde{w}}_{3}|^{2} -|\widehat{\widetilde{w}}_{i}-\widehat{\partial_{i}\widetilde{w}}_{3}|^{2})\bigg)dx_{3}\\ & = \frac{1}{8\pi^{2}L^{2}}\sum\limits_{\xi\in(L^{-1}\mathbb{Z})^{2}}\int_{-b}^{0}g\rho_{0} (|\widetilde{\psi}|^{2}+|\xi_{1}\widetilde{\varphi}+\xi_{2}\widetilde{\theta}+\widetilde{\psi}'|^{2} -|\widetilde{\psi}-\xi_{1}\widetilde{\varphi}-\xi_{2}\widetilde{\theta} -\widetilde{\psi}'|-|\widetilde{\varphi}|^{2}\\ &-|\widetilde{\theta}|^{2}-|\widetilde{\psi}|^{2}-|\xi|^{2}|\widetilde{\psi}|^{2}- |\widetilde{\psi}'|^{2}+|\widetilde{\varphi}+\xi_{1}\widetilde{\psi}|^{2} -|\widetilde{\theta}+\xi_{2}\widetilde{\psi}|^{2}+|\widetilde{\psi}-\widetilde{\psi}'|^{2})dx_{3}\\ & = \frac{1}{4\pi^{2}L^{2}}\sum\limits_{\xi\in(L^{-1}\mathbb{Z})^{2}\backslash \{0\}}\int_{-b}^{0}g\rho_{0} (2\xi_{1}\mathfrak{R}\widetilde{\varphi}\mathfrak{R}\widetilde{\psi} +2\xi_{1}\mathfrak{I}\widetilde{\varphi}\mathfrak{I}\widetilde{\psi} +2\xi_{2}\mathfrak{R}\widetilde{\theta}\mathfrak{R}\widetilde{\psi} +2\xi_{2}\mathfrak{I}\widetilde{\theta}\mathfrak{I}\widetilde{\psi})dx_{3}, \end{split} \end{equation} (2.27)

    here \mathfrak{R}(\widetilde{\varphi}, \widetilde{\theta}, \widetilde{\psi}) and \mathfrak{I}(\widetilde{\varphi}, \widetilde{\theta}, \widetilde{\psi}) denote the real and imaginary parts of (\widetilde{\varphi}, \widetilde{\theta}, \widetilde{\psi}) , respectively.

    By the similar argument, we can get

    \begin{equation} \begin{split} &\int_{\Omega}P'(\rho_{0})\rho_{0}|\mbox{div}\widetilde{w}|^{2}dx +\int_{\Sigma_{0}}\sigma|\nabla_{x_{1}, x_{2}}\widetilde{w}_{3}|^{2}dS_{0} +\int_{\Omega}\kappa\rho_{0}\bar{u}^{2}(\frac{1}{2}|\mathbb{D} (\widetilde{w})|^{2}-|\mbox{div}\widetilde{w}|^{2})dx\\ & = \frac{1}{4\pi^{2}L^{2}}\sum\limits_{\xi\in(L^{-1}\mathbb{Z})^{2}\backslash \{0\}}\bigg(\sigma|\xi|^{2}|\widetilde{\psi}(\xi, 0)|^{2}+\int_{-b}^{0}P'(\rho_{0})\rho_{0} |\xi_{1}\widetilde{\varphi}+\xi_{2}\widetilde{\theta}+\widetilde{\psi}'|^{2}dx_{3}\\ &\quad+\int_{-b}^{0}\kappa\rho_{0}\bar{u}^{2}(|\widetilde{\varphi}'-\xi_{1}\widetilde{\psi}|^{2}+ |\widetilde{\psi}'-\xi_{1}\widetilde{\varphi}|^{2}+\xi_{2}^{2}(|\widetilde{\varphi}|^{2} +|\widetilde{\theta}|^{2}+|\widetilde{\psi}|^{2})+\xi_{1}^{2}|\widetilde{\theta}|^{2} +|\widetilde{\theta}'|^{2}\\ &\quad -2\xi_{2}\mathfrak{R}\widetilde{\theta}'\mathfrak{R}\widetilde{\psi} -2\xi_{2}\mathfrak{I}\widetilde{\theta}'\mathfrak{I}\widetilde{\psi} -2\xi_{2}\mathfrak{R}\widetilde{\theta}\mathfrak{R}\widetilde{\psi}' -2\xi_{2}\mathfrak{I}\widetilde{\theta}\mathfrak{I}\widetilde{\psi}')dx_{3}\bigg)\\ &+\frac{1}{4\pi^{2}L^{2}}\int_{-b}^{0}P'(\rho_{0})\rho_{0}|\widetilde{\psi}'(0, x_{3})|^{2}dx_{3}+ \kappa\rho_{0}\bar{u}^{2}(|\widetilde{\varphi}'(0, x_{3})|^{2}+ |\widetilde{\psi}'(0, x_{3})|^{2}+|\widetilde{\theta}'(0, x_{3})|^{2})dx_{3}. \end{split} \end{equation} (2.28)

    Because the above equalities are invariant under simultaneous rotations of \xi and (\widetilde{\varphi}, \widetilde{\theta}) , without loss of generality we may assume that \xi = (|\xi|, 0) with |\xi| > 0 and \widetilde{\theta} = 0 . Then from (2.23), combining with (2.27)–(2.28), one can see

    \begin{equation*} \begin{split} &\sum\limits_{\xi\in(L^{-1}\mathbb{Z})^{2}\backslash \{0\}}\int_{-b}^{0}2g\rho_{0}|\xi| (\mathfrak{R}\widetilde{\varphi}\mathfrak{R}\widetilde{\psi} +\mathfrak{I}\widetilde{\varphi}\mathfrak{I}\widetilde{\psi})dx_{3}\\ & > \sum\limits_{\xi\in(L^{-1}\mathbb{Z})^{2}\backslash \{0\}}\bigg(\sigma|\xi|^{2}|\widetilde{\psi}(\xi, 0)|^{2}+\int_{-b}^{0}P'(\rho_{0})\rho_{0} ||\xi|\widetilde{\varphi}+\widetilde{\psi}'|^{2}dx_{3}\\ &\quad\qquad+\int_{-b}^{0}\kappa\rho_{0}\bar{u}^{2}(|\widetilde{\varphi}'-|\xi|\widetilde{\psi}|^{2}+ |\widetilde{\psi}'-|\xi|\widetilde{\varphi}|^{2})dx_{3}\bigg)\\ &\quad+\int_{-b}^{0}P'(\rho_{0})\rho_{0}|\widetilde{\psi}'(0, x_{3})|^{2}dx_{3}+ \kappa\rho_{0}\bar{u}^{2}(|\widetilde{\varphi}'(0, x_{3})|^{2}+ |\widetilde{\psi}'(0, x_{3})|^{2})dx_{3}, \end{split} \end{equation*}

    which follows that

    \begin{equation} \begin{split} &\sum\limits_{\xi\in(L^{-1}\mathbb{Z})^{2}\backslash \{0\}}\int_{-b}^{0}g\rho_{0}|\xi| (|\widetilde{\varphi}|^{2}+|\widetilde{\psi}|^{2}-|\widetilde{\varphi}-\widetilde{\psi}|^{2})dx_{3}\\ & > \sum\limits_{\xi\in(L^{-1}\mathbb{Z})^{2}\backslash \{0\}}\bigg(\sigma|\xi|^{2}|\widetilde{\psi}(\xi, 0)|^{2}+\int_{-b}^{0}P'(\rho_{0})\rho_{0} ||\xi|\widetilde{\varphi}+\widetilde{\psi}'|^{2}dx_{3}\\ &\quad\qquad+\int_{-b}^{0}\kappa\rho_{0}\bar{u}^{2}(|\widetilde{\varphi}'-|\xi|\widetilde{\psi}|^{2}+ |\widetilde{\psi}'-|\xi|\widetilde{\varphi}|^{2})dx_{3}\bigg). \end{split} \end{equation} (2.29)

    (2.29) implies that there is a \xi\in(L^{-1}\mathbb{Z})^{2}\backslash \{0\} , such that

    \begin{equation*} \begin{split} &\sigma|\xi|^{2}|\widetilde{\psi}(\xi, 0)|^{2}+\int_{-b}^{0}P'(\rho_{0})\rho_{0} ||\xi|\widetilde{\varphi}+\widetilde{\psi}'|^{2} -g\rho_{0}|\xi|(|\widetilde{\varphi}|^{2}+|\widetilde{\psi}|^{2} -|\widetilde{\varphi}-\widetilde{\psi}|^{2})dx_{3}\\ &\quad\qquad+\int_{-b}^{0}\kappa\rho_{0}\bar{u}^{2}(|\widetilde{\varphi}'-|\xi|\widetilde{\psi}|^{2}+ |\widetilde{\psi}'-|\xi|\widetilde{\varphi}|^{2})dx_{3} < 0. \end{split} \end{equation*}

    Then we complete the proof of Lemma 2.2.

    Remark 2.2. Owning to Lemma 2.2, there is an unstable frequency \xi\in(L^{-1}\mathbb{Z})^{2}\backslash \{0\} and a pair of functions (\widetilde{\varphi}, \widetilde{\psi})\in H_{0}^{1}((-b, 0))\times H_{0}^{1}((-b, 0)) satisfying E_{0}(\widetilde{\varphi}, \widetilde{\psi}; |\xi|) < 0 , if \kappa < \kappa_{c} . Thus, the unstable frequency-set \mathbb{F} consisted of all unstable frequencies is not empty for \kappa < \kappa_{c} .

    We want to show that there is a fixed point s so that \lambda(|\xi|, s) = s , which will then allow us to construct a solution to the original problem (2.8)–(2.9). To this end, we study the behavior of \alpha(s): = \alpha(|\xi|; s) as a function of s > 0 .

    Lemma 2.3. Let \alpha(s):(0, \infty)\rightarrow \mathbb{R} be defined by (2.20). Then the following hold.

    (1) \alpha(s)\in C_{loc}^{0, 1}((0, \infty)) and \alpha(s) is strictly increasing in s .

    (2) For any \xi\in\mathbb{F} , there exist constants c_{2}, c_{3} > 0 depending on g, \kappa, \rho_{0}, \varepsilon_{0}, \delta_{0}, \sigma and |\xi| , such that

    \begin{equation} \begin{split} \alpha(s)\leq -c_{2}+sc_{3}, \; \; {{for\; any}}\; s\in(0, \infty). \end{split} \end{equation} (2.30)

    Proof. (1) Fix a compact interval Q = [a, b]\subset\subset(0, \infty) . From (2.18), E_{1}(\varphi, \psi; |\xi|)\geq0 implies that E(\varphi, \psi; |\xi|, s) is non-decreasing in s with (\varphi, \psi)\in\mathcal{C} kept fixed. And, from Lemma 2.1, we can find a pair (\varphi_{s}, \psi_{s})\in\mathcal{C} so that

    \begin{equation} \begin{split} E(\varphi_{s}, \psi_{s};|\xi|, s) = \inf\limits_{(\varphi, \psi)\in\mathcal{C}} E(\varphi, \psi;|\xi|, s) = \alpha(s), \; \mbox{for fixed}\; s > 0. \end{split} \end{equation} (2.31)

    Note that if 0 < s_{1} < s_{2} < \infty , then the decomposition (2.18) and (2.31) implies that

    \begin{equation} \begin{split} \alpha(s_{1}) = E(\varphi_{s_{1}}, \psi_{s_{1}};|\xi|, s_{1})\leq E(\varphi_{s_{2}}, \psi_{s_{2}};|\xi|, s_{1})\leq E(\varphi_{s_{2}}, \psi_{s_{2}};|\xi|, s_{2}) = \alpha(s_{2}), \end{split} \end{equation} (2.32)

    which shows that \alpha(s) is non-decreasing in s . Supposed that \alpha(s_{1}) = \alpha(s_{2}) , from (2.32), we can obtain

    \begin{equation*} \begin{split} s_{1}E_{1}(\varphi_{s_{2}}, \psi_{s_{2}};|\xi|) = s_{2}E_{1}(\varphi_{s_{2}}, \psi_{s_{2}};|\xi|), \end{split} \end{equation*}

    which yields that E_{1}(\varphi_{s_{2}}, \psi_{s_{2}}; |\xi|) = 0 . This means that \varphi_{s_{2}} = \psi_{s_{2}} = 0 , which contradicts the fact that (\varphi_{s_{2}}, \psi_{s_{2}})\in\mathcal{C} . Then we can get that \alpha(s_{1}) < \alpha(s_{2}) , i.e., \alpha(s) is strictly increasing in s .

    Nextly, we show the continuity of \alpha(s) . Fixed any pair (\varphi_{0}, \psi_{0})\in\mathcal{C} , the fact that E(\varphi, \psi; |\xi|, s) is non-decreasing in s , the minimality of (\varphi_{s}, \psi_{s}) , and the equality (2.19) ensure that

    \begin{equation*} \begin{split} E(\varphi_{0}, \psi_{0};|\xi|, b)\geq E(\varphi_{0}, \psi_{0};|\xi|, s)\geq sE_{1}(\varphi_{s}, \psi_{s};|\xi|) -g|\xi|, \; \; \forall\; s\in Q, \end{split} \end{equation*}

    which implies that there exists a constant 0 < K = K(a, b, \varphi_{0}, \psi_{0}, g, |\xi|) < \infty so that

    \begin{equation} \begin{split} \sup\limits_{s\in Q}E_{1}(\varphi_{s}, \psi_{s};|\xi|)\leq K. \end{split} \end{equation} (2.33)

    Let s_{i}\in Q for i = 1, 2 . Using (2.31) and (2.33), we can see

    \begin{equation*} \begin{split} \alpha(s_{1})\leq E(\varphi_{s_{2}}, \psi_{s_{2}};|\xi|, s_{1})&\leq E(\varphi_{s_{2}}, \psi_{s_{2}};|\xi|, s_{2})+|s_{1}-s_{2}|E(\varphi_{s_{2}}, \psi_{s_{2}};|\xi|)\\ &\leq \alpha(s_{2})+K|s_{1}-s_{2}|. \end{split} \end{equation*}

    Reversing the role of the indices 1 and 2 in the derivation of this inequality gives the same bound with the indices switched, i.e.,

    \alpha(s_{2})-\alpha(s_{1})\leq K|s_{1}-s_{2}|.

    The above two inequalities ensure that

    \begin{equation*} \begin{split} |\alpha(s_{1})-\alpha(s_{2})|\leq K|s_{1}-s_{2}|, \end{split} \end{equation*}

    which proves \alpha(s)\in C_{\mbox{ loc}}^{0, 1}((0, \infty)) .

    (2) Since \xi\in\mathbb{F} , by the definition of \mathbb{F} and Lemma 2.2, there exists (\widetilde{\varphi}, \widetilde{\psi})\in H_{0}^{1}((-b, 0))\times H_{0}^{1}((-b, 0)) , such that

    \begin{equation*} \begin{split} &E_{1}(\widetilde{\varphi}, \widetilde{\psi};|\xi|) = \frac{\sigma|\xi|^{2}|\widetilde{\psi}(\xi, 0)|^{2}}{2} +\frac{1}{2}\int_{-b}^{0}P'(\rho_{0})\rho_{0} ||\xi|\widetilde{\varphi}+\widetilde{\psi}'|^{2}-g\rho_{0}|\xi|(|\widetilde{\varphi}|^{2}+|\widetilde{\psi}|^{2} -|\widetilde{\varphi}-\widetilde{\psi}|^{2})dx_{3} \\ &\quad\qquad\qquad\qquad+\frac{1}{2}\int_{-b}^{0}\kappa\rho_{0}\bar{u}^{2} (|\widetilde{\varphi}'-|\xi|\widetilde{\psi}|^{2}+ |\widetilde{\psi}'-|\xi|\widetilde{\varphi}|^{2})dx_{3} < 0. \end{split} \end{equation*}

    Thus, one can see

    \begin{equation} \begin{split} \alpha(s)& = \inf\limits_{(\varphi, \psi)\in\mathcal{C}}E(\varphi, \psi;|\xi|, s) = \inf\limits_{(\varphi, \psi)\in H_{0}^{1}((-b, 0))\times H_{0}^{1}((-b, 0))}\frac{E(\varphi, \psi;|\xi|, s)}{J(\varphi, \psi)}\\ &\leq\frac{E(\widetilde{\varphi}, \widetilde{\psi};|\xi|, s)}{J(\widetilde{\varphi}, \widetilde{\psi})} = \frac{E_{1}(\widetilde{\varphi}, \widetilde{\psi};|\xi|)}{J(\widetilde{\varphi}, \widetilde{\psi})}+ s\frac{E_{0}(\widetilde{\varphi}, \widetilde{\psi};|\xi|)}{J(\widetilde{\varphi}, \widetilde{\psi})} = :-c_{2}+sc_{3}. \end{split} \end{equation} (2.34)

    Then, we complete the proof of Lemma 2.3.

    Given \xi\in\mathbb{F} , from (2.30), there exists a s_{0} > 0 depending on the quantities g, \kappa, \rho_{0}, \varepsilon_{0}, \delta_{0}, |\xi| so that \alpha(|\xi|; s) < 0 for any s\in(0, s_{0}] . Now, we define

    \begin{equation*} \begin{split} S\overset{\mbox{def}}{ = }\sup\{s|\; \alpha(\tau) < 0\; \mbox{for any}\; \tau\in(0, s)\}. \end{split} \end{equation*}

    Then S > 0 . Applying the monotonicity of \alpha(s) and the fact \alpha(s) = \inf_{(\varphi, \psi)\in\mathcal{C}}E(\varphi, \psi; |\xi|, s) > -\infty , one can see that

    \begin{equation*} \begin{split} \lim\limits_{s\rightarrow 0^{+}}\alpha(s)\; \mbox{exists and the limit is a negative constant}. \end{split} \end{equation*}

    Using (2.19), we find that there exists a constant c_{4} depends on \varepsilon_{0}, \delta_{0}, |\xi| , such that

    \begin{equation*} \begin{split} \alpha(s)\geq-g|\xi|+sc_{4}, \end{split} \end{equation*}

    where \xi\in\mathbb{F} . Thus, if s\geq\frac{g|\xi|}{c_{4}} , then \alpha(s)\geq0 . Hence, S < +\infty , and \lim_{s\rightarrow S^{-}}\alpha(s) = 0 .

    Therefore, combining with the above argument, we can employ a fixed-point argument to obtain the following Lemma.

    Lemma 2.4. Let \xi\in\mathbb{F} . Then there exists a unique s\in (0, S) so that \lambda(|\xi|; s) = \sqrt{-\alpha(|\xi|; s)} > 0 and

    \begin{equation} \begin{split} s = \lambda(|\xi|;s). \end{split} \end{equation} (2.35)

    Remark 2.3. By Lemma 2.4, for each fixed \xi\in\mathbb{F} , we can find a unique s\in (0, S) so that s = \lambda(|\xi|; s) . Then, we can write s = s(|\xi|) and \lambda = \lambda(|\xi|) .

    Proposition 2.1. For \xi\in (L^{-1}\mathbb{Z})^{2} , if \kappa < \kappa_{c} , there exists a solution \varphi = \varphi(\xi, x_{3}), \; \theta = \theta(\xi, x_{3}), \; \psi = \psi(\xi, x_{3}) , and \lambda = \lambda(|\xi|) > 0 to (2.4)–(2.6) satisfying the boundary conditions. The solutions are smooth in (-b, 0) , and they are equivariant in \xi in the sense that if R\in SO(2) is a rotation operator, then

    \begin{equation*} \begin{split} \left( \begin{array}{lll} \varphi(R\xi, x_{3})\\ \theta(R\xi, x_{3})\\ \psi(R\xi, x_{3}) \\ \end{array}\right) = \left( \begin{array}{lll} R_{11} & R_{12}& 0\\ R_{21}& R_{22} &0\\ 0 &0& 1 \\ \end{array}\right) \left( \begin{array}{lll} \varphi(\xi, x_{3})\\ \theta(\xi, x_{3})\\ \psi(\xi, x_{3}) \\ \end{array}\right). \end{split} \end{equation*}

    Proof. We may find a rotation operator R\in SO(2) so that R\xi = (|\xi|, 0) . For s = s(|\xi|) given in (2.35), we define (\varphi(\xi, x_{3}), \theta(\xi, x_{3})) = R^{-1}(\varphi(|\xi|, x_{3}), 0) and \psi(\xi, x_{3})) = \psi(|\xi|, x_{3}) , where the functions \varphi(|\xi|, x_{3}) and \psi(|\xi|, x_{3}) are the solutions to (2.8) and (2.9) from Lemma 2.1. This gives a solution to (2.4)–(2.6). The equivariance in \xi follows from the definition.

    To obtain a largest growth rate, we next show that \lambda(|\xi|) is a bounded, continuous function of |\xi| . We assume throughout that \xi\in \mathbb{F} .

    Lemma 2.5. The function \xi\in\mathbb{F}\longmapsto\lambda(|\xi|)\in(0, \infty) is bounded, continuous, and satisfies

    \begin{equation} \begin{split} \lim\limits_{|\xi|\rightarrow 0}\lambda(|\xi|) = 0. \end{split} \end{equation} (2.36)

    Proof. Since \lambda = \sqrt{-\alpha} , it suffices to prove the continuity of \alpha(|\xi|) . By Lemma 2.1, for every \xi\in\mathbb{F} , there exist functions (\varphi_{|\xi|}, \psi_{|\xi|})\in\mathcal{C} satisfying (2.13) and (2.14) so that \alpha(|\xi|; s) = E(\varphi_{|\xi|}, \psi_{|\xi|}; |\xi|, s) . From Lemma 2.2, we know that \alpha(|\xi|; s) < 0 , which along with (2.19), yields the bound

    \begin{equation*} \begin{split} -g|\xi|\leq \alpha(|\xi|;s) < 0, \end{split} \end{equation*}

    which ensures that \lambda(|\xi|)\in(0, \infty) ( \xi\in\mathbb{F} ) is bounded and satisfies (2.36). Now suppose |\xi|_{n} is a sequence so that |\xi|_{n}\rightarrow |\xi| ( \xi\in\mathbb{F} ). By (2.34), there exist positive constants C_{0}, C_{1} such that \alpha(|\xi|_{n})\leq-C_{0}+s(|\xi|_{n})C_{1} , combining with -\alpha(|\xi|_{n}) = \lambda^{2}(|\xi|_{n}) = s^{2}(|\xi|_{n}) , so

    \begin{equation*} \begin{split} s^{2}(|\xi|_{n})+C_{1}s(|\xi|_{n})-C_{0}\geq 0, \end{split} \end{equation*}

    which implies s(|\xi|_{n}) is bounded below by a positive constant. Then (\varphi_{|\xi|_{n}}, \psi_{|\xi|_{n}})\in\mathcal{C} and the fact that

    \begin{equation*} \begin{split} &-g|\xi|_{n}+\frac{s(|\xi|_{n})}{2}\int_{-b}^{0}\bigg((\delta_{0}+\frac{1}{3}\varepsilon_{0}) (\psi'_{|\xi|_{n}}+|\xi|_{n}\varphi_{|\xi|_{n}})^{2}\\ &\qquad\qquad+\varepsilon_{0}((\varphi'_{|\xi|_{n}}-|\xi|_{n}\psi_{|\xi|_{n}})^{2} +(\psi'_{|\xi|_{n}}-|\xi|_{n}\varphi_{|\xi|_{n}})^{2})\bigg)dx_{3}\leq\alpha(|\xi|_{n}) < 0 \end{split} \end{equation*}

    imply that \varphi_{|\xi|_{n}} and \psi_{|\xi|_{n}} are uniformly bounded in H^{1}((-b, 0)) . Plugging into the ODE (2.21)–(2.22) in (-b, 0) , we find that \varphi_{|\xi|_{n}} and \psi_{|\xi|_{n}} are uniformly bounded in H^{2}((-b, 0)) . Then there exists a subsequence(still denoted by |\xi|_{n} ) such that

    \begin{equation*} \begin{split} (\varphi_{|\xi|_{n}}, \psi_{|\xi|_{n}})\rightarrow (\varphi_{|\xi|}, \psi_{|\xi|})\; \mbox{strongly in}\; H^{1}((-b, 0)), \end{split} \end{equation*}

    which implies

    \begin{equation*} \begin{split} \alpha(|\xi|_{n}) = E(\varphi_{|\xi|_{n}}, \psi_{|\xi|_{n}})\rightarrow E(\varphi_{|\xi|}, \psi_{|\xi|}) = \alpha(|\xi|). \end{split} \end{equation*}

    i.e., \alpha(|\xi|_{n})\rightarrow \alpha(|\xi|) , hence \alpha(|\xi|) is continuous.

    Lemma 2.5 then allows us to define

    \begin{equation} \begin{split} 0 < \Lambda: = \sup\limits_{\xi\in\mathbb{F}}\lambda(|\xi|) < \infty. \end{split} \end{equation} (2.37)

    Lemma 2.6. Suppose \xi\in \mathbb{F} . Let (\varphi, \theta, \psi) be the solutions constructed in Proposition 2.1. Then for each k\geq 0 there exists a constant A_{k} > 0 depending on the parameters \rho_{0}, P, g, \varepsilon_{0}, \delta_{0}, \sigma, b ,

    \begin{equation*} \begin{split} \|\varphi(\xi, \cdot)\|_{H^{k}((-b, 0))}+\|\theta(\xi, \cdot)\|_{H^{k}((-b, 0))}+\|\psi(\xi, \cdot)\|_{H^{k}((-b, 0))}\leq A_{k}. \end{split} \end{equation*}

    Proof. From Proposition 2.1, it suffices to prove

    \begin{equation*} \begin{split} \|\varphi(|\xi|, \cdot)\|_{H^{k}((-b, 0))}+\|\psi(|\xi|, \cdot)\|_{H^{k}((-b, 0))}\leq A_{k}. \end{split} \end{equation*}

    for the solutions \varphi = \varphi(|\xi|, x_{3}) , \psi = \psi(|\xi|, x_{3}) constructed in Proposition 2.1. By (1.5), we can see that \rho_{0} , P'(\rho_{0}) , \kappa\rho_{0}\bar{u}^{2} , \varepsilon_{0} and \delta_{0} are smooth in (-b, 0) and bounded above and below.

    We prove this lemma by induction on k . For k = 0 , the fact that (\varphi(|\xi|, \cdot), \psi(|\xi|, \cdot))\in\mathcal{C} implies that there exists a constant A_{0} > 0 depending on the various parameters so that

    \begin{equation*} \begin{split} \|\varphi(|\xi|, \cdot)\|_{L^{2}((-b, 0))}+\|\psi(|\xi|, \cdot)\|_{L^{2}((-b, 0))}\leq A_{0}. \end{split} \end{equation*}

    Suppose now that the bound holds some k\geq0 , i.e.,

    \begin{equation*} \begin{split} \|\varphi(|\xi|, \cdot)\|_{H^{k}((-b, 0))}+\|\psi(|\xi|, \cdot)\|_{H^{k}((-b, 0))}\leq A_{k}. \end{split} \end{equation*}

    From Lemmas 2.4 and 2.5, \lambda(|\xi|) = s(|\xi|) is bounded above and below by positive quantities as functions of |\xi| . Then, we differentiate the Eqs (2.13) and (2.14) to get that there exists a constant C > 0 depending on the various parameters so that

    \begin{equation*} \begin{split} \|\varphi(|\xi|, \cdot)&\|_{H^{k+1}((-b, 0))}+\|\psi(|\xi|, \cdot)\|_{H^{k+1}((-b, 0))}\\ &\leq C(\|\varphi(|\xi|, \cdot)\|_{H^{k}((-b, 0))}+\|\psi(|\xi|, \cdot)\|_{H^{k}((-b, 0))})\leq CA_{k}: = A_{k+1}, \end{split} \end{equation*}

    which shows that the bound holds for k+1 . Then, by induction the bound holds for all k\geq0 .

    We may now construct a growing mode solution to the linearized problem (1.8).

    Proposition 2.2. Let \xi_{1}, \xi_{2}\in (L^{-1}\mathbb{Z})^{2} be lattice points such that \xi_{1} = -\xi_{2} and \lambda(|\xi_{j}|) = \Lambda , for j = 1, 2 , where \Lambda is defined by (2.37). Define

    \begin{equation*} \label{EB-multi-2.76} \begin{split} \widehat{w}(\xi, x_{3}) = -i\varphi(\xi, x_{3})e_{1}-i\theta(\xi, x_{3})e_{2}+\psi(\xi, x_{3})e_{3}, \end{split} \end{equation*}

    where \varphi , \theta , \psi are the solutions provided by Proposition 2.1. Writing x' = x_{1}e_{1}+x_{2}e_{2} , we define

    \begin{equation*} \label{EB-multi-2.77} \begin{split} \eta(x, t) = e^{\Lambda t}\sum\limits_{j = 1}^{2}\widehat{w}(\xi_{j}, x_{3})e^{ix'\cdot\xi_{j}}, \end{split} \end{equation*}
    \begin{equation*} \label{EB-multi-2.78} \begin{split} u(x, t) = \Lambda e^{\Lambda t}\sum\limits_{j = 1}^{2}\widehat{w}(\xi_{j}, x_{3})e^{ix'\cdot\xi_{j}}, \end{split} \end{equation*}
    \begin{equation*} \label{EB-multi-2.79} \begin{split} q(x, t) = -e^{\Lambda t}\rho_{0}(x_{3})\sum\limits_{j = 1}^{2}(\xi_{1}\varphi(\xi_{j}, x_{3})+ \xi_{2}\theta(\xi_{j}, x_{3})+\partial_{3}\psi(\xi_{j}, x_{3}))e^{ix'\cdot\xi_{j}}, \end{split} \end{equation*}

    and

    \begin{equation*} \label{EB-multi-2.80} \begin{split} V(x, t) = e^{\Lambda t}\sum\limits_{j = 1}^{2}B(\xi_{j}, x_{3})\bar{U}e^{ix'\cdot\xi_{j}}, \end{split} \end{equation*}

    where

    \begin{equation*} \label{EB-multi-2.81} \begin{split} &B(\xi_{j}, x_{3}) = \left( \begin{array}{lll} \xi_{1}\varphi & \xi_{2}\varphi & -i\partial_{3}\varphi\\ \xi_{1}\theta & \xi_{2}\theta & -i\partial_{3}\theta\\ i\xi_{1}\psi & i\xi_{2}\psi & \; \; \partial_{3}\psi \\ \end{array}\right). \end{split} \end{equation*}

    Then \eta, u, q, V are real solutions to (1.8). For every t\geq 0 , we have \eta(t), u(t), q(t), V(t)\in H^{k}(\Omega) and

    \begin{equation} \begin{split} &\|q(t)\|_{H^{k}(\Omega)} = e^{\Lambda t}\|q(0)\|_{H^{k}(\Omega)}, \; \; \|u(t)\|_{H^{k}(\Omega)} = e^{\Lambda t}\|u(0)\|_{H^{k}(\Omega)}, \\ &\|V(t)\|_{H^{k}(\Omega)} = e^{\Lambda t}\|V(0)\|_{H^{k}(\Omega)}, \; \; \|\eta(t)\|_{H^{k}(\Omega)} = e^{\Lambda t}\|\eta(0)\|_{H^{k}(\Omega)}. \end{split} \end{equation} (2.38)

    Proof. By direct calculation, \eta, u, q, V defined in this way are solutions to (1.8). That they are real-valued follows from the equivariance in \xi stated in Proposition 2.1. From Lemma 2.6, the solutions are in H^{k}(\Omega) at t = 0 . The definitions of \eta, u, q, V ensure the growth in time stated in (2.38).

    In this section we will prove estimates for the growth in time of arbitrary solutions to (1.8) in terms of the largest growing mode \Lambda defined by (2.37). Firstly, we differentiate (1.8)_{3} in time and eliminate the q , \eta and V terms using the other equations. This yields the equation

    \begin{equation} \begin{split} &\rho_{0}\partial_{tt}u-\nabla(P'(\rho_{0})\rho_{0}\mbox{div}u)+g\rho_{0}\nabla u_{3}-g\rho_{0}\mbox{div}u e_{3}\\ & = \mbox{div}(\varepsilon_{0}(\mathbb{D}(\partial_{t}u)-\frac{2}{3}(\mbox{div}\partial_{t}u)\mathbb{I}) +\delta_{0}(\mbox{div}\partial_{t}u)\mathbb{I}+\kappa\rho_{0}\bar{u}^{2}(\mathbb{D}(u)-(\mbox{div}u)\mathbb{I}), \end{split} \end{equation} (3.1)

    coupled to the boundary conditions:

    \begin{equation*} \label{EB-multi-3.2} \begin{split} (P'(\rho_{0})\rho_{0}\mbox{div}u\mathbb{I}+&\varepsilon_{0}\mathbb{D}(\partial_{t}u) +(\delta_{0}-\frac{2}{3}\varepsilon_{0})\mbox{div}\partial_{t}u\mathbb{I}\\ &+\kappa\rho_{0}\bar{u}^{2}(\mathbb{D}(u)-\mbox{div}u\mathbb{I}))e_{3} = \sigma\Delta_{x_{1}, x_{2}}u_{3}e_{3}, \; \mbox{on}\; \Sigma_{0}, \end{split} \end{equation*}

    and \partial_{t}u(x_{1}, x_{2}, -b, t) = 0 .

    First, we state the following energy identity.

    Lemma 2.7. Let u solve (3.1) and the corresponding boundary conditions. Then

    \begin{equation} \begin{split} &\partial_{t}\int_{\Omega}\rho_{0}\frac{|\partial_{t}u|^{2}}{2}dx +\int_{\Omega}\frac{\varepsilon_{0}}{2}\bigg|\mathbb{D}(\partial_{t}u)-\frac{2}{3} (\mathit{\mbox{div}}\partial_{t}u)\mathbb{I}\bigg|^{2}+\delta_{0}|\mathit{\mbox{div}}\partial_{t}u|^{2}dx \\ & = \partial_{t}\int_{\Omega}(-\frac{P'(\rho_{0})\rho_{0}}{2}|\mathit{\mbox{div}}u|^{2}+g\rho_{0}u_{3}\mathit{\mbox{div}}u +\frac{g\rho_{0}'}{2}|u_{3}|^{2}-\frac{\kappa\rho_{0}\bar{u}^{2}}{2} (\frac{1}{2}|\mathbb{D}(u)|^{2}-|\mathit{\mbox{div}}u|^{2}))dx\\ &\qquad-\partial_{t}\int_{\Sigma_{0}}\frac{g\rho_{0}}{2}|u_{3}|^{2}dS_{0} -\partial_{t}\int_{\Sigma_{0}}\frac{\sigma}{2}|\nabla_{x_{1}, x_{2}}u_{3}|^{2}dS_{0}. \end{split} \end{equation} (3.2)

    Proof. Take the dot product of (3.1) with \partial_{t}u and integrate over \Omega . After integrating by parts, we get

    \begin{equation*} \label{EB-multi-3.4} \begin{split} &\int_{\Omega}\rho_{0}\partial_{t}u\cdot\partial_{tt}u+P'(\rho_{0})\rho_{0}(\mbox{div}\, u) (\mbox{div}\, \partial_{t}u)-g\rho_{0}(u_{3}\mbox{div}\, \partial_{t}u+\partial_{t}u_{3}\mbox{div}\, u)\\ &+\int_{\Omega}\frac{\varepsilon_{0}}{2}\bigg|\mathbb{D}(\partial_{t}u)-\frac{2}{3}dx (\mbox{div}\partial_{t}u)\mathbb{I}\bigg|^{2}+\delta_{0}|\mbox{div}\partial_{t}u|^{2}dx -\int_{\Omega}g\rho_{0}'u_{3}\partial_{t}u_{3}dx\\ &+\int_{\Omega}\kappa\rho_{0}\bar{u}^{2}(\mathbb{D}(u):\nabla\partial_{t}u-(\mbox{div}\, u) (\mbox{div}\, \partial_{t}u)) = -\int_{\Sigma_{0}}g\rho_{0}u_{3}\partial_{t}u_{3}dS_{0}+\int_{\Sigma_{0}}Te_{3}\cdot\partial_{t}udS_{0} \end{split} \end{equation*}

    where

    \begin{equation*} \label{EB-multi-3.5} \begin{split} T = &(P'(\rho_{0})\rho_{0}\mbox{div}\, u)\mathbb{I}+\varepsilon_{0}(\mathbb{D}(\partial_{t}u) -\frac{2}{3}(\mbox{div}\, \partial_{t}u)\mathbb{I})+\delta_{0}\mbox{div}\, \partial_{t}u\mathbb{I} +\kappa\rho_{0}\bar{u}^{2}(\mathbb{D}(u)-\mbox{div}\, u\mathbb{I}). \end{split} \end{equation*}

    We may pull time derivatives out of the first integrals on each side of the equation to arrive at the equality

    \begin{equation} \begin{split} &\partial_{t}\int_{\Omega}\rho_{0}\frac{|\partial_{t}u|^{2}}{2}dx +\int_{\Omega}\frac{\varepsilon_{0}}{2}\bigg|\mathbb{D}(\partial_{t}u)-\frac{2}{3} (\mbox{div}\partial_{t}u)\mathbb{I}\bigg|^{2}+\delta_{0}|\mbox{div}\partial_{t}u|^{2}dx \\ & = \partial_{t}\int_{\Omega}(-\frac{P'(\rho_{0})\rho_{0}}{2}|\mbox{div}u|^{2}+g\rho_{0}u_{3}\mbox{div}u +\frac{g\rho_{0}'}{2}|u_{3}|^{2}-\frac{\kappa\rho_{0}\bar{u}^{2}}{2} (\frac{1}{2}|\mathbb{D}(u)|^{2}-|\mbox{div}u|^{2}))dx\\ &\qquad-\partial_{t}\int_{\Sigma_{0}}\frac{g\rho_{0}}{2}|u_{3}|^{2}dS_{0} +\int_{\Sigma_{0}}Te_{3}\cdot\partial_{t}udS_{0}. \end{split} \end{equation} (3.3)

    Using the upper boundary condition, one can see

    \begin{equation} \begin{split} \int_{\Sigma_{0}}Te_{3}\cdot\partial_{t}udS_{0}& = \int_{\Sigma_{0}}\sigma\Delta_{x_{1}, x_{2}}u_{3} \partial_{t}u_{3}dS_{0}\\ & = -\sigma\int_{\Sigma_{0}}\nabla_{x_{1}, x_{2}}u_{3}\cdot \nabla_{x_{1}, x_{2}}\partial_{t}u_{3}dS_{0} = -\partial_{t}\int_{\Sigma_{0}}\frac{\sigma}{2} |\nabla_{x_{1}, x_{2}}u_{3}|^{2}dS_{0}. \end{split} \end{equation} (3.4)

    Combining with (3.3) and (3.4), we can obtain (3.2).

    Lemma 2.8. Let v\in H^{1}_{0}(\Omega) be arbitrary. We have the inequality

    \begin{equation} \begin{split} &\int_{\Omega}(-\frac{P'(\rho_{0})\rho_{0}}{2}|\mathit{\mbox{div}}\, v|^{2}+g\rho_{0}v_{3}\, \mathit{\mbox{div}}\, v +\frac{g\rho_{0}'}{2}|v_{3}|^{2}-\frac{\kappa\rho_{0}\bar{u}^{2}}{2} (\frac{1}{2}|\mathbb{D}(v)|^{2}-|\mathit{\mbox{div}}\, v|^{2}))dx\\ &\qquad-\int_{\Sigma_{0}}\frac{g\rho_{0}}{2}|v_{3}|^{2}dS_{0} -\int_{\Sigma_{0}}\frac{\sigma}{2}|\nabla_{x_{1}, x_{2}}v_{3}|^{2}dS_{0}\\ &\leq\frac{\Lambda^{2}}{2}\int_{\Omega}\rho_{0}|v|^{2}dx+\frac{\Lambda}{2}\int_{\Omega} \frac{\varepsilon_{0}}{2}\bigg|\nabla v+\nabla v^{T}-\frac{2}{3} (\mathit{\mbox{div}}\, v)\mathbb{I}\bigg|^{2}+\delta_{0}|\mathit{\mbox{div}}\, v|^{2}dx. \end{split} \end{equation} (3.5)

    Proof. Integrating by parts, we can get

    \begin{equation} \begin{split} &\int_{\Omega}(-\frac{P'(\rho_{0})\rho_{0}}{2}|\mbox{div}\, v|^{2}+g\rho_{0}v_{3}\mbox{div}\, v +\frac{g\rho_{0}'}{2}|v_{3}|^{2}-\frac{\kappa\rho_{0}\bar{u}^{2}}{2} (\frac{1}{2}|\mathbb{D}(v)|^{2}-|\mbox{div}\, v|^{2}))dx\\ &\qquad-\int_{\Sigma_{0}}\frac{g\rho_{0}}{2}|v_{3}|^{2}dS_{0} -\int_{\Sigma_{0}}\frac{\sigma}{2}|\nabla_{x_{1}, x_{2}}v_{3}|^{2}dS_{0}\\ & = \frac{1}{2}\int_{\Omega}g\rho_{0}(v_{3}\mbox{div}\, v-\nabla v_{3}\cdot v)-P'(\rho_{0})\rho_{0}|\mbox{div}\, v|^{2} -\kappa\rho_{0}\bar{u}^{2}(\frac{1}{2}|\mathbb{D}(v)|^{2}-|\mbox{div}\, v|^{2})dx\\ &\qquad -\int_{\Sigma_{0}}\frac{\sigma}{2}|\nabla_{x_{1}, x_{2}}v_{3}|^{2}dS_{0}. \end{split} \end{equation} (3.6)

    Firstly, we consider all v\in H_{0}^{1} satisfying (2.23). Writing \varphi(x_{3}) = i\, \widehat{v}_{1}, \; \theta(x_{3}) = i\, \widehat{v}_{2}, \; \psi(x_{3}) = \widehat{v}_{3} , by the similar argument as the proof of Lemma 2.2, we take the horizontal Fourier transform to see that

    \begin{equation*} \begin{split} &\frac{1}{2}\int_{\Omega}g\rho_{0}(v_{3}\mbox{div}\, v-\nabla v_{3}\cdot v)-P'(\rho_{0})\rho_{0}|\mbox{div}\, v|^{2} -\kappa\rho_{0}\bar{u}^{2}(\frac{1}{2}|\mathbb{D}(v)|^{2}-|\mbox{div}\, v|^{2})dx\\ &\qquad -\int_{\Sigma_{0}}\frac{\sigma}{2}|\nabla_{x_{1}, x_{2}}v_{3}|^{2}dS_{0} \end{split} \end{equation*}
    \begin{equation*} \begin{split} & = \frac{1}{4\pi^{2}L^{2}}\sum\limits_{\xi\in(L^{-1}\mathbb{Z})^{2}\backslash \{0\}} \frac{1}{2}\bigg(\int_{-b}^{0}g\rho_{0} (2\xi_{1}\mathfrak{R}\varphi\mathfrak{R}\psi +2\xi_{1}\mathfrak{I}\varphi\mathfrak{I}\psi +2\xi_{2}\mathfrak{R}\theta\mathfrak{R}\psi +2\xi_{2}\mathfrak{I}\theta\mathfrak{I}\psi)dx_{3}\\ &\quad-\sigma|\xi|^{2}|\psi(\xi, 0)|^{2}-\int_{-b}^{0}P'(\rho_{0})\rho_{0} |\xi_{1}\varphi+\xi_{2}\theta+\psi'|^{2}dx_{3} \end{split} \end{equation*}
    \begin{equation*} \label{EB-multi-3.10} \begin{split} &\quad-\int_{-b}^{0}\kappa\rho_{0}\bar{u}^{2}(|\varphi'-\xi_{1}\psi|^{2}+ |\psi'-\xi_{1}\varphi|^{2}+\xi_{2}^{2}(|\varphi|^{2} +|\theta|^{2}+|\psi|^{2})+\xi_{1}^{2}|\theta|^{2} +|\theta'|^{2}\\ &\quad -2\xi_{2}\mathfrak{R}\theta'\mathfrak{R}\psi -2\xi_{2}\mathfrak{I}\theta'\mathfrak{I}\psi -2\xi_{2}\mathfrak{R}\theta\mathfrak{R}\psi' -2\xi_{2}\mathfrak{I}\theta\mathfrak{I}\psi')dx_{3}\bigg) \end{split} \end{equation*}
    \begin{equation*} \begin{split} &-\frac{1}{8\pi^{2}L^{2}}\int_{-b}^{0}P'(\rho_{0})\rho_{0}|\psi'(0, x_{3})|^{2}+ \kappa\rho_{0}\bar{u}^{2}(|\varphi'(0, x_{3})|^{2}+ |\psi'(0, x_{3})|^{2}+|\theta'(0, x_{3})|^{2})dx_{3}\\ & = :\frac{1}{4\pi^{2}L^{2}}\sum\limits_{\xi\in(L^{-1}\mathbb{Z})^{2}\backslash \{0\}}Z(\varphi, \theta, \psi;\xi)\\ \end{split} \end{equation*}
    \quad-\frac{1}{8\pi^{2}L^{2}}\int_{-b}^{0}P'(\rho_{0})\rho_{0}|\psi'(0, x_{3})|^{2}+ \kappa\rho_{0}\bar{u}^{2}(|\varphi'(0, x_{3})|^{2}+ |\psi'(0, x_{3})|^{2}+|\theta'(0, x_{3})|^{2})dx_{3}.

    Notice that Z(\varphi, \theta, \psi; \xi) is obviously invariant under simultaneous rotations of \xi and (\varphi, \theta) , so without loss of generality we may assume that \xi = (|\xi|, 0) with |\xi| > 0 and \theta = 0 . Then, for \xi\in \mathbb{F} , we have

    \begin{equation*} \label{EB-multi-3.11} \begin{split} Z(\varphi, \theta, \psi;\xi)& = \frac{1}{2}\int_{-b}^{0}g\rho_{0}|\xi| (|\varphi|^{2}+|\psi|^{2}-|\varphi-\psi|^{2})dx_{3} -\frac{\sigma|\xi|^{2}|\psi(\xi, 0)|^{2}}{2}\\ &-\frac{1}{2}\int_{-b}^{0}P'(\rho_{0})\rho_{0} ||\xi|\varphi+\psi'|^{2}dx_{3} +\kappa\rho_{0}\bar{u}^{2}(|\varphi'-|\xi|\psi|^{2}+ |\psi'-|\xi|\varphi|^{2})dx_{3}\\ & = -E(\varphi, \psi;\lambda(|\xi|))+\frac{\lambda(|\xi|)}{2}\int_{-b}^{0}\delta_{0} (\psi'+|\xi|\varphi)^{2}dx_{3}\\ &\qquad+\frac{\lambda(|\xi|)}{2}\int_{-b}^{0}\varepsilon_{0}((\varphi'-|\xi|\psi)^{2}+(\psi'-|\xi|\varphi)^{2} +\frac{1}{3}(\psi'+|\xi|\varphi)^{2})dx_{3}, \end{split} \end{equation*}

    and hence

    \begin{equation} \begin{split} Z(\varphi, \theta, \psi;\xi)&\leq\frac{\Lambda^{2}}{2}\int_{-b}^{0}\rho_{0} (|\varphi|^{2}+|\psi|^{2})+\frac{\Lambda}{2}\int_{-b}^{0}\delta_{0} (\psi'+|\xi|\varphi)^{2}dx_{3}\\ &\qquad+\frac{\Lambda}{2}\int_{-b}^{0}\varepsilon_{0}((\varphi'-|\xi|\psi)^{2}+(\psi'-|\xi|\varphi)^{2} +\frac{1}{3}(\psi'+|\xi|\varphi)^{2})dx_{3}, \end{split} \end{equation} (3.7)

    where we have used the following variational characterization for \Lambda , which follows from the definitions (2.20) and (2.37),

    \begin{equation*} \label{EB-multi-3.13} \begin{split} 0 > E(\varphi, \psi;\lambda(|\xi|))\geq-\lambda(|\xi|)^{2}J(\varphi, \psi)\geq -\frac{\Lambda^{2}}{2}\int_{-b}^{0}\rho_{0}(|\varphi|^{2}+|\psi|^{2}). \end{split} \end{equation*}

    For \xi\in(L^{-1}\mathbb{Z})^{2}\backslash \mathbb{F} , owning to the definition of \mathbb{F} , one can see

    \begin{equation*} \label{EB-multi-3.14} \begin{split} Z(\varphi, \theta, \psi;\xi)\leq 0. \end{split} \end{equation*}

    Then, for all \xi\in(L^{-1}\mathbb{Z})^{2} , Z(\varphi, \theta, \psi; \xi) satisfies (3.7). Combining with (3.6), and translating the resulting inequality back to the original notation for fixed \xi , by the Fubini and Parseval theorems, we find

    \begin{equation} \begin{split} &\frac{1}{2}\int_{\Omega}g\rho_{0}(v_{3}\mbox{div}\, v-\nabla v_{3}\cdot v)-P'(\rho_{0})\rho_{0}|\mbox{div}\, v|^{2} -\kappa\rho_{0}\bar{u}^{2}(\frac{1}{2}|\mathbb{D}(v)|^{2}-|\mbox{div}\, v|^{2})dx\\ &\leq\frac{1}{4\pi^{2}L^{2}}\sum\limits_{\xi\in(L^{-1}\mathbb{Z})^{2}}\bigg(\frac{\Lambda^{2}}{2}\int_{-b}^{0}\rho_{0} (|\varphi|^{2}+|\psi|^{2})+\frac{\Lambda}{2}\int_{-b}^{0}\delta_{0} (\psi'+|\xi|\varphi)^{2}dx_{3}\\ &\qquad+\frac{\Lambda}{2}\int_{-b}^{0}\varepsilon_{0}((\varphi'-|\xi|\psi)^{2}+(\psi'-|\xi|\varphi)^{2} +\frac{1}{3}(\psi'+|\xi|\varphi)^{2})dx_{3}\bigg)\\ & = \frac{\Lambda^{2}}{2}\int_{\Omega}\rho_{0}|v|^{2}dx+\frac{\Lambda}{2}\int_{\Omega} \frac{\varepsilon_{0}}{2}\bigg|\nabla v+\nabla v^{T}-\frac{2}{3} (\mbox{div}\, v)I\bigg|^{2}+\delta_{0}|\mbox{div}\, v|^{2}dx. \end{split} \end{equation} (3.8)

    For any v\in H_{0}^{1} not satisfying (2.23), we have

    \begin{equation} \begin{split} &\frac{1}{2}\int_{\Omega}g\rho_{0}(v_{3}\mbox{div}\, v-\nabla v_{3}\cdot v)-P'(\rho_{0})\rho_{0}|\mbox{div}\, v|^{2} -\kappa\rho_{0}\bar{u}^{2}(\frac{1}{2}|\mathbb{D}(v)|^{2}-|\mbox{div}\, v|^{2})dx\\ &\qquad -\int_{\Sigma_{0}}\frac{\sigma}{2}|\nabla_{x_{1}, x_{2}}v_{3}|^{2}dS_{0}\leq0, \end{split} \end{equation} (3.9)

    which implies that (3.5) holds trivially since the right of (3.5) is non-negative. Combining with (3.6), (3.8) and (3.9), we conclude Lemma 3.2.

    Now we can prove our main result.

    Proof of Theorem 1.1. Integrating the result of Lemma 3.1 in time from 0 to t , by Lemma 3.2, we get

    \begin{equation} \begin{split} &\int_{\Omega}\rho_{0}\frac{|\partial_{t}u(t)|^{2}}{2}dx +\int_{0}^{t}\int_{\Omega}\frac{\varepsilon_{0}}{2}\bigg|\mathbb{D}(\partial_{t}u)(s)-\frac{2}{3} (\mbox{div}\partial_{t}u(s))\mathbb{I}\bigg|^{2}+\delta_{0}|\mbox{div}\partial_{t}u(s)|^{2}dxds \\ &\leq K_{0}+\int_{\Omega}(-\frac{P'(\rho_{0})\rho_{0}}{2}|\mbox{div}u(t)|^{2}+g\rho_{0}u_{3}(t)\mbox{div}u(t) +\frac{g\rho_{0}'}{2}|u_{3}(t)|^{2}\\ &-\frac{\kappa\rho_{0}\bar{u}^{2}}{2} (\frac{1}{2}|\mathbb{D}(u)(t)|^{2}-|\mbox{div}u(t)|^{2}))dx- \int_{\Sigma_{0}}\frac{g\rho_{0}}{2}|u_{3}(t)|^{2}dS_{0} -\int_{\Sigma_{0}}\frac{\sigma}{2}|\nabla_{x_{1}, x_{2}}u_{3}(t)|^{2}dS_{0}\\ &\leq K_{0}+\frac{\Lambda^{2}}{2}\int_{\Omega}\rho_{0}|u(t)|^{2}dx+\frac{\Lambda}{2}\int_{\Omega} \frac{\varepsilon_{0}}{2}\bigg|\mathbb{D}(u(t))-\frac{2}{3} (\mbox{div} u(t))I\bigg|^{2}+\delta_{0}|\mbox{div} u(t)|^{2}dx. \end{split} \end{equation} (3.10)

    where

    \begin{equation*} \label{EB-multi-3.18} \begin{split} K_{0} = &\int_{\Omega}\rho_{0}\frac{|\partial_{t}u(0)|^{2}}{2}+\frac{\kappa\rho_{0}}{4}|\mathbb{D}(u(0))|^{2}+ \frac{P'(\rho_{0})\rho_{0}}{2}|\mbox{div} u(0)|^{2}dx +\int_{\Sigma_{0}}\frac{g\rho_{0}}{2}|u_{3}(0)|^{2}dS_{0}\\ &\quad+\int_{\Omega}g\rho_{0}(|u_{3}(0)|^{2}+|\mbox{div} u(0)|^{2})dx +\int_{\Sigma_{0}}\frac{\sigma}{2}|\nabla_{x_{1}, x_{2}}u_{3}(0)|^{2}dS_{0}. \end{split} \end{equation*}

    Using the definitions of the norms \|\cdot\|_{\ast} , \|\cdot\|_{\ast\ast} given in (1.9), we may compactly rewrite (3.10) as

    \begin{equation} \begin{split} \frac{1}{2}\|\partial_{t}u(t)\|_{\ast}^{2}+\int_{0}^{t}\|\partial_{t}u(s)\|_{\ast\ast}^{2}\, ds\leq K_{0} +\frac{\Lambda^{2}}{2}\|u(t)\|_{\ast}^{2}+\frac{\Lambda}{2}\|u(t)\|_{\ast\ast}^{2}. \end{split} \end{equation} (3.11)

    Integrating in time and using Cauchy's inequality, we may bound

    \begin{equation} \begin{split} \Lambda\|u(t)\|_{\ast\ast}^{2}& = \Lambda\|u(0)\|_{\ast\ast}^{2}+\Lambda\int_{0}^{t}2\langle u(s), \partial_{t}u(s)\rangle _{\ast\ast}\, ds\\ &\leq\Lambda\|u(0)\|_{\ast\ast}^{2}+\int_{0}^{t}\|\partial_{t}u(s)\|_{\ast\ast}^{2}\, ds+\Lambda^{2} \int_{0}^{t}\|u(s)\|_{\ast\ast}^{2}\, ds. \end{split} \end{equation} (3.12)

    On the other hand

    \begin{equation} \begin{split} \Lambda\partial_{t}\|u(t)\|_{\ast}^{2} = 2\Lambda\langle \partial_{t}u(t), u(t)\rangle_{\ast}\leq\Lambda^{2} \|u(t)\|_{\ast}^{2}+\|\partial_{t}u(t)\|_{\ast}^{2}. \end{split} \end{equation} (3.13)

    Hence, combining (3.12) and (3.13) with (3.11), we derive the differential inequality

    \begin{equation} \begin{split} \partial_{t}\|u(t)\|_{\ast}^{2}+\|u(t)\|_{\ast\ast}^{2}\leq K_{1}+2\Lambda(\|u(t)\|_{\ast}^{2}+\int_{0}^{t} \|u(s)\|_{\ast\ast}^{2}\, ds) \end{split} \end{equation} (3.14)

    for K_{1} = 2K_{0}/\Lambda+2\|u(0)\|_{\ast\ast}^{2} . Applying Gronwall's inequality to (3.14), we can get

    \begin{equation} \begin{split} \|u(t)\|_{\ast}^{2}+\int_{0}^{t}\|u(s)\|_{\ast\ast}^{2}\, ds\leq e^{2\Lambda t}\|u(0)\|_{\ast}^{2}+\frac{K_{1}}{2\Lambda} (e^{2\Lambda t}-1) \end{split} \end{equation} (3.15)

    for all t\geq 0 . Now plugging (3.15) and (3.12) into (3.11), we find that

    \begin{equation} \begin{split} \frac{1}{\Lambda}\|\partial_{t}u(t)\|_{\ast}^{2}+\|u(t)\|_{\ast\ast}^{2}\leq K_{1} +\Lambda\|u(t)\|_{\ast}^{2}+2\Lambda\int_{0}^{t}\|u(s)\|_{\ast\ast}^{2}\, ds\leq e^{2\Lambda t}(2\Lambda\|u(0)\|_{\ast}^{2}+K_{1}). \end{split} \end{equation} (3.16)

    By the trace theorem, we have

    \begin{equation*} \label{EB-multi-3.25} \begin{split} K_{0}, K_{1}\leq C(\|\partial_{t}u(0)\|_{\ast}^{2}+\|u(0)\|_{\ast}^{2}+\|u(0)\|_{\ast\ast}^{2} +\sigma\int_{\Sigma_{0}}|\nabla_{x_{1}, x_{1}}u_{3}(0)|^{2}) \end{split} \end{equation*}

    for a constant C > 0 depending on \rho_{0}, P, \Lambda, \varepsilon, \delta, \kappa, \sigma, g, b .

    For the estimates for \eta, q, V , we can get from (1.8)_{1} , (1.8)_{2} , (1.8)_{4} , (3.16) and the Korn's lemma (seen Lemma 3.6 in [4]) that

    \begin{equation*} \begin{split} \|\eta(t)\|_{H^{1}}&\leq \|\eta_{0}\|_{H^{1}}+\int_{0}^{t}\|\partial_{s}\eta(s)\|_{H^{1}}ds \leq \|\eta_{0}\|_{H^{1}}+\int_{0}^{t}\|u(s)\|_{H^{1}}ds\\ &\leq Ce^{\Lambda t}(\|\eta_{0}\|_{H^{1}}+\sqrt{I_{0}}), \end{split} \end{equation*}
    \begin{equation*} \begin{split} \|q(t)\|_{L^{2}}&\leq \|q_{0}\|_{L^{2}}+\int_{0}^{t}\|\partial_{s}q(s)\|_{L^{2}}ds \leq \|q_{0}\|_{L^{2}}+\int_{0}^{t}\|u(s)\|_{H^{1}}ds\\ &\leq Ce^{\Lambda t}(\|q_{0}\|_{L^{2}}+\sqrt{I_{0}}), \end{split} \end{equation*}

    and

    \begin{equation*} \begin{split} \|V(t)\|_{L^{2}}&\leq \|V_{0}\|_{L^{2}}+\int_{0}^{t}\|\partial_{s}V(s)\|_{L^{2}}ds \leq \|V_{0}\|_{L^{2}}+\int_{0}^{t}\|u(s)\|_{H^{1}}ds\\ &\leq Ce^{\Lambda t}(\|V_{0}\|_{L^{2}}+\sqrt{I_{0}}), \end{split} \end{equation*}

    where I_{0} = \|\partial_{t}u(0)\|_{\ast}^{2}+\|u(0)\|_{\ast}^{2}+\|u(0)\|_{\ast\ast}^{2}+ \sigma\int_{\Sigma_{0}}|\nabla_{x_{1}, x_{2}}u_{3}(0)|^{2} . Then we complete the proof of Theorem 1.1.

    This paper considers the linear Rayleigh-Taylor instability of an equilibrium state of 3D gravity-driven compressible viscoelastic fluid with the elasticity coefficient \kappa is less than a critical number \kappa_{c} in a moving horizontal periodic domain. We apply a method of studying a family of modified variational problems in order to produce the maximal growing mode solutions to the linearized equations, and then prove an estimate for arbitrary solutions to the linearized equations in terms of the fastest possible growth rate for the growing modes.

    The authors would like to express their sincere appreciation to the anonymous refree for valuable comments.

    This work is supported in part by the National Natural Science Foundation of China under Grant 11801443.

    All authors declare no conflicts of interest in this paper.



    [1] S. Chandrasekhar, Hydrodynamic and hydromagnetic stability, Clarendon Press, 1961.
    [2] R. Duan, F. Jiang, On the Rayleigh-Taylor instability for incompressible, inviscid magnetohydrodynamic flows, SIAM J. Appl. Math., 71 (2011), 1990–2013. https://doi.org/10.1137/110830113 doi: 10.1137/110830113
    [3] D. Ebin, Ill-posedness of the Rayleigh-Taylor and Helmholtz problems for incompressible fluids, Commun. Part. Diff. Eq., 13 (1988), 1265–1295. https://doi.org/10.1080/03605308808820576 doi: 10.1080/03605308808820576
    [4] G. L. Gui, Z. F. Zhang, Global stability of the compressible viscous surface waves in an infinite layer, 2022, arXiv: 2208.06654.
    [5] Y. Guo, I. Tice, Compressible, inviscid Rayleigh-Taylor instability, Indiana U. Math. J., 60 (2011), 677–712.
    [6] Y. Guo, I. Tice, Linear Rayleigh-Taylor instability for viscous, compressible fluids, SIAM J. Math. Anal., 42 (2010), 1688–1720. http://dx.doi.org/10.1137/090777438 doi: 10.1137/090777438
    [7] G. J. Huang, F. Jiang, W. W. Wang, On the nonlinear Rayleigh-Taylor instability of nonhomogeneous incompressible viscoelastic fluids under L^{2}-norm, J. Math. Anal. Appl., 455 (2017), 873–904. https://doi.org/10.1016/j.jmaa.2017.06.022 doi: 10.1016/j.jmaa.2017.06.022
    [8] H. Hwang, Y. Guo, On the dynamical Rayleigh-Taylor instability, Arch. Ration. Mech. An., 167 (2003), 235–253. https://doi.org/10.1007/s00205-003-0243-z doi: 10.1007/s00205-003-0243-z
    [9] F. Jiang, On effects of viscosity and magnetic fields on the largest growth rate of linear Rayleigh-Taylor instability, J. Math. Phys., 57 (2016), 111503. https://doi.org/10.1063/1.4966924 doi: 10.1063/1.4966924
    [10] F. Jiang, S. Jiang, On instability and stability of three-dimensional gravity flows in a bounded domain, Adv. Math., 264 (2014), 831–863. https://doi.org/10.1016/j.aim.2014.07.030 doi: 10.1016/j.aim.2014.07.030
    [11] F. Jiang, S. Jiang, On linear instability and stability of the Rayleigh-Taylor problem in magnetohydrodynamics, J. Math. Fluid Mech., 17 (2015), 639–668. https://doi.org/10.1007/s00021-015-0221-x doi: 10.1007/s00021-015-0221-x
    [12] F. Jiang, S. Jiang, On the stabilizing effect of the magnetic field in the magnetic Rayleigh-Taylor problem, SIAM J. Math. Anal., 50 (2018), 491–540. https://doi.org/10.1137/16M1069584 doi: 10.1137/16M1069584
    [13] F. Jiang, S. Jiang, G. X. Ni, Nonlinear instability for nonhomogeneous incompressible viscous fluids, Sci. China Math., 56 (2013), 665–686. https://doi.org/10.1007/s11425-013-4587-z doi: 10.1007/s11425-013-4587-z
    [14] F. Jiang, S. Jiang, Y. J. Wang, On the Rayleigh-Taylor instability for the incompressible viscous magnetohydrodynamic equations, Commun. Part. Diff. Eq., 39 (2014), 399–438. https://doi.org/10.1080/03605302.2013.863913 doi: 10.1080/03605302.2013.863913
    [15] F. Jiang, S. Jiang, G. C. Wu, On stabilizing effect of elasticity in the Rayleigh-Taylor problem of stratified viscoelastic fluids, J. Funct. Anal., 272 (2017), 3763–3824. https://doi.org/10.1016/j.jfa.2017.01.007 doi: 10.1016/j.jfa.2017.01.007
    [16] J. Jang, I. Tice, Instability theory of the Navier-Stokes-Poisson equations, Anal. PDE, 6 (2013), 1121–1181. https://doi.org/10.2140/apde.2013.6.1121 doi: 10.2140/apde.2013.6.1121
    [17] J. Jang, I. Tice, Y. J. Wang, The compressible viscous surface-internal wave problem: Nonlinear Rayleigh-Taylor instability, Arch. Ration. Mech. An., 221 (2016), 215–272. https://doi.org/10.1007/s00205-015-0960-0 doi: 10.1007/s00205-015-0960-0
    [18] H. Kull, Theory of the Rayleigh-Taylor instability, Phys. Rep., 206 (1991), 197–325. https://doi.org/10.1016/0370-1573(91)90153-D doi: 10.1016/0370-1573(91)90153-D
    [19] J. Prüss, G. Simonett, On the Rayleigh-Taylor instability for the two-phase Navier-Stokes equations, Indiana Univ. Math. J., 59 (2010), 1853–1872. https://doi.org/10.1512/iumj.2010.59.4145 doi: 10.1512/iumj.2010.59.4145
    [20] L. Rayleigh, Investigation of the character of the equilibrium of an in compressible heavy fluid of variable density, P. Lond. Math. Soc., s1-14 (1882), 170–177. https://doi.org/10.1112/plms/s1-14.1.170 doi: 10.1112/plms/s1-14.1.170
    [21] G. I. Taylor, The stability of liquid surface when accelerated in a direction perpendicular to their planes. I., Proc. Roy. Soc. London A, 201 (1950), 192–196. https://doi.org/10.1098/rspa.1950.0052 doi: 10.1098/rspa.1950.0052
    [22] W. W. Wang, Y. Y. Zhao, On the Rayleigh-Taylor instability in compressible viscoelastic fluids, J. Math. Anal. Appl., 463 (2018), 198–221. https://doi.org/10.1016/j.jmaa.2018.03.018 doi: 10.1016/j.jmaa.2018.03.018
    [23] Y. J. Wang, I. Tice, The viscous surface-internal wave problem: nonlinear Rayleigh-Taylor instability, Commun. Part. Diff. Eq., 37 (2012), 1967–2028. https://doi.org/10.1080/03605302.2012.699498 doi: 10.1080/03605302.2012.699498
    [24] Y. J. Wang, I. Tice, C. Kim, The viscous surface-internal wave problem: Global well-posedness and decay, Arch. Ration. Mech. Anal., 212 (2014), 1–92. https://doi.org/10.1007/s00205-013-0700-2 doi: 10.1007/s00205-013-0700-2
    [25] Y. Y. Zhao, W. W. Wang, On the Rayleigh-Taylor instability incompressible viscoelastic fluids under L^{1}-norm, J. Comput. Appl. Math., 383 (2021), 113130. https://doi.org/10.1016/j.cam.2020.113130 doi: 10.1016/j.cam.2020.113130
  • Reader Comments
  • © 2023 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)
通讯作者: 陈斌, bchen63@163.com
  • 1. 

    沈阳化工大学材料科学与工程学院 沈阳 110142

  1. 本站搜索
  2. 百度学术搜索
  3. 万方数据库搜索
  4. CNKI搜索

Metrics

Article views(1708) PDF downloads(168) Cited by(0)

Figures and Tables

Figures(1)

Other Articles By Authors

/

DownLoad:  Full-Size Img  PowerPoint
Return
Return

Catalog