Loading [MathJax]/jax/element/mml/optable/MathOperators.js
Research article

-Ricci tensor on (κ,μ)-contact manifolds

  • Received: 30 December 2021 Revised: 07 March 2022 Accepted: 10 March 2022 Published: 13 April 2022
  • MSC : 53C21, 53D10, 53D15

  • We introduce the notion of semi-symmetric -Ricci tensor and illustrate that a non-Sasakian (κ,μ)-contact manifold is -Ricci semi-symmetric or has parallel -Ricci operator if and only if it is -Ricci flat. Then we find that among the non-Sasakian (κ,μ)-contact manifolds with the same Boeckx invariant IM, only one is -Ricci flat, so we can think of it as the representative of such class. We also give two methods to construct -Ricci flat (κ,μ)-contact manifolds.

    Citation: Rongsheng Ma, Donghe Pei. -Ricci tensor on (κ,μ)-contact manifolds[J]. AIMS Mathematics, 2022, 7(7): 11519-11528. doi: 10.3934/math.2022642

    Related Papers:

    [1] Aissa Boukarou, Kaddour Guerbati, Khaled Zennir, Mohammad Alnegga . Gevrey regularity for the generalized Kadomtsev-Petviashvili I (gKP-I) equation. AIMS Mathematics, 2021, 6(9): 10037-10054. doi: 10.3934/math.2021583
    [2] Xiaochun Sun, Yulian Wu, Gaoting Xu . Global well-posedness for the 3D rotating Boussinesq equations in variable exponent Fourier-Besov spaces. AIMS Mathematics, 2023, 8(11): 27065-27079. doi: 10.3934/math.20231385
    [3] Muhammad Imran Liaqat, Fahim Ud Din, Wedad Albalawi, Kottakkaran Sooppy Nisar, Abdel-Haleem Abdel-Aty . Analysis of stochastic delay differential equations in the framework of conformable fractional derivatives. AIMS Mathematics, 2024, 9(5): 11194-11211. doi: 10.3934/math.2024549
    [4] William O. Bray, Ellen Hunter . Wave equations & energy. AIMS Mathematics, 2019, 4(3): 463-481. doi: 10.3934/math.2019.3.463
    [5] Phuong Nguyen Duc, Ahmet Ocak Akdemir, Van Tien Nguyen, Anh Tuan Nguyen . Remarks on parabolic equation with the conformable variable derivative in Hilbert scales. AIMS Mathematics, 2022, 7(11): 20020-20042. doi: 10.3934/math.20221095
    [6] Kedong Wang, Xianguo Geng, Mingming Chen, Ruomeng Li . Long-time asymptotics for the generalized Sasa-Satsuma equation. AIMS Mathematics, 2020, 5(6): 7413-7437. doi: 10.3934/math.2020475
    [7] Said Mesloub, Faten Aldosari . Well posedness for a singular two dimensional fractional initial boundary value problem with Bessel operator involving boundary integral conditions. AIMS Mathematics, 2021, 6(9): 9786-9812. doi: 10.3934/math.2021569
    [8] Zhongdi Cen, Jian Huang, Aimin Xu . A posteriori mesh method for a system of singularly perturbed initial value problems. AIMS Mathematics, 2022, 7(9): 16719-16732. doi: 10.3934/math.2022917
    [9] Murat A. Sultanov, Vladimir E. Misilov, Makhmud A. Sadybekov . Numerical method for solving the subdiffusion differential equation with nonlocal boundary conditions. AIMS Mathematics, 2024, 9(12): 36385-36404. doi: 10.3934/math.20241726
    [10] Dumitru Baleanu, Babak Shiri . Generalized fractional differential equations for past dynamic. AIMS Mathematics, 2022, 7(8): 14394-14418. doi: 10.3934/math.2022793
  • We introduce the notion of semi-symmetric -Ricci tensor and illustrate that a non-Sasakian (κ,μ)-contact manifold is -Ricci semi-symmetric or has parallel -Ricci operator if and only if it is -Ricci flat. Then we find that among the non-Sasakian (κ,μ)-contact manifolds with the same Boeckx invariant IM, only one is -Ricci flat, so we can think of it as the representative of such class. We also give two methods to construct -Ricci flat (κ,μ)-contact manifolds.



    Sparse optimization is a core problem of compressed sensing [1,2,3], signal and image processing [4,5,6,7], and high-dimensional statistical learning [4,8], etc. Sparsity is usually characterized by the 0 norm, which is the cardinality of the support set of vector xRn, denoted by x0=|supp(x)|=|{i{1,2,,n}:xi0}|. The penalized formulation of sparse optimization can be expressed as the following cardinality regularized optimization:

    minxRnf(x)+λx0, (1.1)

    where f:RnR is a loss function depending on the application, λ>0 is the penalty parameter.

    Compared with sparse optimization, composite sparse optimization problems enforce certain structural constraints instead of pure sparsity on the coefficients, which arise from many important applications in various fields, such as structural health monitoring [10], fault diagnosis [11], motion planning [12] and impact force identification[13], etc. The most important method is to promote the sparsity of variables through linear transformation [14]. By imposing a regularization matrix W=(W1,,Wp)Rp×n on vector x, composite sparse optimization is nicely encapsulated as the following optimization formulation:

    minxRnf(x)+λWx0. (1.2)

    A typical choice of function f in problem (1.2) is the 2 loss function given by f(x)=12Axb22, where A=(A1,,Am)Rm×n and b=(b1,,bm)Rm, and the 1 relaxation of the 0 norm given by Wx1, which was first defined and summarized as generalized LASSO with the general formulation of W [15]. Unlike traditional LASSO, solving generalized LASSO efficiently on high- dimensional data is very challenging. A few attempts have been made to improve the efficiency of generalized LASSO, but this requires a specific form of the W to work well [14,15,16], such as the fused LASSO problem [17], the TV regularizer [18] and trending filtering [19].

    However, many loss functions of the composite sparse optimization problems cannot be expressed in the form of differentiable functions. The results in [20] showed that the least squares loss function can solve a class of linear regression problems but is not suitable for all types of data. We can choose the outlier that has a strong interference loss function, such as the 1 function, quantile regression function, or more general Huber class function. On the other hand, J. Fan et al. [20] pointed out that using the 1 relaxation often results in a biased estimator, various continuous nonconvex relaxation functions for 0 norm were proposed, such as the smoothly clipped absolute deviation (SCAD) function [20], the hard thresholding function [21], capped-1 function [22,23,24,25,26], the transformed 1 function [27], etc. Here, we are interested in the capped-1 function as the relaxation function of the 0 norm, which is a simple relaxation function that satisfies specific properties. Z. Shen et al. [40] applied locally Lipschitz continuous scaled folded concave functions to the approximate 0 pseudo-norm. A generic nonsmooth but convex framework was established to gradually approximate the scaled folded concave functions. Numerical experimental results showed the proposed framework and algorithms admitted the exact sparsity-induced capability of the 0 pseudo-norm. Q. Chen et al.[41] first explored using a class of locally Lipschitz scale folded concave functions to approach the 0. Then, a convex half-absolute method was proposed to precisely approximate these nonconvex nonsmooth functions. A double iterative algorithm was considered to solve the convex-relaxed composite optimization problems. Both [40] and [41] established a generic nonsmooth convex framework that gradually approximates these scale-folded concave functions based on the Legendre-Fenchel transformation to avoid directly solving a nonconvex optimization problem. However, we use a smoothing function for approximation to achieve the goal of solving the nonconvex optimization problem. The advantages of capped-1 function have been explored in various fields. For example, the authors in [28] put forward a capped 1-norm Sparse Representation method (CSR) for graph clustering. The proposed model learned the optimal graph for clustering by integrating sparse representation and capped 1-norm loss function. In order to utilize the advantages of the twin extreme learning machine and FDA, Z. Xue et al. [29] first put forward a novel classifier named Fisher-regularized twin extreme learning machine (FTELM). Also considering the instability of the 2-norm for the outliers, authors introduced the capped 1-norm into the FTELM model and proposed a more robust capped 1-norm FTELM (C1-FTELM) model. The capped 1 function was also discussed in the context of sparse group 0 regularized algorithms by [30]. It's worth noting that reference [9] gave an exact continuous relaxation problem with capped-1 penalty for nonsmooth convex loss function with cardinality penalty in the sense that both problems have the same optimal solution set. Moreover, a smoothing proximal gradient algorithm for finding a lifted stationary point of the continuous relaxation model was proposed. Regarding the solution of relaxation problems, T. Zhang [42] presented a multi-stage convex relaxation scheme for solving problems with non-convex objective functions. However, only parameter estimation performance was analyzed in [42]. Unfortunately, the result in [42] does not directly imply that multi-stage convex relaxation achieves unbiased recovery of the support set. H. Zhou et al. [43] proposed a new unified algorithm based on the local linear approximation (LLA) for maximizing the penalized likelihood for a broad class of concave penalty functions. It did not eliminate the bias issue. Here, we extend the results in [9] to composite sparse optimization and give a smoothing gradient descent algorithm for the continuous relaxation problem. The new algorithm exploits the piecewise linearity of the capped-1 penalty term in the relaxation problem. In view of the composite sparsity, if the subproblem in the algorithm does not have a closed solution, then our relaxation problem model analogizes the 1 penalty model for solving LASSO problems, using the smoothing gradient method to solve the subproblem. We prove that if the sequence generated by the algorithm has an accumulation point, then it is a lifted stationary point of relaxation problem.

    In this paper, we consider the following composite sparse regression problem with cardinality penalty,

    minxΩW0(x):=f(x)+λWx0, (1.3)

    where f:RnR is a convex (not necessarily smooth) and bounded from below function, λ is a positive parameter, and Ω={xRn:lWxu}. For example, the 1 loss function given by

    f(x)=1mAxb1, (1.4)

    or the censored regression problem with

    f(x)=1mmi=1|max{Aix,0}bi|. (1.5)

    For a given parameter ν>0, the continuous relaxation of the 0 penalty with the capped-1 function ϕ is given by

    ϕ(t)=min{1,|t|ν}. (1.6)

    We consider the following continuous optimization problem to solve (1.3):

    minxΩW(x):=f(x)+λΦ(Wx), (1.7)

    where Φ(Wx)=pi=1ϕ(Wix).

    Composite sparse optimization has attracted much attention recently. In [15], a dual path algorithm was proposed for the generalized LASSO problem with any formulation of W. If the composite optimization problem is convex and the W is the general linear map, one feasible approach is to apply the alternating direction method of multipliers (ADMM)[31]. In [31], the author proposed a dual method for the variational problem in the form of inf{f(Av)+g(v)}. Z. J. Bai [32] aimed to provide a coordinate gradient descent method with stepsize chosen by an Armijo-type rule to solve the problem minxf(x)+cLx1 and minxAxb22+cLx1 efficiently, especially when the problems dimension is large.

    In this paper, we use the exact continuous relaxation problem with capped-1 function to solve optimization problem (1.3) and present a smoothing gradient descent (SGD) algorithm. Since W is a general linear mapping and Φ(Wx) is an inseparable function, which makes the proximal gradient algorithm unable to be explicitly applied, we approximately solve the subproblem in the SGD algorithm. We prove that if there is an accumulation point, then the accumulation point is a lifted stationary point of (1.7).

    Notation. We denote N={0,1,}. For xRn and δ>0, let x:=x2 and Bδ(x) means the open ball centered at x with radius δ. For a nonempty, closed, and convex set ΩRn, NΩ(x) means the normal cone to Ω at xΩ. σmin(W) is the minimum singular value of W. We denote x=max{|x1|,|x2|,,|xn|}.

    Before starting this section, we first make the following two assumptions:

    Assumption 2.1. Function f is Lipschitz continuous on Ω with Lipschitz constant Lf>0 and matrix W has full column rank.

    Assumption 2.2. Parameter ν in (1.6) satisfies 0<ν<¯ν:=λσmin(W)Lf.

    We suppose Assumptions 2.1 and 2.2 hold throughout the paper and assume that Lf is large enough such that Lfλσmin(W)Γ, where

    Γ:=min{|li|,ui:li0,ui0,i=1,,p}.

    When f is defined by the 1 loss function (1.4) or the censored regression loss function (1.5), Lf can be taken as Lf=A.

    We first give the definition of lifted stationary points of (1.7) as that in [33]. Since function ϕ in (1.6) can be rephrased as

    ϕ(t)=1ν|t|max{θ1(t),θ2(t),θ3(t)}

    with θ1(t)=0, θ2(t)=tν1 and θ3(t)=tν1 for tR, we denote

    D(t):={i{1,2,3}:θi(t)=max{θ1(t),θ2(t),θ3(t)}} (2.1)

    and

    Dp(Wx):=Πpi=1D(Wix). (2.2)

    Definition 2.1. Point xΩ is called a lifted stationary point of (1.7) if there exists d=(d1,,dp)Dp(Wx) such that

    λpi=1(θdi(Wix))f(x)+λνpi=1Wiϑi(x)+NΩ(x), (2.3)

    where

    ϑi(x){=1ifWix>0,[1,1]if|Wix|=0,=1ifWix<0. (2.4)

    Under the definition of the range of ν in Assumption 2.2, we first prove that the element in Dp(Wx) for a lifted stationary point satisfying (2.3) is unique.

    Proposition 2.2. If ˉx is a lifted stationary point of (1.7), then the vector dWˉx=(dWˉx1,,dWˉxp)Dp(Wˉx) satisfying (2.3) is unique. In particular, for i=1,,p,

    dWˉxi={1if|Wiˉx|<ν,2ifWiˉxν,3ifWiˉxν. (2.5)

    Proof. If |Wiˉx|ν, the statement is clearly valid. Hence, we only need to consider the case |Wiˉx|=ν. When Wiˉx=ν, since D(Wiˉx)={1,2}, arguing by contradiction, we assume (2.3) holds with dWˉxi=1, so ϑi(ˉx)=1. By ν<ˉν, we have Wiˉx(li,ui), and by (2.3), there exists ξ(ˉx)f(ˉx) such that

    0=ξ(ˉx)+λνi:|Wiˉx|νWiϑi(ˉx), (2.6)

    where ϑi(ˉx) is defined as (2.4).

    It is easy to observe that the following relation holds

    i:|Wiˉx|νWiϑi(ˉx)2σmin(W). (2.7)

    In fact, from the definition of the minimum singular value of W,

    σmin(W)=min{Wx2x2:x0}=min{Wx2:x2=1},

    we have

    σmin(W)=min{Wx2x2:x0}min{Wx2x2:x21,x1}min{Wx2:x21,x1}min{Wx2:x2=1}=σmin(W).

    Then, we see that

    σmin(W)=min{Wx2:x21,x1}.

    From the definition of ϑi(ˉx) (2.4), this yields that

    i:|Wiˉx|νWiϑi(ˉx)2σmin(WI)σmin(W),

    where WI is the submatrix consisting of the rows in W indexed by I:={i:|Wiˉx|ν} [34].

    Combining (2.6) and (2.7), we have

    λσmin(W)νλνi:|Wiˉx|νWiϑi(ˉx)=ξ(ˉx)Lf.

    This leads to a contradiction to ν<λσmin(W)Lf. Then, (2.5) holds for Wiˉx=ν. Similar analysis can be given for the case that Wiˉx=ν, which completes the proof.

    For a given d=(d1,,dp)Dp:={dRp:di{1,2,3},i=1,,p}, we define

    Φd(Wx):=pi=1|Wix|νpi=1θdi(Wix), (2.8)

    which is convex with respect to x. It can be verified that

    Φ(Wx)=mindDpΦd(Wx),xΩ.

    In particular, for a fixed ˉxΩ, Φ(Wˉx)=ΦdWˉx(Wˉx) and the following relation holds

    ˉxisaliftedstationarypointof(1.7)ˉxargminxΩf(x)+λΦdWˉx(Wx). (2.9)

    Next lemma describes a lower bound property.

    Lemma 2.3. If ˉxΩ is a lifted stationary point of (1.7), then it holds that

    Wiˉx(ν,ν)Wiˉx=0,i=1,,p. (2.10)

    Proof. Suppose that ˉx is a lifted stationary point of (1.7). Now we assume that Wiˉx(ν,ν){0} for some i1,,p. So from (2.5) and Assumption 2.1, we have dWˉxi=1 and Wiˉx(li,ui). By (2.3), there exists ξ(ˉx)f(ˉx). We have

    0=ξ(ˉx)+λνi:|Wiˉx|<νWiϑi(ˉx).

    Through the analysis in the proof of Proposition 2.2, combining (2.6) and (2.7), we have

    λσmin(W)νλνi:|Wiˉx|<νWiϑi(ˉx)=ξ(ˉx)Lf,

    which leads to a contradiction to ν<λσmin(W)Lf. Consequently, Wiˉx(ν,ν) implies Wiˉx=0 for i1,,p {and} the proof is completed.

    This subsection discusses the relationship between the global minimizers and local minimizers of (1.3) and (1.7). First, Theorem 2.4 discusses the relationship between the local minimizers of (1.3) and (1.7). Second, Theorem 2.5 states that (1.3) and (1.7) have the same global minimizers. We use the lower bound property mentioned in Lemma 2.3 to prove Theorems 2.4 and 2.5.

    Theorem 2.4. If ˉx is a lifted stationary point of (1.7), then it is a local minimizer of (1.3) and the objective functions have the same value at ˉx, i.e., f(ˉx)+λΦ(Wˉx)=f(ˉx)+λWˉx0.

    Proof. Combining the lower bound property of Wˉx in (2.10) and the definition of ΦdWˉx defined in (2.8), for any xRn, we have

    ΦdWˉx(Wx):=pi=1|Wix|νpi=1θdWˉxi(Wix)=i:|Wiˉx|ν1+i:|Wiˉx|<ν|Wix|ν=Wˉx0+i:Wiˉx=0|Wix|ν.

    Then

    ΦdWˉx(Wx)Wx0,xBϱ(ˉx),ϱ>0. (2.11)

    Combining this with Φ(Wˉx)=Wˉx0 and (2.9), we have

    f(ˉx)+λWˉx0f(x)+λWx0,xΩBϱ(ˉx).

    Thus, ˉx is a local minimizer of (1.7).

    Theorem 2.4 indicates that any lifted stationary point of (1.7) is a local minimizer of (1.3), which means that any local minimizer of (1.7) is also certainly a local minimizer of (1.3).

    Theorem 2.5. If ˉxΩ is a global minimizer of (1.3) if and only if it is a global minimizer of (1.7). Moreover, problems (1.3) and (1.7) have the same optimal value.

    Proof. On the one hand, let ˉxΩ be a global minimizer of (1.7), and according to Definition 2.1, then we can obtain that ˉx is a lifted stationary point of (1.7). By (2.10), from Wiˉx(ν,ν), then Wiˉx=0, so it gives Φ(Wˉx)=Wˉx0. We have

    f(ˉx)+λWˉx0=f(ˉx)+λΦ(Wˉx) f(x)+λΦ(Wx)f(x)+λWx0,xΩ,

    where the last inequality uses Φ(Wx)Wx0. Therefore, ˉx is a global minimizer of (1.3).

    On the other hand, let ˉxΩ be a global minimizer of (1.3). Assume on the contrary ˉx is not a solution of (1.7). Let ˆx be a global minimizer of (1.7), we obtain

    f(ˆx)+λΦ(Wˆx)<f(ˉx)+λΦ(Wˉx).

    From

    Φ(Wˆx)=Wˆx0andΦ(Wˉx)Wˉx0,

    we have

    f(ˆx)+λWˆx0<f(ˉx)+λWˉx0.

    This contradicts the global optimality of ˉx for (1.3). Hence ˉx is a global minimizer of (1.7). Therefore, (1.3) and (1.7) have the same global minimizers and optimal values.

    When f is convex, ˉx is a local minimizer of (1.3) if and only if ˉxΩ satisfies

    0f(ˉx)+NΩ(ˉx). (2.12)

    which is often used as a criterion for the local minimizers of problem (1.3).

    Definition 2.6. We call ˉxΩ a ν-strong local minimizer of (1.3), if there exists ¯ξf(ˉx) and ¯ηNΩ(ˉx) such that for any isupp(Wˉx), it holds

    0=¯ξ+¯ηand|Wiˉx|ν.

    By (2.12), any ν-strong local minimizer of (1.3) is a local minimizer of it. Below we provide a result on the relationship between the ν-strong local minimizers of (1.3) and the lifted stationary points of (1.7).

    Proposition 2.7. ˉxΩ is a ν-strong local minimizer of (1.3) if and only if it is a lifted stationary point of (1.7).

    Proof. First, by (2.9), we see that if ˉx is a lifted stationary point of (1.7), then

    W0(ˉx)=f(ˉx)+λWˉx0=f(ˉx)+λΦ(Wˉx)=f(ˉx)+λΦdWˉx(Wˉx)f(x)+λΦdWˉx(Wx),xΩ.

    Combining the Lemma 2.3 and ΦdWˉx(Wx)Wx0,xBϱ(ˉx),ϱ>0 in (2.11), then we have

    W0(ˉx)W0(x),xΩBϱ(ˉx),

    so ˉx is a ν-strong local minimizer of (1.3).

    Next, because ˉx is a ν-strong local minimizer of (1.3), it is also a local minimizer of (1.3), suppose ˉx is a local minimizer of (1.3) but not a local minimizer of (1.7). Then there exists a local minimizer of (1.7) denoted by ˆx, combining (2.9), ΦdWˆx(Wˉx)Wˉx0,ˉxBϱ(ˆx) in (2.11) and Φ(Wˆx)=Wˆx0, we have

    f(ˆx)+λWˆx0=f(ˆx)+λΦ(Wˆx)=f(ˆx)+λΦdWˆx(Wˆx)f(ˉx)+λΦdWˆx(Wˉx)f(ˉx)+λWˉx0,ˉxBϱ(ˆx),

    which leads to a contradiction. Thus, the local minimizer of (1.3) is the local minimizer of (1.7), that is to say, the ν-strong local minimizer of (1.3) is the lifted stationary point of (1.7).

    We use Table 1 to clearly demonstrate the relationship between (1.3) and (1.7).

    Table 1.  Link between problems (1.3) and (1.7).

     | Show Table
    DownLoad: CSV

    The main content of this section is to find the lifted stationary point of (1.7). Due to the existence of matrix W, we cannot express the explicit solution using the proximal gradient method. We first approximate f by a smoothing function and propose some preliminary theories on the smoothing methods; the second section proposes our algorithm; and the third section conducts a convergence analysis on the proposed algorithm for solving (1.7).

    Throughout this paper, we approximate the loss function f by a smoothing function ˜f in (1.7). When it is clear from the context, the derivative of ˜f(x,μ) with respect to x is simply denoted as ˜f(x,μ). We denote

    ˜Wd(x,μ):=˜f(x,μ)+λΦd(Wx),˜W(x,μ):=˜f(x,μ)+λΦ(Wx),

    where smoothing parameter μ>0 and dDp. For any fixed μ>0 and dDp, ˜Wd(x,μ) is nonsmooth and convex, and ˜W(x,μ) is nonsmooth and nonconvex. Due to

    Φ(Wx)=mindDpΦd(Wx),xΩ,

    we obtain

    ˜Wd(x,μ)˜W(x,μ),dDp,xΩ,μ(0,ˉμ]. (3.1)

    The following definition describes some theories about the smoothing function ˜f, which is frequently used in the proof of convergence analysis.

    Definition 3.1 We call ˜f:Rn×[0,ˉμ]R with ˉμ>0 a smoothing function of the convex function f in (1.7), if ˜f(,μ) is continuously differentiable in Rn for any fixed μ>0 and satisfies the following conditions:

    (ⅰ) limzx,μ0˜f(z,μ)=f(x),xΩ;

    (ⅱ) (convexity) ˜f(x,μ) is convex with respect to x in Ω for any fixed μ>0;

    (ⅲ) (gradient consistency) {limzx,μ0z˜f(z,μ)}f(x),xΩ;

    (ⅳ) (Lipschitz continuity with respect to μ) there exists a positive constant κ such that

    |˜f(x,μ2)˜f(x,μ1)|κ|μ1μ2|,xΩ,μ1,μ2[0,ˉμ];

    (ⅴ) (Lipschitz continuity with respect to x) there exists a constant L>0 such that for any μ(0,ˉμ],x˜f(,μ) is Lipschitz continuous on Ω with Lipschitz constant Lμ1.

    By virtue of Definition 3.1-(ⅳ), we obtain that

    |˜f(x,μ)f(x)|κμ,xΩ,0<μˉμ. (3.2)

    Next, we aim to solve the following problem with μ>0 and vector dDp

    minxΩ˜Wd(x,μ)=˜f(x,μ)+λΦd(Wx), (3.3)

    by introducing an approximation of ˜Wd(,μ) around a given point z as follows:

    Gd,γ(x,z,μ)=˜f(z,μ)+xz,˜f(z,μ)+12γμ1xz2+λΦd(Wx) (3.4)

    with a constant γ>0. Φd(Wx) is convex with respect to x for any fixed dDp, function Gd,γ(x,z,μ) is a strongly convex function with respect to x for any fixed d,γ,z and μ. Then, we solve the following problem

    minxΩGd,γ(x,z,μ)

    to find the approximate solution of (3.3).

    In this subsection, we propose a new algorithm (see Algorithm 1) for finding a lifted stationary point of (1.7). Specially, since W is a general linear mapping and Φ(Wx) is an inseparable function, which makes the smoothing proximal gradient algorithm [9] cannot explicitly solve a subproblem. The proposed algorithm combines the smoothing method and the smoothing gradient descent algorithm, so we call it the smoothing gradient descent (SGD) algorithm. We use the SGD algorithm to obtain approximate solutions of the subproblem. Let

    Ps={kN:μk+1μk},

    and denote psr the rth smallest number in Ps. Then, we can obtain the following updating method of {μk}

    μk=μpsr+1=μ0(psr+1)σ,psr+1kpsr+1, (3.5)

    which will be used in the proof of Lemmas 3.2 and 3.4.

    Algorithm 1: Smoothing Gradient Descent (SGD) algorithm
    Require: Take x1=x0Ω and μ1=μ0(0,ˉμ]. Give parameters ρ>1, σ>12, α>0 and 0<γ_<ˉγ. Set k=0.
    1: while a termination criterion is not met do
    2:      Step 1.Choose γk[γ_,ˉγ] and let dk, where d^{W\mathbf{x}^k} is defined in (2.5).
    3:      Step 2.
    3:      2a) Compute
    4:
    {\hat{\mathbf{x}}}^{k+1} = \arg\min\limits_{\mathbf{x}\in\Omega}\; G_{d^k, \gamma^k}(\mathbf{x}, \mathbf{x}^{k}, \mu_k).           (3.6)
    4:      2b) If {\hat{\mathbf{x}}}^{k+1} satisfies
    5:
    \widetilde{\mathcal{W}}^{d^k}({\hat{\mathbf{x}}}^{k+1}, \mu_k)\leq G_{d^k, \gamma^k}({\hat{\mathbf{x}}}^{k+1}, \mathbf{x}^{k}, \mu_k).           (3.7)
    6:          Set
    7:
    \mathbf{x}^{k+1} = {\hat{\mathbf{x}}}^{k+1}           (3.8)
    8:          and go to Step 3. Otherwise, let \gamma_k = \rho\gamma_k and return to 2a).
    9:      Step 3. If
    10:
    \widetilde{\mathcal{W}}(\mathbf{x}^{k+1}, \mu_k)+\kappa\mu_k-\widetilde{\mathcal{W}}(\mathbf{x}^{k}, \mu_{k-1})-\kappa\mu_{k-1}\leq-\alpha\mu_k^2,           (3.9)
    11:              set \mu_{k+1} = \mu_k , otherwise, set
    12:
    \mu_{k+1} = \frac{\mu_0}{(k+1)^\sigma}.           (3.10)
    13:              Increment k by one and return to Step 1.
    14: end while

    Lemma 3.2. The proposed SGD algorithm is well-defined, and the sequences \{\mathbf{x}^k\} , \{\gamma^k\} and \{\mu_k\} generated by it own the following properties:

    (ⅰ) \{\mathbf{x}^k\}\subseteq\Omega and \{\gamma_k\}\subseteq[\underline{\gamma}, \max\{\overline{\gamma}, \rho L\}];

    (ⅱ) there are infinite elements in \mathcal{P}^s and \lim_{k\rightarrow \infty}\mu_k = 0.

    Proof. (ⅰ) By organizing (3.7), we can obtain

    \tilde{f}(\hat{\mathbf{x}}^{k+1},\mu_k)\leq\tilde{f}({\mathbf{x}}^{k},\mu_k)+\langle\nabla\tilde{f}({\mathbf{x}}^{k},\mu_k),\hat{\mathbf{x}}^{k+1}-\mathbf{x}^k\rangle+\frac{1}{2}\gamma_k\mu_k^{-1}\|\hat{\mathbf{x}}^{k+1}-\mathbf{x}^k\|^2.

    According to Definition 3.1-(ⅴ), (3.7) holds when \gamma_k\geq L . Thus the updating of \gamma_k in Step 2 is at most \log_\rho(\frac{L}{\underline{\gamma}})+1 times at each iteration. Hence, the SGD algorithm is well-defined, and we have that \gamma_k\leq\max\{\overline{\gamma}, \rho L\}, \forall k\in\mathbb{N}. From (3.8), it is easy to verify that \mathbf{x}^{k+1}\in\Omega by \mathbf{x}^{k}\in\Omega and \hat{\mathbf{x}}^{k+1}\in\Omega .

    (ⅱ) Since \{\mu_k\} is non-increasing, to prove (ⅱ), we assume that \lim_{k\rightarrow \infty}\mu_k = \hat{\mu} > 0 by contradiction. If \{\mu_k\} converges to a non-zero value, then the iteration of (3.10) is finite, which means that there exists \mathrm{K}\in\mathbb{N} such that \mu_k = \hat{\mu}, \forall k\geq\mathrm{K} . Substituting \hat{\mu} into (3.9), we obtain

    \widetilde{\mathcal{W}}(\mathbf{x}^{k+1},\mu_k)+\kappa\mu_k-\widetilde{\mathcal{W}}(\mathbf{x}^{k},\mu_{k-1})-\kappa\mu_{k-1}\leq-\alpha\hat{\mu}^2, \forall k\geq\mathrm{K}+1.

    By the above inequality, we have

    \begin{equation} \begin{aligned} \lim\limits_{k\rightarrow \infty}\widetilde{\mathcal{W}}(\mathbf{x}^{k+1},\mu_k)+\kappa\mu_k = -\infty. \end{aligned} \end{equation} (3.11)

    However, by \{\mathbf{x}^k\}\subseteq\Omega , (3.2) and Theorem 2.5, then

    \begin{equation} \begin{aligned} \widetilde{\mathcal{W}}(\mathbf{x}^{k+1},\mu_k)+\kappa\mu_k\geq\mathcal{W}(\mathbf{x}^{k+1})\geq\min\limits_{\mathbf{x}\in\Omega}\mathcal{W}(\mathbf{x}) = \min\limits_{\mathbf{x}\in\Omega}\mathcal{W}_{\ell_0}(\mathbf{x}),\; \; \forall k\geq\mathrm{K}, \end{aligned} \end{equation} (3.12)

    (3.11) and (3.12) are contradictory; (ⅱ) holds.

    Lemma 3.3. For any k\in\mathbb{N} , we have

    \begin{equation} \begin{aligned} \widetilde{\mathcal{W}}(\mathbf{x}^{k+1},\mu_k)-\widetilde{\mathcal{W}}(\mathbf{x}^{k},\mu_k)\leq-\frac{1}{2}\gamma_k\mu_k^{-1}\|\mathbf{x}^{k+1}-\mathbf{x}^k\|^2, \end{aligned} \end{equation} (3.13)

    which implies \{\widetilde{\mathcal{W}}(\mathbf{x}^{k+1}, \mu_k)+\kappa\mu_k\} is non-increasing and \lim\limits_{k\rightarrow \infty}\widetilde{\mathcal{W}}(\mathbf{x}^{k+1}, \mu_k) = \lim\limits_{k\rightarrow \infty}\mathcal{W}(\mathbf{x}^{k}).

    Proof. Since G_{d^k, \gamma_k}(\mathbf{x}, \mathbf{x}^k, \mu_k) is strongly convex with modulus \gamma_k\mu_k^{-1}, we have

    \begin{equation} \begin{aligned} G_{d^k,\gamma_k}(\mathbf{x},\mathbf{x}^k,\mu_k)\geq G_{d^k,\gamma_k}(\hat{\mathbf{x}}^{k+1},\mathbf{x}^k,\mu_k)+\langle\nabla G_{d^k,\gamma_k}(\hat{\mathbf{x}}^{k+1},\mathbf{x}^k,\mu_k),\mathbf{x}-\hat{\mathbf{x}}^{k+1}\rangle\\ +\frac{1}{2}\gamma_k\mu_k^{-1}\|\hat{\mathbf{x}}^{k+1}-\mathbf{x}\|^2, \end{aligned} \end{equation} (3.14)

    using the definition of \hat{\mathbf{x}}^{k+1} in (3.6) and \mathbf{x}^{k+1} = \hat{\mathbf{x}}^{k+1} when (3.7) holds, we obtain

    G_{d^k,\gamma_k}(\mathbf{x}^{k+1},\mathbf{x}^{k},\mu_k)\leq G_{d^k,\gamma_k}(\mathbf{x},\mathbf{x}^k,\mu_k)-\frac{1}{2}\gamma_k\mu_k^{-1}\|\mathbf{x}^{k+1}-\mathbf{x}\|^2,\forall \mathbf{x}\in\Omega.

    By the definition of function G_{d^k, \gamma_k} given in (3.4), organizing the inequalities above, we have

    \begin{equation} \begin{aligned} \lambda\Phi^{d^k}(W\mathbf{x}^{k+1})\leq&\lambda\Phi^{d^k}(W\mathbf{x})+\langle \mathbf{x}-\mathbf{x}^{k+1},\nabla\tilde{f}(\mathbf{x}^k,\mu_k)\rangle\\ &+\frac{1}{2}\gamma_k\mu_k^{-1}\|\mathbf{x}-\mathbf{x}^k\|^2 -\frac{1}{2}\gamma_k\mu_k^{-1}\|\mathbf{x}^{k+1}-\mathbf{x}^k\|^2\\ &-\frac{1}{2}\gamma_k\mu_k^{-1}\|\mathbf{x}^{k+1}-\mathbf{x}\|^2. \end{aligned} \end{equation} (3.15)

    Moreover, (3.7) can be written as

    \begin{equation} \begin{aligned} \widetilde{\mathcal{W}}^{d^k}(\mathbf{x}^{k+1},\mu_k)\leq&\tilde{f}(\mathbf{x}^k,\mu_k)+\langle \mathbf{x}^{k+1}-\mathbf{x}^k,\nabla\tilde{f}(\mathbf{x}^k,\mu_k)\rangle\\ &+\frac{1}{2}\gamma_k\mu_k^{-1}\|\mathbf{x}^{k+1}-\mathbf{x}^k\|^2 +\lambda\Phi^{d^k}(W\mathbf{x}^{k+1}). \end{aligned} \end{equation} (3.16)

    Summing up (3.15) and (3.16), we obtain that

    \begin{equation} \begin{aligned} \widetilde{\mathcal{W}}^{d^k}(\mathbf{x}^{k+1},\mu_k)\leq&\tilde{f}(\mathbf{x}^k,\mu_k)+\lambda\Phi^{d^k}(W\mathbf{x})+\langle \mathbf{x}-\mathbf{x}^k,\nabla\tilde{f}(\mathbf{x}^k,\mu_k)\rangle\\ &+\frac{1}{2}\gamma_k\mu_k^{-1}\|\mathbf{x}-\mathbf{x}^k\|^2-\frac{1}{2}\gamma_k\mu_k^{-1}\|\mathbf{x}^{k+1}-\mathbf{x}\|^2,\forall \mathbf{x}\in\Omega. \end{aligned} \end{equation} (3.17)

    For a fixed \mu > 0 , the convexity of \tilde{f}(\mathbf{x}, \mu) with respect to \mathbf{x} indicates

    \begin{equation} \begin{aligned} \tilde{f}(\mathbf{x}^k,\mu_k)+\langle \mathbf{x}-\mathbf{x}^k,\nabla\tilde{f}(\mathbf{x}^k,\mu_k)\rangle\leq\tilde{f}(\mathbf{x},\mu_k),\forall \mathbf{x}\in\Omega. \end{aligned} \end{equation} (3.18)

    Combining (3.17) and (3.18) and utilizing the definition of \widetilde{\mathcal{W}}^{d^k} , one has

    \begin{equation} \begin{aligned} \widetilde{\mathcal{W}}^{d^k}(\mathbf{x}^{k+1},\mu_k)\leq\widetilde{\mathcal{W}}^{d^k}(\mathbf{x},\mu_k)+\frac{1}{2}\gamma_k\mu_k^{-1}\|\mathbf{x}-\mathbf{x}^k\|^2-\frac{1}{2}\gamma_k\mu_k^{-1}\|\mathbf{x}^{k+1}-\mathbf{x}\|^2,\\ \forall \mathbf{x}\in\Omega. \end{aligned} \end{equation} (3.19)

    Letting \mathbf{x} = \mathbf{x}^k in (3.19) and by d^k = d^{{W\mathbf{x}}^k} , we obtain

    \begin{equation} \begin{aligned} \widetilde{\mathcal{W}}^{d^k}(\mathbf{x}^{k+1},\mu_k)+\frac{1}{2}\gamma_k\mu_k^{-1}\|\mathbf{x}^{k+1}-\mathbf{x}^k\|^2\leq\widetilde{\mathcal{W}}(\mathbf{x}^{k},\mu_k). \end{aligned} \end{equation} (3.20)

    Because \widetilde{\mathcal{W}}^{d^k}(\mathbf{x}^{k+1}, \mu_k)\geq\widetilde{\mathcal{W}}(\mathbf{x}^{k+1}, \mu_k), (3.20) leads to (3.13). Due to Definition 3.1-(iv), we have

    \tilde{f}(\mathbf{x}^k,\mu_{k-1})\geq\tilde{f}(\mathbf{x}^k,\mu_k)-\kappa(\mu_{k-1}-\mu_{k}),

    then, it follows that

    \widetilde{\mathcal{W}}(\mathbf{x}^{k},\mu_k)\leq\widetilde{\mathcal{W}}(\mathbf{x}^{k},\mu_{k-1})+\kappa(\mu_{k-1}-\mu_k),

    by (3.13), we obtain

    \begin{equation} \begin{aligned} \widetilde{\mathcal{W}}(\mathbf{x}^{k+1},\mu_k)+\kappa\mu_k+\frac{1}{2}\gamma_k\mu_k^{-1}\|\mathbf{x}^{k+1}-\mathbf{x}^k\|^2\leq\widetilde{\mathcal{W}}(\mathbf{x}^{k},\mu_{k-1})+\kappa\mu_{k-1}, \end{aligned} \end{equation} (3.21)

    (3.21) implies the non-increasing property of \{\widetilde{\mathcal{W}}(\mathbf{x}^{k+1}, \mu_k)+\kappa\mu_k\}. This result and (3.12) ensure the existence of \lim_{k\rightarrow \infty}\widetilde{\mathcal{W}}(\mathbf{x}^{k+1}, \mu_k)+\kappa\mu_k . By virtue of \lim_{k\rightarrow \infty}\mu_k = 0 and Definition 3.1-(i), we obtain

    \lim\limits_{k\rightarrow \infty}\widetilde{\mathcal{W}}(\mathbf{x}^{k+1},\mu_k) = \lim\limits_{k\rightarrow \infty}\mathcal{W}(\mathbf{x}^k).

    The proof is completed.

    Lemma 3.4. The following statements hold:

    (ⅰ) \sum_{k = 0}^\infty\gamma_k\mu_k^{-1}\|\mathbf{x}^{k+1}-\mathbf{x}^k\|^2\leq2(\mathcal{W}(\mathbf{x}^0, \mu_{-1})+\kappa\mu_{-1}-\min_\Omega\mathcal{W});

    (ⅱ) \sum_{k = 0}^\infty\mu_k^2\leq\Lambda with \Lambda = \frac{1}{\alpha}(\widetilde{\mathcal{W}}(\mathbf{x}^0, \mu_{-1})+\kappa\mu_{-1}-\min_{\mathbf{x}\in\Omega}\mathcal{W}(\mathbf{x}))+\frac{2\mu_0^2\sigma}{2\sigma-1} < \infty ;

    Proof. (ⅰ) Recalling (3.21), for all k\in\mathbb{N} , we obtain

    \begin{equation} \begin{aligned} \gamma_k\mu_k^{-1}\|\mathbf{x}^{k+1}-\mathbf{x}^k\|^2\leq2(\widetilde{\mathcal{W}}(\mathbf{x}^k,\mu_{k-1})+\kappa\mu_{k-1}-\widetilde{\mathcal{W}}(\mathbf{x}^{k+1},\mu_{k})-\kappa\mu_{k}). \end{aligned} \end{equation} (3.22)

    Now adding up the above inequality over k = 0, \ldots, \mathrm{K} , it gives

    \begin{equation} \begin{aligned} \sum\limits_{k = 0}^\mathrm{K}\gamma_k\mu_k^{-1}\|\mathbf{x}^{k+1}-\mathbf{x}^k\|^2\leq2(\widetilde{\mathcal{W}}(\mathbf{x}^0,\mu_{-1})+\kappa\mu_{-1}-\widetilde{\mathcal{W}}(\mathbf{x}^{\mathrm{K}+1},\mu_{\mathrm{K}})-\kappa\mu_{\mathrm{K}}). \end{aligned} \end{equation} (3.23)

    By letting \mathrm{K} in (3.22) tend to infinity and along with (3.12), we obtain (ⅰ).

    (ⅱ) From (3.5), we have

    \begin{equation} \begin{aligned} \sum\limits_{k\in\mathcal{P}^s}\mu_k^2 = \sum\limits_{r = 1}^\infty\frac{\mu_0^2}{(p_r^s+1)^{2\sigma}}\leq\sum\limits_{k = 1}^\infty\frac{\mu_0^2}{k^{2\sigma}}\leq\frac{2\mu_0^2\sigma}{2\sigma-1}, \end{aligned} \end{equation} (3.24)

    where p_r^s is the r th smallest element in \mathcal{P}^s . When k\notin\mathcal{P}^s , (3.9) gives

    \alpha\mu_k^2\leq\widetilde{\mathcal{W}}(\mathbf{x}^k,\mu_{k-1})+\kappa\mu_{k-1}-\widetilde{\mathcal{W}}(\mathbf{x}^{k+1},\mu_{k})-\kappa\mu_{k},

    which together with the non-increasing property of \{\widetilde{\mathcal{W}}(\mathbf{x}^{k+1}, \mu_k)+\kappa\mu_k\} and (3.12) implies

    \begin{equation} \begin{aligned} \sum\limits_{k\notin\mathcal{P}^s}\mu_k^2\leq\frac{1}{\alpha}(\widetilde{\mathcal{W}}(\mathbf{x}^0,\mu_{-1})+\kappa\mu_{-1}-\min\limits_\Omega\mathcal{W}). \end{aligned} \end{equation} (3.25)

    Combining (3.24) and (3.25), the proof of (ii) is completed.

    Theorem 3.5. If there is an accumulation point in \{\mathbf{x}^k:k\in\mathcal{P}^s\} , then the accumulation point is a lifted stationary point of (1.7).

    Proof. Since (3.9) fails for k\in\mathcal{P}^s , by rearranging (3.21), we obtain that

    \gamma_{k}\mu_{k}^{-1}\|\mathbf{x}^{k+1}-\mathbf{x}^{k}\|^2\leq2\alpha\mu_{k}^2,

    which gives

    \|\mathbf{x}^{k+1}-\mathbf{x}^{k}\|\leq\sqrt{2\alpha\gamma_{k}^{-1}\mu_{k}^3}.

    Thus,

    \gamma_{k}\mu_{k}^{-1}\|\mathbf{x}^{k+1}-\mathbf{x}^{k}\|\leq\sqrt{2\alpha\gamma_{k}\mu_{k}},

    which together with \lim_{k\rightarrow \infty}\mu_{k} = 0 and \{\gamma_{k}\}\subseteq[\underline{\gamma}, \max\{\bar{\gamma}, \rho L\}] implies

    \begin{equation} \begin{aligned} \lim\limits_{k\rightarrow \infty}\gamma_{k}\mu_{k}^{-1}\|\mathbf{x}^{k+1}-\mathbf{x}^{k}\| = 0 \; \; \mathrm{and} \; \lim\limits_{k\rightarrow \infty}\|\mathbf{x}^{k+1}-\mathbf{x}^k\| = 0. \end{aligned} \end{equation} (3.26)

    Let \bar{\mathbf{x}} be an accumulation point of \{\mathbf{x}^k\}_{k\in\mathcal{P}^s} , (3.26) indicates that \{\mathbf{x}^k\} exists a subsequence \{\mathbf{x}^{k_t}\}_{k_t\in\mathcal{P}^s} converges to \bar{\mathbf{x}} . Similar analysis can be given for the case that k_t\in\mathcal{P}^s implies

    \begin{equation} \begin{aligned} \lim\limits_{t\rightarrow \infty}\gamma_{k_t}\mu_{k_t}^{-1}\|\mathbf{x}^{k_t+1}-\mathbf{x}^{k_t}\| = 0 \; \; \mathrm{and} \; \lim\limits_{t\rightarrow \infty}\mathbf{x}^{k_t+1} = \bar{\mathbf{x}}. \end{aligned} \end{equation} (3.27)

    Recalling \mathbf{x}^{k_t+1} = \hat{\mathbf{x}}^{k_t+1} defined in (3.6) and by its first-order necessary optimality condition, we have

    \begin{equation} \begin{aligned} &\langle\nabla\tilde{f}(\mathbf{x}^{k_t},\mu_{k_t})+\gamma_{k_t}\mu_{k_t}^{-1}(\mathbf{x}^{k_t+1}-\mathbf{x}^{k_t})+\lambda\zeta^{k_t},\mathbf{x}-\mathbf{x}^{k_t+1}\rangle\geq0,\\ & \ \ \ \ \forall\zeta^{k_t}\in\partial\Phi^{d^{k_t}}(W\mathbf{x}^{k_t+1}),\mathbf{x}\in\Omega. \end{aligned} \end{equation} (3.28)

    Since the elements in \{d^{k_t}:t\in\mathbb{N}\} are finite and \lim_{t\rightarrow \infty}\mathbf{x}^{k_t+1} = \bar{\mathbf{x}} , there exists a subsequence of \{k_t\} , denoted as \{k_{t_j}\} , and \bar{d}\in\mathbb{D}^p(W\bar{\mathbf{x}}) such that d^{k_{t_j}} = \bar{d}, \forall j\in\mathbb{N} . By the upper semicontinuity of \partial\Phi^{\bar{d}} and \lim_{j\rightarrow \infty}\mathbf{x}^{k_{t_j}+1} = \bar{\mathbf{x}} , it gives

    \begin{equation} \begin{aligned} \left\{\lim\limits_{j\rightarrow \infty}\zeta^{k_{t_j}}:\zeta^{k_{t_j}}\in\partial\Phi^{d^{k_{t_j}}}(W\mathbf{x}^{k_{t_j}+1})\right\}\subseteq\partial\Phi^{\bar{d}}(W\bar{\mathbf{x}}). \end{aligned} \end{equation} (3.29)

    Along with the subsequence \{k_{t_j}\} and letting j\rightarrow \infty in (3.28), from Definition 3.1-(ⅲ), (3.27) and (3.29), we obtain that there exist \bar{\xi}\in\partial f(\bar{\mathbf{x}}) and \bar{\zeta}^{\bar{d}}\in\partial\Phi^{\bar{d}}(W\bar{\mathbf{x}}) such that

    \begin{equation} \begin{aligned} \langle\bar{\xi}+\lambda\bar{\zeta}^{\bar{d}},\mathbf{x}-\bar{\mathbf{x}}\rangle\geq0,\forall \mathbf{x}\in\Omega. \end{aligned} \end{equation} (3.30)

    By \bar{d}\in\mathbb{D}^p(W\bar{\mathbf{x}}) , thanks to the convexity of f+\lambda\Phi^{\bar{d}} , (3.30) implies

    f(\mathbf{x})+\lambda\Phi^{\bar{d}}(W\mathbf{x})-f(\bar{\mathbf{x}})-\lambda\Phi^{\bar{d}}(W\bar{\mathbf{x}}) \geq\langle\bar{\xi}+\lambda\bar{\zeta}^{\bar{d}},\mathbf{x}-\bar{\mathbf{x}}\rangle\geq0, \forall \mathbf{x}\in\Omega,

    which implies that \bar{\mathbf{x}} is a lifted stationary point of (1.7).

    The purpose of this part is to test and verify the theoretical results and the properties of the SGD algorithm by the numerical experiments. We present Examples 1 and 2, which are respectively an under-determined linear regression problem and an over-determined censored regression problem. Especially, the process of solving subproblem (3.6) is very similar to the algorithm process of solving the LASSO problem.

    All experiments are performed in MATLAB 2016a on a Lenovo PC with an Intel(R) Core(TM) i5-8250U CPU @1.60GHz 1801 Mhz and 8GB RAM. In the following examples, stopping criterion is set as

    \begin{equation} \begin{aligned} \mathrm{number \; of\; iterations} \leq \mathbf{\mathtt{Maxiter}}\; \; \; \mathrm{or}\; \; \; \mu_k\leq\varepsilon. \end{aligned} \end{equation} (4.1)

    We stop the proposed algorithm if the number of iterations exceeds \mathbf{\mathtt{Maxiter}} or the smoothing parameter is less than \varepsilon . Denote \bar{\mathbf{x}} the output of iterate \mathbf{x}^k . Set the fixed parameter \alpha = 1 throughout the numerical experiments.

    Example 4.1. (Linear regression problem) Linear regression problems have been widely used in information theory[1], signal processing [35,36] and image restoration[6,36]. As pointed out in [20], \mathcal{\ell}_1 loss function is nonsmooth, but more robust and has stronger capability of outlier-resistance than the least squares loss function in the linear regression problems. Then we consider the following \mathcal{\ell}_0 regularized linear regression problem with \mathcal{\ell}_1 loss function:

    \begin{equation} \begin{aligned} \min\limits_ {\mathbf{x}\in\Omega} \; \; \; \mathcal{W}_{\mathcal{\ell}_0}(\mathbf{x}): = \frac{1}{m}\|A\mathbf{x}-b\|_1+\lambda\|W\mathbf{\mathbf{x}}\|_0, \end{aligned} \end{equation} (4.2)

    where A\in \mathbb{R}^{m\times n} with m = n , b\in\mathbb{R}^m . A smoothing function of the \mathcal{\ell}_1 loss function can be defined by

    \begin{equation} \begin{aligned} &\tilde{f}(\mathbf{x},\mu) = \frac{1}{m}\sum\limits_{i = 1}^m\tilde{\theta}(A_i\mathbf{x}-b_i,\mu)\; \; with \; \; \tilde{\theta}(s,\mu) = \begin{cases} |s| & if \; |s| > \mu,\\ \frac{s^2}{2\mu}+\frac{\mu}{2} & if \; |s|\leq\mu.\\ \end{cases} \end{aligned} \end{equation} (4.3)

    Denote s the \mathcal{\ell}_0 norm of true solution \mathbf{x}^* , i.e., \|W\mathbf{x}^*\|_0 = s . For the given positive integers m, n and s , the data are generated by

    {\mathbf{\mathtt{W = randn(p,n)}}; \mathbf{\mathtt{B = randn(n,m)}}; \mathbf{ \mathtt{A = orth(B)'}}; \; \mathbf{\mathtt{b = A* \mathbf{x}^* +0.01*randn(m,1)}}.}

    In the algorithm, we set the parameters as below: \underline{\gamma} = \overline{\gamma} = 1, \; \mu_0 = 3.533, \; \mathbf{\mathtt{Maxiter}} = 10^3, \; \nu = 35.6014, \; \sigma = 3.0003, \; \rho = 1.0001, \; \kappa = \frac{1}{2}. Generate A, b and \mathbf{x}^* with m = n = 45 , p = 45 and s = 2 , set \lambda = 10^{-3} in (4.2) and \varepsilon = 10^{-3} in the stopping criterion (4.1). We set \mathbf{x}_0 = ones(n, 1) . Figure 1 shows the numerical results. Figure 1 plots \mathbf{x}^* and \overline{\mathbf{x}} , where \mathbf{x}^* and \overline{\mathbf{x}} denote the original signal (which can also be expressed as true solution) and the output of iterate \mathbf{x}^k from the SGD algorithm. From Figure 1, we can see that the output of \mathbf{x}^k is very close to the original generated signal.

    Figure 1.  Digital experiment of the SGD algorithm in Example 4.1 under the first form of W .

    Now we use another form of matrix W to solve Example 4.1:

    W = \begin{pmatrix} 0& 0 & 0 & \cdots &0& 0 \\ -3 & 1 & 0 & \cdots &0& 0 \\ 0 &-3 & 1 & \cdots &0& 0 \\ \vdots & \vdots & \vdots & \ddots & \vdots& \vdots\\ 0 & 0 & 0 & \cdots &-3& 1 \end{pmatrix}_{p \times n}.

    Set \underline{\gamma} = \overline{\gamma} = 1, \; \mu_0 = 3.533, \; \mathbf{\mathtt{Maxiter}} = 10^3, \; \nu = 36, \; \sigma = 7, \; \rho = 1.0001 \; \mathrm{and}\; \kappa = \frac{1}{2}. We randomly generate the data as follows:

    {\mathbf{\mathtt{B = randn(n,m)}};\mathbf{ \mathtt{A = orth(B)'}}; \; \mathbf{\mathtt{b = A* \mathbf{x}^* +0.01*randn(m,1)}}.}

    We run numerical experiments with (m, n, p, s) = (45, 45, 45, 2). Set \lambda = 10^{-3} in (4.2) and \varepsilon = 10^{-3} in the stopping criterion (4.1). We define \mathbf{x}_0 = randn(n, 1) . From Figure 2, we can see that the output of \mathbf{x}^k obtained by the SGD algorithm is also close to the true solution \mathbf{x}^* .

    Figure 2.  Digital experiment of SGD algorithm in Example 4.1 under the second form of W .

    The last special case of W is the penalty matrix in 1-dimensional Fused LASSO [39]:

    W = \begin{pmatrix} -1& 1 & 0 & \cdots &0& 0 \\ 0 & -1 & 1 & \cdots &0& 0 \\ 0 &0 & -1 & \cdots &0& 0 \\ \vdots & \vdots & \vdots & \ddots & \vdots& \vdots\\ 0 & 0 & 0 & \cdots &-1& 1 \end{pmatrix}_{p \times n}.

    Set \nu = 40, \; p = 45, \; n = 46 and \mathbf{x}_0 = ones(n, 1) . The remaining parameter values are the same as the previous situation (see Figure 3). From Figures 13, we can see that the output of \mathbf{x}^k obtained by the SGD algorithm is close to the true solution \mathbf{x}^* .

    Figure 3.  Digital experiment of SGD algorithm in Example 4.1 under the third form of W .

    Example 4.2 (Censored regression problem) The application of censored regression problems has been studied in machine learning[37], economics[38], biomedical, and technical systems. The following censored regression problem is also a typical class of composite sparse optimization problems with nonsmooth convex loss functions. Now we consider the following {\mathcal{\ell}_0} regularized censored regression problem:

    \min \limits_ {\mathbf{x}\in\Omega}\; \; \; \mathcal{W}_{\mathcal{\ell}_0}(\mathbf{x}): = \frac{1}{m}\|\max \{A\mathbf{x},0\}-b\|_1+\lambda\|W\mathbf{\mathbf{x}}\|_0,

    where A\in\mathbb{R}^{m\times n} and b\in\mathbb{R}^m . For the loss function in (1.5), a smoothing function of it can be defined by

    \begin{equation*} \tilde{f}(\mathbf{x},\mu) = \frac{1}{m}\sum\limits_{i = 1}^m\tilde{\theta}(\tilde{\phi}(A_i\mathbf{x},\mu)-b_i,\mu)\; \; with \; \; \tilde{\phi}(s,\mu) = \begin{cases} \max\{s,0\} & if \; |s| > \mu,\\ \frac{(s+\mu)^2}{4\mu} & if \; |s|\leq\mu.\\ \end{cases} \end{equation*}

    Set \varepsilon = 10^{-2} , \nu = 16.0009 , \lambda = 10^{-3} , \mu_0 = 10.8999 , \sigma = 4.0003 , \kappa = \frac{1}{2} , \rho = 1.2006 and \mathbf{x}_0 = randn(n, 1) . In this example, we run numerical experiments with (m, n, p, s) = (40, 40, 40, 2), we randomly generate the problem data as follows:

    \mathbf{\mathtt{A = randn(m,n)}}; \; \mathbf{\mathtt{W = randn(p,n)}}; \; \mathbf{ \mathtt{b = max(A* \mathbf{x}^* +0.01*randn(m,1),0)}}.

    The computational results of \mathbf{x}^* and \mathbf{x} are shown in Figure 4.

    Figure 4.  Numerical results of the SGD algorithm for Example 4.2.

    We use the following form of W to solve Example 4.2:

    W = \begin{pmatrix} 0& -2 & 0 & 0&\cdots &0& 0 \\ -2 & 1 & -3 & 0&\cdots &0& 0 \\ 0 &-3 & 1 & -4&\cdots &0& 0 \\ \vdots & \vdots & \vdots & \ddots & \vdots& \vdots& \vdots \\ 0 & 0 & 0 & 0&\cdots &1& -m\\ 0 & 0 & 0 & 0&\cdots &-m& 1 \end{pmatrix}_{p\times n}.

    Set \underline{\gamma} = \overline{\gamma} = 3, \; \mu_0 = 2, \; \mathbf{\mathtt{Maxiter}} = 10^3, \; \nu = 1.2200, \; \sigma = 2.6915, \; \rho = 1.0001 \; \mathrm{and}\; \kappa = 0.711. For the given positive integers m, n and s , the data are generated by

    \mathbf{\mathtt{A = randn(m,n)}}; \; \mathbf{\mathtt{b = A* \mathbf{x}^* +0.01*randn(m,1)}}.

    We run numerical experiments with (m, n, p, s) = (40, 40, 40, 2). Set \lambda = 10^{-3} in (4.2), \mathbf{x}_0 = ones(n, 1) and \varepsilon = 10^{-3} . From Figures 4 and 5, it can be seen that the output of \mathbf{x}^k is very close to the true solution.

    Figure 5.  Numerical results of the SGD algorithm for Example 4.2.

    We have intensively studied the composite sparse optimization problem consisting of the sum of a nonsmooth convex function and the {\mathcal{\ell}_0} penalty term of a matrix times the coefficient vector. Considering the original cardinality penalty problem and an exact continuous relaxation problem with capped- {\mathcal{\ell}_1} penalty, we have proved several novel and interesting results: the consistency between global minimizers of the relaxation problem and the original problem, and local minimizers of relaxation problems are local minimizers of the original problem. We propose the SGD algorithm based on the smoothing method and the smoothing gradient descent algorithm. Then SGD algorithm has been investigated from both a theoretical and an algorithmic point of view. So we prove that if the sequence generated by the algorithm has an accumulation point, then it is a lifted stationary point of relaxation problem. This well explains why the algorithm is expected to enjoy an appealing performance from the theoretical perspective, which is testified by the numerical experiments. Our initial numerical results confirm the predicted underlying theoretical results.

    Wei Yang: Conceptualization, writing-original draft, validation, software; Lili Pan: Conceptualization, supervision, funding acquisition, validation, software; Jinhui Wan: Software, methodology, data curation. All authors have read and approved the final version of the manuscript for publication.

    The work of the authors was supported by the National Natural Science Foundation of China grants 12271309.

    The authors declare no conflicts of interest.



    [1] R. Ma, D. Pei, Some curvature properties on Lorentzian generalized Sasakian-space-forms, Adv. Math. Phys., 2019 (2019). https://doi.org/10.1155/2019/5136758 doi: 10.1155/2019/5136758
    [2] D. E. Blair, T. Koufogiorgos, B. J. Papantoniou, Contact metric manifolds satisfying a nullity condition, Israel J. Math., 91 (1995), 189–214. https://doi.org/10.1007/BF02761646 doi: 10.1007/BF02761646
    [3] D. Perrone, Contact metric manifolds whose characteristic vector field is a harmonic vector field, Differ. Geom. Appl., 20 (2004), 367–378. https://doi.org/10.1016/j.difgeo.2003.12.007 doi: 10.1016/j.difgeo.2003.12.007
    [4] T. Koufogiorgos, C. Tsichlias, On the existence of a new class of contact metric manifolds, Canad. Math. Bull., 43 (2000), 440–447. https://doi.org/10.4153/CMB-2000-052-6 doi: 10.4153/CMB-2000-052-6
    [5] E. Boeckx, A class of locally \phi-symmetric contact metric spaces, Arch. Math., 72 (1999), 466–472. https://doi.org/10.1007/s000130050357 doi: 10.1007/s000130050357
    [6] E. Boeckx, A full classification of contact metric (k, \mu)-spaces, Illinois J. Math., 44 (2000), 212–219. http://projecteuclid.org/euclid.ijm/1255984960
    [7] E. Loiudice, A. Lotta, On the classification of contact metric (k, \mu)-spaces via tangent hyperquadric bundles, Math. Nachr., 291 (2018), 1851–1858. https://doi.org/10.1002/mana.201600442 doi: 10.1002/mana.201600442
    [8] D. S. Patra, A. Ghosh, On Einstein-type contact metric manifolds, J. Geom. Phys., 169 (2021), 10. https://doi.org/10.1016/j.geomphys.2021.104342 doi: 10.1016/j.geomphys.2021.104342
    [9] S. Tachibana, On almost-analytic vectors in certain almost-Hermitian manifolds, Tohoku Math. J., 11 (1959), 351–363. https://doi.org/10.2748/tmj/1178244533 doi: 10.2748/tmj/1178244533
    [10] S. Tachibana, On almost-analytic vectors in almost-Kählerian manifolds, Tohoku Math., 11 (1959) 247–265. https://doi.org/10.2748/tmj/1178244584 doi: 10.2748/tmj/1178244584
    [11] T. Hamada, Real hypersurfaces of complex space forms in terms of Ricci \ast-tensor, Tokyo J. Math., 25 (2002), 473–483. https://doi.org/10.3836/tjm/1244208866 doi: 10.3836/tjm/1244208866
    [12] G. Kaimakamis, K. Panagiotidou, *-Ricci solitons of real hypersurfaces in non-flat complex space forms, J. Geom. Phys., 86 (2014), 408–413.
    [13] X. Dai, Y. Zhao, U. Chand De, *-Ricci soliton on (\kappa, \mu)'-almost Kenmotsu manifolds, Open Math., 17 (2019), 874–882. https://doi.org/10.1515/math-2019-0056
    [14] K. Mandal, S. Makhal, *-Ricci solitons on three-dimensional normal almost contact metric manifolds, Lobachevskii J. Math., 40 (2019), 189–194. https://doi.org/10.1134/s1995080219020100 doi: 10.1134/s1995080219020100
    [15] Y. L. Li, A. H. Alkhaldi, A. Ali, L. I. Pișcoran, On the topology of warped product pointwise semi-slant submanifolds with positive curvature, Mathematics, 9 (2021), 3156. https://doi.org/10.3390/math9243156 doi: 10.3390/math9243156
    [16] Y. L. Li, A. Ali, R. Ali, A general inequality for CR-warped products in generalized Sasakian space form and its applications, Adv. Math. Phys., 2021 (2021), 5777554. https://doi.org/10.1155/2021/5777554 doi: 10.1155/2021/5777554
    [17] Y. L. Li, S. Dey, S. Pahan, A. Ali, Geometry of conformal \eta-Ricci solitons and conformal \eta-Ricci almost solitons on paracontact geometry, Open Math., 20 (2022), 1–20. https://doi.org/10.1515/math-2022-0443 doi: 10.1515/math-2022-0443
    [18] Venkatesha, D. M. Naik, H. A. Kumara, *-Ricci solitons and gradient almost *-Ricci solitons on Kenmotsu manifolds, Math. Slovaca., 69 (2019), 1447–1458. https://doi.org/10.1515/ms-2017-0321 doi: 10.1515/ms-2017-0321
    [19] P. Majhi, U. C. De, Y. J. Suh, *-Ricci solitons and Sasakian 3-manifolds, Publ. Math. Debrecen, 93 (2018), 241–252. https://doi.org/10.5486/pmd.2018.8245 doi: 10.5486/pmd.2018.8245
    [20] A. Ghosh, D. S. Patra, *-Ricci soliton within the frame-work of Sasakian and (\kappa, \mu)-contact manifold, Int. J. Geom. Methods Mod. Phys., 15 (2018), 1850120. https://doi.org/10.1142/S0219887818501207 doi: 10.1142/S0219887818501207
    [21] R. Ma, D. Pei, Reeb flow invariant *-Ricci operators on trans-Sasakian three-manifolds, Math. Slovaca, 71 (2021), 749–756. https://doi.org/10.1515/ms-2021-0017 doi: 10.1515/ms-2021-0017
    [22] S. Tanno, The topology of contact Riemannian manifolds, Illinois J. Math., 12 (1968), 700–717.
    [23] Y. Wang, H. Wu, Invariant vector fields on contact metric manifolds under D-homothetic deformation, AIMS Math., 5 (2020), 7711–7718. https://doi.org/10.3934/math.2020493 doi: 10.3934/math.2020493
    [24] D. E. Blair, Riemannian geometry of contact and symplectic manifolds, Progress in Mathematics, 2 Eds., Birkhäuser Boston, Inc., Boston, MA, 2010. https://doi.org/10.1007/978-0-8176-4959-3
  • Reader Comments
  • © 2022 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)
通讯作者: 陈斌, bchen63@163.com
  • 1. 

    沈阳化工大学材料科学与工程学院 沈阳 110142

  1. 本站搜索
  2. 百度学术搜索
  3. 万方数据库搜索
  4. CNKI搜索

Metrics

Article views(2455) PDF downloads(168) Cited by(1)

Other Articles By Authors

/

DownLoad:  Full-Size Img  PowerPoint
Return
Return

Catalog