Generalized $(f, \lambda)$-projection operator on closed nonconvex sets and its applications in reflexive smooth Banach spaces

1.   Introduction and preliminaries

Let $X$ be a Banach space with dual space $X^*$. The duality pairing between $X$ and $X^*$ will be denoted by $\langle\cdot, \cdot\rangle$. We denote by $\mathbb B$ and $\mathbb B_*$ the closed unit ball in $X$ and $X^*$, respectively. The normalized duality mapping $J:X _ \to ^ \to X^*$ is defined by

where $\|\cdot\|$ stands for both norms on $X$ and $X^*$. Many properties of $J$ are well known and we refer the reader, for instance, to the book ^[17].

Definition 1.1. For a fixed closed subset $S$ of $X$, a fixed function $f:S\to \mathbb R\cup\{\infty\}$, and a fixed $\lambda > 0$, we define the following functional: $G^V_{\lambda, f}:X^*\times S \to \mathbb R\cup\{\infty\}$

where $V(x^*, x) = \|x^*\|^2-2\langle x^*, x\rangle +\|x\|^2$. Using the functional $G^V_{\lambda, f}$, we define the generalized $(f, \lambda)$-projection on $S$ as follows:

Remark 1.1.

● If $f = 0$, then $\pi^{f, \lambda}_S$ coincides with the generalized projection $\pi_S$ introduced for closed convex sets in ^[2,3,13,14] and for closed nonconvex sets in ^[6,7].

● If $\lambda = \frac{1}{2}$, then $\pi^{f, \lambda}_S$ coincides with the $f$-generalized projection introduced for closed convex sets in ^[19,20].

● If $X$ is a Hilbert space and $f = 0$, then $\pi^{f, \lambda}_S$ coincides with the well-known metric projection $Proj_S$ in ^[11].

● If $X$ is a Hilbert space, the functional $x^*\mapsto e^V_{\lambda, S}f(x^*) = \inf\limits_{s\in S}G^V_{\lambda, f}(x^*, s)$ coincides with the Moreau envelope of $f$ with index $\lambda > 0$.

Motivated by the previous remarks, we are going to study the above concept of generalized projection in smooth Banach spaces. Our results will extend many existing works in the literature.

2.   $(f, \lambda)$-generalized projections on closed nonconvex sets

We commence by considering the following example, which serves to demonstrate that $\pi^{f, \lambda}_S(x^*)$ may be empty for nonconvex closed sets even for convex continuous functions $f$ in uniformly convex and uniformly smooth Banach spaces.

Example 2.1. Let $X = \ell_p$ ($p\ge 1$), ${\bf 0}_{X^*} = (0, \dots, 0, \dots)\in (l_p)^*$, and let $S: = \{e_1, e_2, \cdots, e_n, \cdots\}$ with $e_j = (0, \cdots, 0, \frac{j+1}{j}, 0, \cdots)$. Let $\lambda > 0$ and $f:X\to \mathbb R$ be defined as $f(x) = \|x\|-1$, then $S$ is a closed nonconvex subset in $X$ with $\pi^{f, \lambda}_S({\bf 0}_{X^*}) = \emptyset$.

Proof. Undoubtedly, the set $S$ is a closed nonconvex set and the function $f$ is convex continuous on $X$. Let us show that the $(f, \lambda)$-generalized projection of ${\bf 0}_{X^*}$ is empty. Let $x$ be any element of $S$, then for some $n\ge 1$, $x = e_n$ and

Then, for any $x\in S$, we have $f(x)+\frac{1}{2\lambda} V({\bf 0}_{X^*}; x) > \frac{1}{2\lambda}$; that is,

and so

This ensures that $\pi^{f, \lambda}_S({\bf 0}_{X^*}) = \emptyset$. Indeed, if $\pi^{f, \lambda}_S({\bf 0}_{X^*})\not = \emptyset$, then there exists some $\bar x\in \pi^{f, \lambda}_S({\bf 0}_{X^*})$, and so $\bar x = e_{n_0}$ for some $n_0\ge 1$, with

Hence,

and

which is not possible, and so $\pi^{f, \lambda}_S({\bf 0}_{X^*}) = \emptyset$. □

From the previous example, we see that even in uniformly convex and uniformly smooth Banach spaces, the $(f, \lambda)$-generalized projection $\pi^{f, \lambda}_S(x^*)$ may be empty for closed nonconvex sets. Thus, it is hopeless to get the conclusion of Theorem 2.1 in ^[19] and Theorem 3.1 in ^[20] saying that $\pi^{f, \lambda}_S(x^*)\not = \emptyset$, $\forall x^*\in X^*$ whenever $S$ is a closed convex set in reflexive Banach spaces. However, we are going to prove that for closed nonconvex sets, the set of points $x^*\in X^*$ for which $\pi^{f, \lambda}_S(x^*)\not = \emptyset$ is dense in $X^*$. We are going to prove our main result in the following theorem. First, we need to recall that an extended real-valued function $f$ defined on $X$ is said to be lower semicontinous (in short l.s.c.) on its domain provided that its epigraph $\mbox{epi }f: = \{(x, r)\in X\times \mathbb R: f(x)\le r\}$ is closed in $X\times \mathbb R$.

Theorem 2.1. Assume that $X$ is a reflexive Banach space with smooth dual norm, and let $S$ be any closed nonempty set of $X$ and $f:S\to \mathbb R\cup\{\infty\}$ be any l.s.c. function. Then, the set of points in $X^*$ admitting the unique $(f, \lambda)$-generalized projection on $S$ are dense in $X^*$; that is, for any $x^*\in X^*$, there exists $x^*_n \to x^*$ with $\pi^{f, \lambda}_S(x^*_n)\not = \emptyset, \forall n$.

Proof. By definition of $\pi^{f, \lambda}_S(x^*)$, we have

Observe that

with $\ell_{f, \lambda}(z^*): = \sup_{y\in X}\left\{\frac{1}{\lambda}\langle z^*; y\rangle -f(y)-\frac{\|y\|^2}{2\lambda}-\psi_S(y)\right\}$. Clearly, the function $\ell_{f, \lambda}$ is convex on a reflexive Banach space $X^*$, and so there exists a dense set $K$ in $X^*$ in which the Fréchet gradient exists; that is, $\nabla^F \ell_{f, \lambda}(x^*)$ exists for any $x^*\in K$. Now, we use the smoothness of the dual norm to get the existence of $\nabla^F \|\cdot\|(x^*)$ for any $x^*\ne 0$ in $X^*$. Thus, for any $x^*\in K$, the Fréchet derivative $\nabla^F e^V_{\lambda, S}f(x^*)$ exists with $\nabla^F e^V_{\lambda, S}f(x^*) = \frac{1}{2\lambda} \nabla^F \|\cdot\|^2(x^*)- \nabla^F \ell_{f, \lambda}(x^*)$. Fix any $\epsilon > 0$. Now, we use the definition of Fréchet differentiability to obtain some $\delta > 0$ such that

Fix any $t\in (0, \delta)$ and any $v^*\in \mathbb B_*$, then

By definition of the infimum in $e^V_{\lambda, S}f(x^*)$, there exists for any $n\ge 1$ some point $x_n\in S$ such that

Therefore,

Hence,

Using the fact that $\nabla^F \|\cdot\|^2(x^*) = 2J^*x^*$ to write

we find that

which gives

By taking $n\to \infty$ and $\epsilon \to 0$, we obtain the convergence of the sequence $(x_n)_n$ to $\bar x: = J^*x^*-\lambda \nabla^F e^V_{\lambda, S}f(x^*)$. Taking $n\to \infty$ in (2.2), we obtain $e^V_{\lambda, S}f(x^*) = f(\bar x)+\frac{1}{2\lambda}V(x^*, \bar x)$, which means that $\bar x\in \pi^{f, \lambda}_S(x^*)$.

Let us prove that the set $\pi^{f, \lambda}_{S}(x^*)$ reduces to the single limit $\bar x$. Let $\bar y\ne \bar x$ with $\bar y \in \pi^{f, \lambda}_{S}(x^*)$, so $e^V_{\lambda, S}f(x^*) = f(\bar y)+\frac{1}{2\lambda}V(x^*, \bar y)$. Proceeding as before with the constant minimizing sequence $y_n: = \bar y$, we get

and, hence; $\bar y = J^*x^*-\lambda \nabla^F d^V_S(x^*)$, which means that $\bar y = \bar x$. That is, $\pi^{f, \lambda}_S(x^*) = \{\bar x\}$, and so the proof is complete. □

3.   Further properties of $(f, \lambda)$-generalized projections on closed nonconvex sets

After proving the density of the domain of the operator $\pi_S^{f, \lambda}$ in the previous section, in this section we continue with the study of various important properties of the operator $\pi_S^{f, \lambda}$.

Theorem 3.1. Assume that $X$ is a reflexive Banach space with smooth dual norm. Let $S$ be a closed subset in $X$, $\lambda > 0$, and $f:X\to \mathbb R\cup\{\infty\}$ be a l.s.c function, then the following conclusions hold:

(1) If $f$ is bounded below on $S$, then the set $\pi_S^{f, \lambda}(x^*)$ is closed bounded in $X$, for any $x^*\in dom \, \pi_S^{f, \lambda}$;

(2) For any $x_1^*, x_2^*\in dom \, \pi_S^{f, \lambda}$ and any $x_1\in \pi_S^{f, \lambda}(x_1^*)$ and $x_2\in \pi_S^{f, \lambda}(x_2^*)$ we have

(3) For any $x^*\in dom \, \pi_S^{f, \lambda}$, any $\bar x\in \pi_S^{f, \lambda}(x^*)$, and any $t\in [0, 1)$ we have $\pi_S^{f, \lambda}(J(\bar x)+t(x^*-J(\bar x))) = \{\bar x\}$.

Proof. (1) Let $x^*\in dom \, \pi_S^{f, \lambda}$. Let $\alpha \in \mathbb R$ be the lower bound of $f$ on $S$. That is, $f(x) > \alpha$ for any $x\in S$. Then, for any $x\in \pi_S^{f, \lambda}(x^*)$,

which ensures that $\pi_S^{f, \lambda}(x^*)$ is bounded. Now, we prove that $\pi_S^{f, \lambda}(x^*)$ is closed. Let $\{x_n\}_n$ be a sequence in $\pi_S^{f, \lambda}(x^*)$ converging to a limit $\bar x$. We have to prove that $\bar x\in \pi_S^{f, \lambda}(x^*)$. By l.s.c. of $f$ and the continuity of $V$, we have

Thus, $\bar x\in \pi_S^{f, \lambda}(x^*)$.

(2) Let $x_i^*\in dom \, \pi_S^{f, \lambda}$ and $x_i\in \pi_S^{f, \lambda}(x_i^*)$ for $i = 1, 2$, then by definition of $e^V_{\lambda, S}f(x_i^*)$ we have

Adding these two inequalities yields

and, hence, by decomposition and rearrangement, we obtain

and the proof of (2) is complete.

(3) First, we prove that for any $\bar x\in \pi^{f, \lambda}_S(x^*)$ and any $t\in[0, 1)$, we have

Let $\bar x\in \pi^{f, \lambda}_S(x^*)$, then

and

First, observe that

We distinguish two cases:

Case 1. If $\langle x^*-J\bar x; y-\bar x \rangle \le \lambda [f(y)- f(\bar x)]$, then we have

Case 2. If $\langle x^*-J\bar x; y-\bar x \rangle > \lambda [f(y)- f(\bar x)]$, and since $t\in [0, 1)$, we have

and so

Thus,

and, hence,

Therefore, in both Case 1 and 2, we have

which is equivalent to

Hence,

that is, $\bar x\in \pi^{f, t\lambda }_S(J\bar x+t(x^*-J\bar x)), \mbox{ for any } t\in (0, 1)$.

Uniqueness. Now, let us prove the uniqueness. That is, for any $t\in (0, 1)$ and for any $x_t\in \pi^{f, t\lambda }_S(J\bar x+t(x^*-J\bar x))$, we have to prove that $x_t = \bar x$. To do that, fix $t\in(0, 1)$ and let $u_t: = J^*(J\bar x+t(x^*-J\bar x))$. Let $x_t\not = x$ with $x_t\in \pi^{f, t\lambda }_S(Ju_t)$, then

and so

A direct decomposition of the left hand side of this equality yields to

and, hence,

Thus,

On the other hand, we use the fact that $x_t\in S$ and $\bar x\in \pi^{f, \lambda}_S(x^*)$ to write $f(\bar x)+\frac{1}{2\lambda }V(x^*; \bar x)\le f(x_t)+\frac{1}{2\lambda }V(x^*; x_t)$, which, together with the previous equality (3.1), ensures that $V(J\bar x; x_t)\le0$ and $V(J\bar x; x_t) = 0$. This equality ensures that $x_t = \bar x$, thus ending the proof of the uniqueness and the proof of property (3) is achieved. □

4.   Applications to nonconvex variational problems

In this section, we assume that $X$ is a $p$-uniformly convex and $q$-uniformly smooth Banach space. That is, there exists a constant $c > 0$ such that

where $\delta_X$ is the moduli of convexity of $X$, and $\rho_X$ is the moduli of smoothness of $X$, given respectively by

and

For more details and properties of $p$-uniformly convex and $q$-uniformly smooth Banach space, we refer the reader to books ^[1,17].

Let $F:X _ \to ^ \to X^*$ be a set-valued mapping, $f:X\to \mathbb R\cup \{\infty\}$ be a l.s.c. function (not necessarily convex), and let $S\subset X$ be a nonempty closed set not necessarily convex. First, we define the following set:

Our aim is to use the main result in the previous section to study the following nonconvex variational problem:

This variational problem can be seen as a variant of the following nonconvex variational problem

where $N^P_S(\bar x)$ is the proximal normal cone associated with $S$ at $\bar x$. This problem, denoted as (4.2), has been introduced and studied in ^[8] for prox-regular sets in the Hilbert space setting. When $f = 0$, it is easy to check that any solution of (4.2) is also a solution of (4.1) for some $\lambda > 0$. Conversely, for any $\lambda > 0$, any solution of (4.1) is a solution of (4.2). If, in addition, $S$ is assumed to be convex, then both variational problems (4.2) and (4.1) coincide. Since the work in ^[8], numerous other works have explored and extended, in various ways, the existence of solutions for (4.2) (see ^{[5,9,10,15,18]} and the references therein). Furthermore, the variational problem (4.2) can be viewed as a reformulation of the well-known generalized equilibrium problem:

which is an extension of the equilibrium problems involving set-valued mappings overs closed sets (see, for instance, ^[4,12,16]). Consequently, our proposed variational problem (4.1) can be considered an appropriate extension of both (4.2) and (4.3).

First, we show that in the convex case, the variational problem (4.1) coincides with the usual variational inequality:

The variational inequality (4.4) is well studied in the convex case in ^[2,3,13,14].

Proposition 4.1. Whenever $S$ is a closed convex set and $f$ is a l.s.c. convex function, we have (4.1) $\Longleftrightarrow$ (4.4).

Proof. It is enough to prove that the set $N^{f, \lambda}_S(\bar x)$ can be characterized in the convex case as

First, we prove that the set $N^{f, \lambda}_S(\bar x)$ is a subset of the righthand side of the above equality. Let $x^*\in N^{f, \lambda}_S(\bar x)$, then by definition of $N^{f, \lambda}_S(\bar x)$, we have $\bar x\in \pi_S^{f, \lambda} (J(\bar x)+\lambda x^*)$. That is,

then

Hence,

Let any $x\in S$ and any $t\in (0, 1)$, then by convexity of $S$, we have that the point $y: = \bar x +t(x-\bar x)$ belongs to $S$. Thus,

Using the convexity of $f$ on $S$ to write

and by dividing by $t$ and taking the limit when $t\downarrow 0$, we obtain

Using the differentiability of $V$ with respect to the second variable and $\nabla_x V(z^*, x) = 2(J(x)-z^*)$, we obtain

Therefore, we get

Conversely, let $x^*\in X^*$ satisfy the last inequality, then

This gives

which means that $\bar x\in \pi_S^{\lambda, f}(J(\bar x)+\lambda x^*)$ and, hence, by definition of the set $N^{f, \lambda}_S(\bar x)$, we obtain $x^*\in N^{f, \lambda}_S(\bar x)$, and the proof is complete. □

Remark 4.1. When the set-valued mapping $F$ is defined as $F(x) = Ax-\xi$ with $A:X\to X^*$ and $\xi^*\in X^*$, the proposed variational problem in the convex case (4.4) corresponds to the following well known and well studied convex variational inequality (see, for instance, ^[2,13,14] and the references therein):

We suggest the following algorithm to solve the proposed nonconvex variational problem (4.1) under some natural and appropriate assumptions on $S$, $f$, and $F$.

Algorithm 1. Let $\delta_n\downarrow 0$ with $\delta_0$ be too small.

● Select $x_0\in S$, $y^*_0\in F(x_0)$ and $\lambda > 0$;

● For $n\ge 0$,

– Compute $z_{n+1}: = J^*(J(x_n)-\lambda y^*_n)$;

– Choose $u_{n+1}\in J^*(J(z_{n+1})+ \delta_n \mathbb B_*)$ with $\pi^{f, \lambda}_S(J(u_{n+1}))\not = \emptyset$;

– Choose $x_{n+1}: = \pi_S(J(u_{n+1}))$ and $y^*_{n+1}\in F(x_{n+1})$.

Since $S$ and $f$ are not necessarily convex, the $(f, \lambda)$-generalized projection $\pi^{f, \lambda}_S$ does not exist necessarily for any $x^*\in X^*\setminus J(S)$ (see Example 2.1). However, our previous algorithm is well defined, as we will prove in the following proposition.

Proposition 4.2. Assume that $X$ is uniformly convex and uniformly smooth Banach space. The above algorithm is well defined.

Proof. Let $n\ge 0$ and $x_n\in S$ with $y_n^*\in F(x_n)$. The point $z_{n+1}$ is well defined since $J$ and $J^*$ are well defined and one-to-one, because the space $X$ is assumed to be uniformly convex and uniformly smooth. Now, since the $(f, \lambda)$-generalized projection of $J(z_{n+1})$ is not ensured, we use our main result in Theorem 2.1 to choose some point $J(u_{n+1})\in X^*$ too close to $J(z_{n+1})$ so that $\|J(z_{n+1})-J(u_{n+1})\|\le \delta_n$ and $\pi^{f, \lambda}_S(J(u_{n+1}))$ is singleton. So, we take $x_{n+1}: = \pi^{f, \lambda}_S(J(u_{n+1}))$, and then we are done. □

After proving the well definedness of the algorithm without any additional assumptions on $\lambda$, $S$, $f$, and $F$, we add some natural assumptions on the data to prove the convergence of the sequence $\{x_n\}_n$ to a solution of (4.1).

In our analysis, we need the following assumptions on $S$, $f$, and $F$:

Assumptions ${\mathcal A}$:

(1) The set $S$ is compact;

(2) $F$ is bounded on $S$ by some constant $L > 0$;

(3) $F$ has a closed graph on $S$. That is, for any convergent sequence $\{(x_n, y^*_n)\}_n$ in $\mbox{gph }F$ the graph of $F$ with $x_n\in S$, we have the limit $(\bar x, \bar y^*): = \lim_n (x_n, y^*_n)$ stays in $\mbox{gph }F$;

(4) There exists some constant $\mu > 0$ and $\xi > 0$ such that

(5) The constants $\mu$, $\delta_0$, and $\lambda$ satisfy:

Theorem 4.1. Let $\{x_n\}_n$ be a sequence generated by Algorithm 1. Assume that the assumptions ${\mathcal A}$ are satisfied, then there exists a subsequence of $\{x_n\}_n$ converging to a solution of (4.1).

Proof. By compactness of the set $S$ and the construction of the sequence $(x_n)_n$, there exists a subsequence $(x_{n_k})$ converging to some point $\bar x \in S$. We have to prove that $\bar x$ is a solution of (4.1). That is, $-F(\bar x)\cap N^{f, \lambda}(\bar x)\not = \emptyset$, meaning there exists $\bar y^*\in F(\bar x)$ such that $-\bar y^*\in N^{f, \lambda}_S(\bar x)$, i.e., $\bar x \in \pi^{f, \lambda}_S(J(\bar x)-\lambda \bar y^*)$. Set $\bar z: = J^*(J(\bar x)-\lambda \bar y^*)$. By construction, we have $x_{n_k+1} = \pi_S^{f, \lambda}(J(u_{n_k+1}))$. By closedness of the graph of $F$ in (4), we obtain easily that the sequence $\{y^*_{n_k}\}_k$ is convergent to some limit $\bar y^*$ belonging to $F(\bar x)$. Also, the continuity of $J$ and $J^*$ ensure that the subsequence $(z_{n_k})_k$ converges to $\bar z: = J^*(J(\bar x)-\lambda \bar y^*)$. Also, by construction we have

which ensures that the subsequence $(J(u_{n_k}))_k$ converges to $J(\bar z)$. Now, we have to prove that $(J(u_n))_n$ and $\bar x$ belong to $J(S)+\mu \mathbb B$. Using the choice of $\lambda$ and the assumptions on the constants $L, \delta_0$ and $\mu$, we can write

and

which ensure that $J(u_n)$ and $J(\bar z)$ belong to $J(S)+\mu \mathbb B_*$. Using the Lipschitz continuity of the $({f, \lambda})$-generalized projection on $J(S)+\mu \mathbb B_*$ we can write

This ensures that $\bar x = \pi^{f, \lambda}_S(J(\bar z)) = \pi^{f, \lambda}_S(J(\bar x)-\lambda \bar y^*)$, which ensures by definition of the set $N^{f, \lambda}_S(\bar x)$ that

Thus,

that is, $\bar x$ is a solution of (4.1), thus completing the proof. □

Example 4.1. Assume that $X$ is $p$-uniformly smooth and 2-uniformly convex space (for instance, $X = L^p$ with $p\in(1, 2]$), and consider the following nonconvex variational inequality: Find $\bar x\in S$ and $y^*\in F(\bar x)$ such that

where $S$ is given by $S: = A_2\cup (A_2+x_0)$ and $f = \psi_A$ with $A: = A_1\cup (A_1+x_0)$. Assume that $A_1$ and $A_2$ are two closed convex sets with $A_1\cap A_2 = \mathbb B$ (i.e., their intersection is the unit ball), and assume that $x_0$ is very far from both sets $A_1$ and $A_2$ so that $A_2\cap (A_2+x_0) = \emptyset$ and $A_1\cap (A_1+x_0) = \emptyset$. We assume, for instance, that $\|x_0\|\ge 2{\rm diam}(A_1\cup A_2)$. Assume that the set $A_2$ is compact, then by Theorem 4.1, the noncovex variational inequality (4.6) admits a solution as a limit of a subsequence of the sequence generated by Algorithm 1.

Proof. To do that, we have to prove that all the assumptions of Theorem 4.1 are fulfilled. First, we need to check the two following equalities:

The first equality follows directly from our assumption on $x_0$ and some simple computations. To prove the second equality, we take $x^*\in X^*$ and any $\lambda > 0$ and let $\bar x\in \pi_S^{f, \lambda}(x^*)$. Then, $\bar x\in S$ with

This inequality ensures that $\bar x$ has to be in $A$ and, hence, $\bar x\in A\cap S$ with

Since $\lambda > 0$, we deduce that

This ensures by definition that $\bar x\in \pi_{A\cap S}(x^*)$. In a similar way, we prove the converse direction.

Now, we recall two results from ^[7].

(1) Example 4.10 in ^[7]. The set $\mathbb B\cup (\mathbb B+x_0)$ with $\|x_0\| > 3$ is a closed nonempty nonconvex set that is uniformly generalized prox-regular for some positive constant $r > 0$ in the sense of ^[7].

(2) Theorem 4.4 in ^[7]. If $X$ is $q$-uniformly convex and $K$ is a bounded set, which is uniformly generalized prox-regular for some positive constant $r > 0$, then there exists some $r'\in (0, r)$ and $\gamma > 0$ such that

Here, $U^V_K(r'): = \{x^*\in X^*: { \inf\limits_{y\in K} }V(x^*, y) \le r'^2\}$.

Since the space $X^*$ is $2$-uniformly smooth, there exists some $\alpha > 0$ such that $V(x^*, y)\le \alpha \|x^*-J(y)\|^2, \quad \forall y\in S.$ Here, $\alpha$ depends only on $S$ and the space $X^*$. Choose $\mu\in (0, {r'\over \sqrt{\alpha}})$, then for any $x^*\in J(S)+\mu \mathbb B_*$ we have

that is, $x^*\in U^V_S(r')$. Combining now the above results (1) and (2), we have $A\cap S = \mathbb B\cup(\mathbb B+x_0)$ is bounded uniformly generalized prox-regular for some $r > 0$, and so there exists $r'\in (0, r)$ such that $\pi_{A\cap S}$ is Lispchitz continuous on $U^V_{A\cap S}(r')$, and in particular on $J(S)+\mu \mathbb B_*$ for some $\mu > 0$. Therefore, all the assumptions of Theorem 4.1 are satisfied, and so the sequence generated by Algorithm 1 admits a subsequence converging to a solution of the nonconvex variational inequality (4.6). □

Remark 4.2. It is very important to mention that the existence of solution of (4.6) cannot be obtained by any other existence results due to the nonconvexity of all the data of the considered problem, the set $S$ and the function $f$.

Use of AI tools declaration

The author declares they have not used Artificial Intelligence (AI) tools in the creation of this article.

Acknowledgments

The author extends his appreciations to Researchers Supporting Project number (RSPD2023R1001), King Saud University, Riyadh, Saudi Arabia.

Conflict of interest

The author declares that they have no conflict of interests.