Electroencephalogram based face emotion recognition using multimodal fusion and 1-D convolution neural network (ID-CNN) classifier

1.   Introduction

In 2005, Rodríguez ^[1] used the Lyapunov-Schmidt method and Brower fixed-point theorem to discuss the following discrete Sturm-Liouville boundary value problem

where $\lambda$ is the eigenvalue of the corresponding linear problem and the nonlinearity $f$ is bounded.

Furthermore, in 2007, Ma ^[2] studied the following discrete boundary value problem

where $f$ is subject to the sublinear growth condition

for some $0\leq\alpha < 1$ and $A, B\in (0, \infty)$. Additional results to the existence of solutions to the related continuous and discrete problems on the nonresonance and the resonance can be found in ^{[3,4,5,6,7,8,9,10,11,12,13]} and the references therein. For example, Li and Shu ^[14] considered the existence of solutions to the continuous Sturm-Liouville problem with random impulses and boundary value problems using the Dhage's fixed-point theorem and considered the existence of upper and lower solutions to a second-order random impulsive differential equation in ^[15] using the monotonic iterative method.

Inspired by the above literature, we use the solution set connectivity theory of compact vector field ^[16] to consider the existence of solutions to discrete resonance problems

where $p:[0, T]_{\mathbb{Z}}\rightarrow (0, \infty), \ q:[1, T]_{\mathbb{Z}}\rightarrow \mathbb{R}$, $\overline{g}:[1, T]_{\mathbb{Z}}\rightarrow \mathbb{R}$, $r(t) > 0, \ t\in[1, T]_{\mathbb{Z}}$, $(\lambda_{k}, \psi_{k})$ is the eigenpair of the corresponding linear problem

It is worth noting that the difference between the problem (1.1) and the above questions is the eigenvalue that not only appears in the equation but also in the boundary conditions, which causes us considerable difficulties. Furthermore, it should be noted that these problems also apply to a number of physical problems, including those involving heat conduction, vibrating strings, and so on. For instance, Fulton and Pruess ^[17] discussed a kind of heat conduction problem, which has the eigenparameter-dependent boundary conditions. However, to discuss this kind of problem, we should know the spectrum of the problem (1.2). Fortunately, in 2016, Gao and Ma ^[18] obtained the eigenvalue theory of problem (1.2) under the conditions listed as follows:

$({\rm{A}}_{1})$ $\delta_{0}: = a_{0}d_{0}-b_{0}c_{0} < 0, \, c_{0}\neq0$, $d_{1}-b_{1}\neq0,$

$({\rm{A}}_{2})$ $\delta_{1}: = a_{1}d_{1}-b_{1}c_{1} > 0, \, c_{1}\neq0$, $b_{0}+d_{0}\neq0,$

which laid a theoretical foundation for this paper.

Under the conditions $({\rm{A}}_{1})$ and $(A_{2})$, we assume the following conditions hold:

$({\rm{H}}_{1})$ (Sublinear growth condition) $f:[1, T]_{\mathbb{Z}}\times\mathbb{R}\rightarrow \mathbb{R}$ is continuous and there exist $\alpha\in[0, 1)$ and $A, B\in(0, \infty)$, such that

$({\rm{H}}_{2})$ (Symbol condition) There exists $\omega > 0$, such that

or

$({\rm{H}}_{3})$ $\overline{g}:[1, T]_{\mathbb{Z}}\rightarrow \mathbb{R}$ satisfies

$({\rm{H}}_{4})$ $f:[1, T]_{\mathbb{Z}}\times\mathbb{R}\rightarrow \mathbb{R}$ is continuous and

uniformly for $t\in[1, T]_{\mathbb{Z}}$.

The organization of this paper is as follows. In the second section, we construct a completely new inner product space. In the new inner product space, we discuss the basic self-adjointness of the corresponding linear operator and the properties of the eigenpair of (1.2). Finally, under the above properties, the Lyapunov-Schmidit method is used to decompose the inner product space and transform our problem to an equivalent system, that is to say, finding the solutions of (1.1) is equivalent to finding the solutions of this system. Under the sublinear condition and sign conditions on nonlinear terms, an existence result of solutions to the problem (1.1) is obtained using Schauder's fixed-point theorem and the connectivity theories of the solution set of compact vector fields. Based on the first result, the existence of two solutions to the problem (1.1) is also obtained in this section.

2.   Preliminaries

Definition 2.1. (^[19]) A linear operator $P$ from the linear space $X$ to itself is called the projection operator, if $P^{2} = P.$

Lemma 2.2. (^[16]) Let $\mathcal{C}$ be a bounded closed convex set in Banach space $E$, $T:[\alpha, \beta]\times\mathcal{C}\rightarrow \mathcal{C}(\alpha < \beta)$ be a continuous compact mapping, then the set

contains a connected branch connecting $\{\alpha\}\times\mathcal{C}$ and $\{\beta\}\times\mathcal{C}$.

Lemma 2.3. (^[20])(Schauder) Let $D$ be a bounded convex closed set in $E$, $A:D\rightarrow D$ is completely continuous, then $A$ has a fixed point in $D$.

First, we construct the inner product space needed in this paper.

Let

then $Y$ is a Hilbert space under the following inner product

and its norm is $\|y\|_{Y}: = \sqrt{\langle y, y\rangle_{Y}}$.

Furthermore, consider the space $H: = Y\oplus\mathbb{R}^{2}$. Define the inner product as follows:

which norm is defined as

where $\top$ is transposition to a matrix.

Let

and

For $y^{*} = [y, \alpha, \beta]^{\top}$, define an operator $L:D\rightarrow H$ as follows:

where $D = \left\{[y, \alpha, \beta]^{\top}:y\in Y, \ y_{0, 1} = \alpha, \ y_{T+1, 1} = \beta\right\}.$ Define $S:D\rightarrow H$ as follows:

Then, the problem (1.2) is equivalent to the eigenvalue problem as follows:

that is, if $(\lambda_{k}, y)$ is the eigenpair of the problem (1.2), then $(\lambda_{k}, y^{*})$ is the eigenpair of the opertor $L$. Conversely, if $(\lambda_{k}, y^{*})$ is the eigenpair of the operator $L$, then $(\lambda_{k}, y)$ is the eigenpair of the problem (1.2).

Eventually, we define $\mathcal{A}:D\rightarrow H$ as follows:

where $\mathcal{F}(t, y^{*}) = \mathcal{F}(t, [y, \alpha, \beta]^{\top}) = [f(t, y), 0, 0]^{\top}$. Obviously, the solution of the problem (1.1) is equivalent to the fixed point of the following operator

It can be seen that there is a homomorphism mapping $(\lambda_{k}, y)\leftrightarrow(\lambda_{k}, y^{*})$ between the problem (1.1) and the operator Eq (2.2).

Next, we are committed to obtaining the orthogonality of the eigenfunction.

Lemma 2.4. Assume that $(\lambda, y^{*})$ and $(\mu, z^{*})$ are eigenpairs of $L$, then

Proof Let $y^{*} = [y, \alpha, \beta]^{\top}\in D, \ z^{*} = [z, \zeta, \rho]^{\top}\in D$, then

Similarly, we have

It can be seen from (2.3) and (2.4)

□

Lemma 2.5. The operator $L$ is the self-adjoint operator in $H$.

Proof For $y^{*} = [y, \alpha, \beta]^{\top}\in D, \, z^{*} = [z, \zeta, \rho]^{\top}\in D$, we just need to prove that $\langle y^{*}, Lz^{*}\rangle = \langle Ly^{*}, z^{*}\rangle$. By the definition of inner product in $H$. we obtain

and

Therefore,

where

and

Moreover, from

and

we have

□

In order to obtain the orthogonality of the eigenfunction, we define a weighted inner product related to the weighted function $r(t)$ in $H$. First, we define the inner product in $Y$ as $\langle y, z\rangle_{r} = \sum\limits_{t = 1}^{T}r(t)y(t)z(t)$.

Similarly, the inner product associated with the weight function $r(t)$ in the space $H$ is defined as follows:

Lemma 2.6. (Orthogonality theorem) Assume that $({\rm{A}}_{1})$ and $(A_{2})$ hold. If $(\lambda, y^{*})$ and $(\mu, z^{*})$ are two different eigenpairs corresponding to $L$, then $y^{*}$ and $z^{*}$ are orthogonal under the weight inner product related to the weight function $r(t)$.

Proof Assume that $(\lambda, y^{*})$ and $(\mu, z^{*})$ is the eigenpair of $L$, then it can be obtained from Lemmas 2.4 and 2.5

Therefore, if $\lambda\neq\mu$, then $\langle y^{*}, z^{*}\rangle_{r} = 0$, which implies that $y^{*}$ and $z^{*}$ are orthogonal to the inner product defined by the weighted function $r(t)$. □

Lemma 2.7. (^[18]) Suppose that $({\rm{A}}_{1})$ and $(A_{2})$ hold. Then (1.2) has at least $T$ or at most $T+2$ simple eigenvalues.

In this paper, we consider that $\lambda_{k}$ is a simple eigenvalue, that is, the eigenspace corresponding to each eigenvalue is one-dimensional. Let $\psi_{k}^{*} = [\psi_{k}, \alpha, \beta]^{\top}\in D$ be the eigenfunction corresponding to $\lambda_{k}$, and assume that it satisfies

Denote by $\mathcal{L}: = L-\lambda_{k}S$, then the operator (2.2) is transformed into

Define $\mathcal{P}: D\rightarrow D$ by

Lemma 2.8. $\mathcal{P}$ is a projection operator and ${\rm Im}(\mathcal{P}) = \rm Ker(\mathcal{L}).$

Proof Obviously, $\mathcal{P}$ is a linear operator, next, we need to prove $\mathcal{P}^{2} = \mathcal{P}$.

It can be obtained from the Definition 2.1, $\mathcal{P}$ is a projection operator. In addition, ${\rm Im}(\mathcal{P}) = {\rm span}\{\psi_{k}^{*}\} = {\rm Ker}(\mathcal{L})$. □

Define $\mathcal{H}: H\rightarrow H$ by

Lemma 2.9. $\mathcal{H}$ is a projection operator and $\rm Im(\mathcal{H}) = Im(\mathcal{L}).$

Proof Obviously, $\mathcal{H}$ is a linear operator, next, we need to prove that $\mathcal{H}^{2} = \mathcal{H}$.

It can be obtained from Definition 2.1 that $\mathcal{H}$ is a projection operator. On the one hand, for any $[y, \alpha, \beta]^{\top}\in H$, we have

thus, ${\rm{Im}}(\mathcal{H})\subset {\rm{Im}}(\mathcal{L})$. On the other hand, for any $y^{*}\in {\rm{Im}}(\mathcal{L})$, we have

In summary, ${\rm{Im}}(\mathcal{H}) = {\rm{Im}}(\mathcal{L}).$ □

Denote that $I$ is a identical operator, then

The restriction of the operator $\mathcal{L}$ on $\mathcal{L}|_{{\rm Im}(I-\mathcal{P})}$ is a bijection from ${\rm Im}(I-\mathcal{P})$ to ${\rm Im}(\mathcal{H})$. Define $\mathcal{M}: {\rm Im} (\mathcal{H})\rightarrow {\rm Im}(I-\mathcal{P})$ by

It can be seen from ${\rm Ker}\mathcal{L} = {\rm span}\{\psi_{k}^{*}\}$ that there is a unique decomposition for any $y^{*} = [y, \alpha, \beta]^{\top}\in D$

where $\rho\in \mathbb{R}, \, x^{*} = [x, \alpha, \beta]^{\top}\in{\rm Im}(I-\mathcal{P}).$

Lemma 2.10. The operator Eq (2.6) is equivalent to the following system

where $\alpha = a_{0}\psi_{k}(0)-c_{0}\Delta\psi_{k}(0), \, \beta = a_{1}\psi_{k}(T+1)-c_{1}\nabla\psi_{k}(T+1).$

Proof (ⅰ) For any $y^{*} = \rho\psi^{*}_{k}+ x^{*}$, we have

(ⅱ) Since $\langle \mathcal{L}y^{*}, \psi_{k}^{*} \rangle = 0$, we have $\langle\mathcal{A}y^{*}, \psi_{k}^{*}\rangle = 0$. Therefore,

Combining $({\rm{H}}_{3})$ with (2.5), we have

where $\alpha = a_{0}\psi_{k}(0)-c_{0}\Delta\psi_{k}(0), \beta = a_{1}\psi_{k}(T+1)-c_{1}\nabla\psi_{k}(T+1).$ □

3.   Main results

Let

Obviously,

Lemma 3.1. Supposed that $({\rm{H}}_{1})$ holds, then there exist constants $M_{0}$ and $M_{1}$, such that

where $(\rho, x^{*})$ is the solution of (2.7) and satisfies $|\rho|\geq M_{0}$.

Proof Since

we have

Denote that

Furthermore, we have

So, if we let

we have

Thus,

This implies that

□

Lemma 3.2. Suppose that $({\rm{H}}_{1})$ holds, then there exist constants $M_{0}$ and $\Gamma$, such that

where $(\rho, x^{*})$ is the solution of (2.7) and satisfies $|\rho|\geq M_{0}$.

According to Lemma 3.2, choose constant $\rho_{0}$, such that

Let

Then, for sufficiently large $\rho\geq\rho_{0}$, there is

and for sufficiently small $\rho\leq-\rho_{0}$, there is

Theorem 3.3. Suppose that $({\rm{A}}_{1})$, $({\rm{A}}_{2})$ and $({\rm{H}}_{1})$–$({\rm{H}}_{3})$ hold, then there exists a non-empty bounded set $\Omega_{\overline{g}}\subset\mathbb{R}$, such that the problem (1.1) has a solution if and only if $\theta\in\Omega_{\overline{g}}$. Furthermore, $\Omega_{\overline{g}}$ contains $\theta = 0$ and has a non-empty interior.

Proof We prove only the case of (1.3) in $({\rm{H}}_{2})$, and the case of (1.4) can be similarly proved.

From (1.3) and (3.2)–(3.5), it is not difficult to see that

for sufficiently large $\rho\geq\rho_{0}$ and for sufficiently small $\rho\leq-\rho_{0}$,

Therefore, if $\rho\geq\rho_{0}$ is sufficiently large,

if $\rho\leq-\rho_{0}$ is sufficiently small,

Let

Define $\mathcal{T}_{\rho}: {\rm Im}(I-\mathcal{P})\rightarrow {\rm Im}(I-\mathcal{P})$ by

Obviously, $\mathcal{T}_{\rho}$ is completely continuous. By (3.1), for $x^{*}\in\mathcal{C}$ and $\rho\in[-\rho_{0}, \rho_{0}]$,

i.e.,

According to Schauder's fixed point theorem, $\mathcal{T}_{\rho}$ has a fixed point on $\mathcal{C}$, such that $\mathcal{T}_{\rho}x^{*} = x^{*}$. It can be seen from Lemma 2.10 that the problem (1.1) is equivalent to the following system

where

At this time, the $\Omega_{\overline{g}}$ in Theorem 3.3 can be given by $\Omega_{\overline{g}} = \Psi(S_{\overline{g}})$. There exists a solution to the problem (1.1) for $\theta\in\Omega_{\overline{g}}$.

From (3.6), (3.7) and $A^{+}\cup A^{-}\neq\emptyset$, we can deduce that for any $x^{*}\in K$

Thus,

According to Lemma 2.2, $S_{\overline{g}}\subset\mathbb{R}\times \overline{B}_{\rho_{0}}$ contains a connected branch $\xi_{-\rho_{0}, \rho_{0}}$ connecting $\{-\rho_{0}\}\times\mathcal{C}$ and $\{\rho_{0}\}\times\mathcal{C}$. Combined with (3.8), $\Omega_{\overline{g}}$ contains $\theta = 0$ and has a non-empty interior. □

Theorem 3.4. Suppose that $({\rm{A}}_{1})$, $({\rm{A}}_{2})$, $({\rm{H}}_{2})$–$({\rm{H}}_{4})$ hold. $\Omega_{\overline{g}}$ as shown in Theorem 3.3, then there exists a nonempty set $\Omega_{\overline{g}}^{*}\subset\Omega_{\overline{g}}\setminus\{0\}$, such that problem (1.1) has at least two solutions for $\theta\in\Omega_{\overline{g}}^{*}$.

Proof We prove only the case of (1.3), and the case of (1.4) can be similarly proved. Since the condition $({\rm{H}}_{4})$ implies that $({\rm{H}}_{1})$, using Theorem 3.3, we know that there exists $\rho_{0} > 0$, such that

Let

then $\delta > 0$.

Next, we prove that problem (1.1) has at least two solutions for any $\theta\in(0, \delta)$.

Let

By $({\rm{H}}_{4})$, there exists a constant $A_{0}$ such that

Similar to the derivation of Theorem 3.3, there exists $\rho^{*} > \rho_{0}$ such that the following results hold:

(ⅰ) For $\rho\geq\rho^{*}$, there is

(ⅱ) For $\rho\leq-\rho^{*}$, there is

Let

According to $({\rm{H}}_{4})$, (3.9) and (3.10), we have

uniformly for $x^{*}\in \overline{K}$, i.e.

Therefore, there exists a constant $l:l > \rho^{*} > \rho_{0} > 0$ such that $S_{\overline{g}}$ contains a connected branch between $\{-l\}\times \mathcal{C}^{*}$ and $\{l\}\times \mathcal{C}^{*}$, and

It can be seen from the connectivity of $\xi_{-l, l}$ that there exist $(\rho_{1}, x_{1}^{*})$ and $(\rho_{2}, x_{2}^{*})$ in $\xi_{-l, l}(\subset S_{\overline{g}})$, such that

where $\rho_{1}\in(-l, \rho_{0}), \rho_{2}\in(\rho_{0}, l)$. It can be proved that $\rho_{1}\psi_{k}^{*}+x_{1}^{*}$ and $\rho_{2}\psi_{k}^{*}+x_{2}^{*}$ are two different solutions of problem (1.1). □

4.   Example

In this section, we give a concrete example of the application of our major results of Theorems 3.3 and 3.4. We choose $T = 3, a_0, d_0, b_1, c_1 = 0$ and $a_1, d_1, b_0, c_0 = 1$, which implies that the interval becomes $[1, 3]_{\mathbb{Z}}$ and the conditions $(A_1), (A_2)$ hold.

First, we consider the eigenpairs of the corresponding linear problem

Define the equivalent matrix of (4.1) as follows,

Consequently, $A_{\lambda}y = 0$ is equivalent to (4.1). Let $|A_{\lambda}| = 0$, we have

which are the eigenvalues of $(4.1).$ Next, we choose $\lambda = \lambda_{1} = -1.4657$, then we obtain the corresponding eigenfunction

Example 4.1. Consider the following problem

where

and

Then, for $f(t, y(t))$, we have $|f(t, y(t))|\leq 3|y(t)|^{\frac{1}{3}}$. If we choose $\omega = 1$, $yf(t, y) > 0$ for $|y(t)| > 1$. For $\overline{g}(t)$, we have $\sum\limits_{s = 1}^{3}\overline{g}(s)\psi_{1}(s) = 0$.

Therefore, the problem (4.2) satisfies the conditions $({\rm{A}}_{1}), \; ({\rm{A}}_{2})$, $({\rm{H}}_{1})$–$({\rm{H}}_{3})$, which implies that the problem (4.2) has at least one solution by Theorem 3.3.

Example 4.2. Consider the following problem

where

and

Then, for $f(t, y(t))$, we always have $yf(t, y) > 0$ for all $y(t) > 0$ or $y(t) < 0$, $f$ is continuous and satisfies

For $\overline{g}(t)$, we have $\sum\limits_{s = 1}^{3}\overline{g}(s)\psi_{1}(s) = 0$.

Therefore, the problem (4.3) satisfies the conditions $({\rm{A}}_{1}), \; ({\rm{A}}_{2})$, $({\rm{H}}_{2})$–$({\rm{H}}_{4})$, which implies that the problem (4.3) has at least two solutions by Theorem 3.4.

Use of AI tools declaration

The authors declare that they have not used Artificial Intelligence (AI) tools in the creation of this article.

Acknowledgments

Supported by National Natural Science Foundation of China [Grant No. 11961060] and Natural Science Foundation of Qinghai Province(No.2024-ZJ-931).

Conflict of interest

The authors declare that there are no conflicts of interest.