Higher order Weighted Random k Satisfiability ($k = 1, 3$) in Discrete Hopfield Neural Network

1.   Introduction

Artificial intelligence (AI) refers to the simulation or approximation of human intelligence by software-coded heuristics. Artificial neural networks (ANNs), inspired by the intricate workings of the human brain, have sparked a worldwide revolution in artificial intelligence ^[1]. ANNs has become a popular standard approach to solve the optimization problems about several ranges of real-life problems such as in tendency forecasting of economic growth ^[2], the definition of "smart city" based on transportation ^[3], the distance education of university students ^[4], and in the business applications ^[5]. ANNs have been desired to have the ability to simulate the work that human brains do. The solid structure of the ANN made it limited to showing the real understanding of human brain activities. Although artificial intelligence excels in complex prediction tasks and most AI researchers rely on the ability of the ANN, it is limited in many areas, often being referred to as black-box models ^[6]. As computers can do complex work faster than humans and can store information that can help to make final decisions, Hopfield and Tank first added a simple variant of ANN into the Hopfield neural network (HNN) in 1985 ^[7]. The HNN consists of interconnected neurons without hidden neurons. Each neuron is connected through synaptic weight. HNN has the capability to access memory address which is called Content addressable memory (CAM). CAM is a special type of memory that can retrieve specific data through input content, helping network devices quickly retrieve specific data and improving network performance ^[8,9]. The HNN has two structures which are namely discrete and continuous ^[10]. Furthermore, as highlighted in other research, the Discrete Hopfield Neural Network (DHNN) has been a central focus due to its rapid development and wide range of applications. Abdullah (1992) added the logic structure named Satisfiability (SAT) into the ANN that leads DHNN to attain better performances obtaining the synaptic weight during the training phase ^[11]. SAT plays an important role in HNN with the storage capacity of the meaningful inspects of information. This unique structure of logic has the capability of transforming various problems into mathematical formulations. By utilizing special neurons that process a large amount of input information, the results from different aspects of the logic can help explain the predicted outputs. SAT is a powerful tool in automatic reasoning, enabling complex problem-solving across various domains in computer science and artificial intelligence.

SAT refers to determining whether there exists an assignment of variables to a Boolean formula that makes the formula true. SAT is of significant importance in computer science, both as a fundamental topic in theoretical computational complexity research and for its wide-ranging practical applications in fields, such as circuit design ^[12], software verification ^[13], artificial intelligence ^[14,15], and compiler optimization ^[16]. One of the earliest works by Abdullah proposed Horn satisfiability (Horn SAT) as a new concept of SAT ^[11]. Horn SAT has been considered a good representation in the field of satisfiability ^[17]. The behavior of Horn SAT allows the cost function to be zero during the learning phase and decides the dynamic of the neuron state of HNN. Despite the results being able to locate more than 90% of global minimum solutions, the properties of Horn SAT seem fixed. The results demonstrate that DHNN can be implemented by this kind of logic rule. Since then, several researchers have conducted a significant amount of outstanding research work on the improvement of the structure of logic. Kasihmuddin et al. ^[18] introduced a systematic logic rule named 2-Satisfiability (2SAT) which consists of strictly two literals per clause. Furthermore, a training algorithm was implemented in the training phase of DHNN which can help to minimize the cost function and can also obtain the optimal synaptic weight. Thereafter, the search for this kind of optimal SAT has come a long way. Zhu et al. ^[19] extended the order of the logic into 3-Satisfiability (3SAT) which proposed a higher-ordered SAT. All the clauses are expressed in Disjunctive Normal Form (DNF), with each literal connected through Conjunctive Normal Form (CNF) within every clause. This study observes that, despite the use of optimized algorithms, the solution space grows exponentially as the size of 3SAT instances increases. Therefore, solving large 3SAT instances demands significant energy consumption on traditional serial electronic computers. Mansor et al. ^[20] discussed various metrics to justify the capabilities of 3SAT. Moreover, they found that when the order of a clause exceeds 3, it can be reduced to a combination of clauses with order 3 or lower, according to the reduction theorem. The logical rule was embedded into DHNN by comparing the cost function with the Lyapunov energy function. The synaptic weights are derived from the comparison of these two functions. When the number of neurons increased, the synaptic weight values seemed suboptimal and the number of global minimal solutions also decreased. Due to the inflexibility of the logical rules, DHNN always lacks universality.

Addressing the need for varying logic clause structures during the retrieval process, Mouhoub et al. ^[21] proposed a non-systematic satisfiability approach. This method checks whether a solution to an SAT problem remains valid when new clauses are added and can efficiently adjust the solution to satisfy both the original formula and the new clauses if necessary. Sathasivam et al. ^[22] later established a new logic rule named Random $k$ satisfiability (RAN$k$SAT) where $k = 2$. This work first mentioned the formula combination consists of first-order logic and second-order logic. The clauses are combined using disjunction, while the literals within each clause are connected through conjunction. The main idea of the implementation of RANkSAT is the flexibility to represent the number of literals that are not limited to only one variable per clause ^[22]. It was proposed by introducing the random structure that involves first and second-order SAT logical rules. This work was later extended using higher orders for non-systematic logical rules. The study of Karim et al. ^[23], shows different variants of Random 3 Satisfiability (RAN3SAT). The combination structures of all possible clauses were all implemented in DHNN. RAN3SAT has proposed a method with a high variation value, high global minima ratio, and low learning error. It also concluded that the logic structure with $k = 2, 3$, which means the combination contains only second-order and third-order logic clauses would reach a higher probability of minimizing the cost function and better management of synaptic weight management, which leads to lower energy. Alway et al. ^[24] developed another variation of logic structure that used second-order clauses as the domain clause in the situation of non-systematic SAT. This special logic was named Major 2-satisfiability (MAJ2SAT) where the ratio of the second-order clause is larger than the ratio of the third-order clause. Notably, the overfitting occurs when the ratio of second-order clauses increases in MAJ2SAT. This is because when the number of third-order clauses decreases the probability of satisfied interpretation will also decrease. The work also showed that it could have a lower similarity value than the existing systematic logic rules.

In a subsequent development, Guo et al. ^[25] introduced a novel logic rule named Y-Random 2 satisfiability (YRAN2SAT), which explored stimulation data within non-systematic logic rule structures. This approach merged both systematic and non-systematic characteristics, combining logical rules with symbolic instructions, thereby significantly enhancing the effectiveness of DHNN. Furthermore, Gao et al. ^[26] introduced a new variation of RAN3SAT named G-Type Random 3-Satisfiability (GRAN3SAT) that capitalized both higher-order systematic and non-systematic logical rules in DHNN. This was the first attempt to add both logical rules into DHNN. In this work, they present all the sets of logical rule structures that have been proposed. The higher-order systematic logical rule provides storage capacity to GRAN3SAT, whereas the higher-order nonsystematic logical rule provides a more diversified third-order logical connection. Despite its wide range of conclusions of logic rules and various metrics discussed in the following work, this work lacks control of the clear distribution of the number of negative literals. Although these fruitful SAT models have yielded effective results in DHNN, there have been fewer outcomes regarding the discussion of the ratio of negative literals in the logic rules. Research in non-systematic logic has advanced the storage capacity of neural networks. However, there has been limited exploration into leveraging higher-order clauses with varying ratios of negative literals. Incorporating these higher-order clauses, which introduce diverse proportions of negative literals, has the potential to significantly broaden the search space and enhance the storage capabilities of neural networks. This represents a promising area for future research that could lead to more efficient and scalable neural network architectures.

To address these challenges mentioned above, our proposed approach will provide a more flexible structure that incorporates first-order and third-order logic which was randomly selected with a given ratio of negative literals in each clause. In the rest of this paper, we propose a newly non-systematic satisfiability logic named Weighted Random $k$ Satisfiability ($k = 1, 3$) (${W_{r3SAT}}$). The proposed work demonstrates the effectiveness of integrating the unique logical structure. The key contributions of the details are outlined as follows:

(1) To formulate a logic rule named Weighted Random $k$ Satisfiability that proposes a structural feature of non-systematic satisfiability. The establishment of kth-order clauses is set randomly with this logic added the ratio of negative literals to the first-order logic and third-order logic clauses. Exhaustive Search (ES) is used to find the correct logic structure for the right number of negative literals.

(2) Implementing the proposed Weighted Random $k$ Satisfiability in DHNN enables effective network governance. This proposed logical rule stores information externally and is considered satisfied only if all clauses are satisfiable. In the present framework, each variable in the logical rule is represented as a neuron.

(3) To propose the function in the learning phase to find the minimum cost function for each clause that is satisfied. The combination structure of this logic has more variables and can provide more literals in representing neuron states in DHNN. The optimal synaptic weights are determined by comparing the cost function with the Lyapunov energy function.

(4) To evaluate the capacity of the proposed network by simulated metrics. We utilized different metrics to evaluate the logical structure, and mentioned that the proposed logical rule has the capacity to perform well with lower error in the learning phase and retrieval phase.

(5) To compare the impact of the proposed non-systematic logical rule with existing logics in DHNN by employing various performance metrics. Several factors are considered such as the management of synaptic weight, learning errors, testing errors, solution variations, and similarity.

The organization of the paper is outlined as follows: The motivation of this work is elaborated in Section 2. The proposed formula of Weighted Random $k$ Satisfiability is introduced in Section 3. In Section 4, we integrate the right logic structure according to the predetermined ratio of negative literals. In Section 5, we add Weighted Random $k$ Satisfiability into DHNN. Various metrics are used to analyze the effectiveness between the proposed logic and other predetermined logic rules in Section 6. In Section 7, we describe the results and discuss the performance of the proposed logic by different metrics. The conclusion and further work are discussed in Section 8 at the end of this paper.

2.   Weighted Random $k$ Satisfiability

SAT is a Boolean algebra logical rule employed to determine if there exists an assignment of variables that makes a given logical formula true. This rule is fundamental in solving problems where the goal is to find an optimal solution or verify the feasibility of a solution within a specific set of constraints. By evaluating various combinations of variable assignments, the SAT can effectively address complex decision-making and optimization challenges. Each clause in satisfiability contains literals with arbitrary numbers. There are two mainstream divisions of SAT logical rules, namely systematic and non-systematic logic rules in DHNN. In terms of the systematic satisfiability logic, there are strictly 2 literals per clause in 2SAT and 3 literals per clause in 3SAT. On the other hand, in terms of non-systematic satisfiability, there are $k$ literals in each clause with $k = 1, 2, 3$. The current study provides a non-systematic logic named Weighted Random $k$ Satisfiability is determined. In order to introduce the novel Weighted Random $k$ Satisfiability in DHNN. The standard formula of Weighted Random $k$ Satisfiability is set as follows:

a) N variables: ${x_{\, 1}}, {x_{\, 2}}, \cdots \cdots, {x_N}$, where ${x_i} \in \left\{ { - 1, 1} \right\}$ has the bipolar values, $i = 1, 2, \cdots, N$. Each literal has the same opportunity to be positive or negative.

b) Structure combinations of clauses where:

Here, $u$ is the number of first-order clauses, $v$ is the number of third-order clauses, $i = 1, 2, \cdots, p$.

c) The general formulation of the proposed logical structure of Weighted Random $k$ Satisfiability (r3SAT) is formulated as follows:

As shown in Eq (4), $Q_i^{(1)}$ and $Q_i^{(3)}$ are refer to the first-order and third-order clauses, respectively. Each clause is combined with CNF.

d) In each clause, literals are combined with the DNF, such structures are stated as follows:

Literals represent bipolar values $\left\{ {{A_i}, \neg {A_i}} \right\}$, $\left\{ {{B_i}, \neg {B_i}} \right\}$, $\left\{ {{C_i}, \neg {C_i}} \right\}$, $\left\{ {{D_i}, \neg {D_i}} \right\}$, and so on. Where ${A_{\, i}}$ represents positive and $\neg {A_{\, i}}$ represents negative. In r3SAT, the ratio of the total number of negative literals is defined as $r$. The approach outlined the range of ratio is between 0.1 and 0.9. The step size is separated randomly. The number of negative literals $n$ in weighted r3SAT can be calculated as follows in Eq (6):

When $r = 0$, it seems that there are no negative literals in every clause. This can be called monotone clauses ^[27]. Additionally, $r = 1$ seems that the whole weighted r3SAT is comprised of literals which are all negative without positive ones. The number of $rN$ sometimes shows the particularity that it is not in nature number. For example, when $r = 0.4$ and $N = 9$ shows that $rN = 3.6$ is not a natural number. The rounding function is used to deal with this kind of conflict. The rounding function means to get the maximum natural number which is not bigger than the real number it has been. Since $rN$ must be a natural number, the number of negative literals is 3 under the condition below:

It can be introduced as $\left[{3.6} \right] = 3$ in short. Equations (8)–(11) show examples under this condition when the number of negative literals is equal to 3.

As illustrated in the examples, the number of negative literals is consistently three. As outlined in Eq (8), the logical structure is composed entirely of first-order clauses, with negative literals randomly distributed across them. Equation (11) presents a structure consisting entirely of third-order clauses, where negative literals are assigned to each clause but distributed randomly. Additionally, Eqs (9) and (10) involve both first-order and third-order clauses, with negative literals randomly assigned to each type.

The logic structure of weighted r3SAT is dependent on the ratio of $r$ and the nature number $N$. The weight for each formula generation of any possible $Q_{\, i}^{(k)}$ for $k = 1, 3$ is defined by Eqs (12) and (13) as below:

where $\alpha _{\, i}^{(k)} \in \left\{ {0, \, 1, \, 2, \, 3} \right\}$ denotes the number of negative literals in the $k$th-order clauses for $k = 1, 3$. When the number of negative literals is equal to 0 per clause, the result of ${\alpha _i}^{(k)}$ is set to be 0. While there exists only one negative literal per clause, the value of ${\alpha _i}^{(k)}$ is equal to 1. It sounds self-evident that when the number of negative literals is equal to 1, 2, or 3. ${\alpha _i}^{(k)}$ is equal to the same value of the number. Equation (12) mentions that in the first-order clause, when the number of negative literals is 1, the value of ${\alpha _i}^{(1)}$ is equal to one. Otherwise, the value is 0. In Eq (13) which shows the clause of third-order, the value of ${\alpha _i}^{(3)}$ is equal to the same value of the number of negative literals. Subsequently, the total number of negative literals can be calculated by the summaries in Eq (14) as follows:

Note that the total weight of negative literals $\eta$ is equivalent to the total number of negative literals present in the weighted r3SAT formulation. One of the most important things during this step is to make sure that Eq (15) can be satisfied.

Building on the established framework of r2SAT ^[28], the Genetic Algorithm (GA) was utilized to identify the optimal logical structure for each possible combination, with the goal of minimizing the fitness function as defined in Eq (15) during the logic phase. In the context of this study, the fitness function is further introduced as a key method for effectively minimizing Eq (15) within the logic phase, offering a systematic approach to refining the logical structure through iterative optimization. The structure of ${W_{r3SAT}}$ represents first and third-order clauses combined with non-redundant literals. As mentioned before, the step size for the ratio is limited to 0.1 while the range of the ratio is between 0.1 to 0.9. The generation of r3SAT is shown in Figure 1. It is worth mentioning that although the correct ${W_{r3SAT}}$ can be generated, the condition determined by Eq (15) often cannot be satisfied. Additional conditions must be imposed during the logic phase to ensure that a valid solution for this distribution exists. The random literals must be generated based on the value of the input $N$ while the weight of the logic is evaluated according to the value of weight ${\alpha _i}$ in each cause. ES is used to generate the correct number of literals in each clause with a linear function ^[29]. The ratio of negative literals is determined by a specified value. After the generation of these literals, they are randomly distributed across the first-order and third-order logic clauses. The detailed process of this generation will be discussed in the following section. After that, the structure of ${W_{r3SAT}}$ seems established.

3.   Logic generation of Weighted Random 3 Satisfiability

In the following work, ES is employed to address the problem of determining the distribution of negative literals in both first-order and third-order logic. In contrast, the studies in ^[28,30] utilized Genetic Algorithms (GA) because GA is well-suited for solving continuous optimization problems due to its balanced operators for exploration and exploitation. If GA successfully optimizes the resulting fitness function, the outcomes will generally be at least suboptimal in most cases ^[30]. GA was employed to propose a structured logic phase, serving as an organized framework for assigning the distribution of negative literals in an unbiased manner ^[28]. Although GA can handle continuous problems, this usually involves discretizing real variables or using special encoding schemes, which may increase complexity and reduce efficiency ^[28]. Hassanat A, et al. ^[31] mentioned that GA usually uses fixed mutation rates and crossover operations, which may show poor adaptability to some problems. ES is an algorithmic strategy that tries all possible solutions in the solution space of a problem one by one to find the optimal solution or a solution that satisfies certain conditions ^[32]. This approach usually ensures that the global optimal solution is found. The ES method has been used in beam angle optimization ^[33]. The proposed approach utilizes ES to solve the problem, owing to its reliability in converging toward optimal solutions. To assess the quality of the solutions, a fitness function is employed, which quantifies the number of negative literals. This fitness function is mathematically formulated in Eq (16), providing a clear metric for solution evaluation and guiding the optimization process:

when ${f_{{W_{r3SAT}}}} = 0$, the fitness function reached the optimal value of ${f_{{W_{r3SAT}}}}$. It also illustrates that Eq (15) is satisfied and the correct structure of ${W_{r3SAT}}$ should be generated. On the other hand, ${f_{{W_{r3SAT}}}} \ne 0$ illustrates that the solution of the function is not satisfied, ES failed to find the correct structure of the logic mentioned and it will also fail to obtain the correct number of negative literals.

While GA will repeat from the initialization process when ${f_{{W_{r3SAT}}}} \ne 0$ until ${f_{{W_{r3SAT}}}} = 0$ is achieved or until a specified number of iterations which has been restricted in advance. The algorithm ES is utilized to get the correct number without duplication testing each possible solution in the solution space of a problem sequentially to identify the optimal solution or a solution that meets specific criteria. For example, in the case where $r = 0.5$ and $N = 9$, while $\left[{rN} \right] = \left[{4.5} \right] = 4$ is a nature number, ES would find a solution satisfies $\eta = 4$ to achieve ${f_{{W_{r3SAT}}}} = 0$ in the case of Eq (16). The separation of the value $\alpha _{\, i}^{(k)}$ for $k = 1, 3$ in each kind of clause should be shown under the condition of absolutely two negative literals in the first-order clause and two negative literals in the third-order clause. Finally, it can be concluded that the possible structure ${W_{r3SAT}}$ will be generated as follows:

Equation (17) shows the example of $r = 0.5 \in \left[{0.1, \, 0.9} \right]$. While excessive cases where $r = 0$ or $r = 1$ exist, it indicates that there are no negative literals or that the whole logic is filled with negative literals monotonously. Monotone clauses can result in an inefficient solution to properly managing the synaptic weights. A clause is considered monotone if it contains only unnegated variables or only negated variables ^[34]. Since a first-order clause has only one satisfied interpretation, its presence has minimal impact on the overall structure of the logic when dealing with negative form literals. The logic phase in Algorithm 1 presents the pseudocode for generating the randomly weighted 3-SAT logic phase.

4.   Weighted r3SAT in the Discrete Hopfield Neural Network

The effectiveness of DHNN primarily relies on its ability to store information, which is crucial for solving optimization problems. Researchers have demonstrated that this capability can be applied to a wide range of problems due to its versatility in addressing complex issues ^[24,25,26]. The fundamental rule for the update of neurons is shown as follows:

where ${W_{ijk}}$ denotes the synaptic weight between neuron $i$, neuron $j$ and neuron $k$. The weight presents the strength of the connections between neurons ^[26]. ${S_i}$ denotes the state of neuron $i$ and ${\theta _{\, i}}$ is a predetermined threshold value. The property of the synaptic weights for two neurons in the state is always symmetric which has the feature that $W_{\, ij}^{\, (2)} = W_{\, ji}^{\, (2)}$, ${W_{ijk}}^{(3)}$ have the same value for all permutation combinations with $i \ne j \ne k$, and has no self-connection $W_{\, ii}^{\, (2)} = W_{\, jj}^{\, (2)} = W_{\, kk}^{\, (2)} = 0$, $W_{\, iii}^{\, (3)} = W_{\, jjj}^{\, (3)} = W_{\, kkk}^{\, (3)} = 0$. The predetermined threshold ${\theta _i}$ is always determined as ${\theta _{\, i}} = 0$ which means that the energy in DHNN decreases uniformly ^[28]. The primary purpose of implementing ${W_{r3SAT}}$ in DHNN is to reduce logical inconsistency. Our main objective of this work is to minimize the cost function, ideally driving it to zero. This is accomplished by minimizing the network's cost function. The cost function ${E_{\, {W_{\, r3SAT}}}}$ for the implementation of the logic ${W_{r3SAT}}$ into DHNN can be formulated as shown in Eq (19) below:

where $u$ is the number of first-order clauses and $v$ is the number of third-order clauses. The inconsistency of $Q_{\, ij}^{\, (k)}$ ($k = 1, 3$) is determined as follows:

Here, ${S_{{A_{\, i}}}}$ is the neuron state and ${A_{\, i}}$ comprised of bipolar literals ${A_{\, i}} \in \left\{ { - 1, 1} \right\}$. To achieve a minimized cost function that approaches zero, the DHNN must identify at least one interpretation that satisfies the conditions and yields the result of ${E_{\, {W_{\, r3SAT}}}} = 0$. The formulation of probability ($P$) of finding the satisfactory interpretation is presented in Eq (21) below ^[22]. $\left({1 - \frac{1}{{{2^k}}}} \right)$ is the probability of satisfying the $k$th-order clauses.

The value of ${E_{\, {W_{r3SAT}}}}$ depends on the number of unsatisfied clauses in ${W_{r3SAT}}$. When the number of unsatisfied clauses increases, the value of ${E_{{W_{r3SAT}}}}$ also increases. The minimum value of ${E_{{W_{r3SAT}}}}$ indicates the structure is minimized and the optimal values of the synaptic weights are obtained. ${E_{{W_{r3SAT}}}} = 0$ indicates that all the clauses in this logic are satisfied. It is ensured that the DHNN can converge to the correct final state. According to Wan Abdullah's method (WA) ^[11], the synaptic weights of the neurons can be determined by comparing the coefficients of the cost function with the Lyapunov energy function of the DHNN. The values of the synaptic weights can be stored in the Content Addressable Memory (CAM). Equation (22) represents the network's local field.

DHNN utilized an activation function of the Hyperbolic Tangent Activation Function (HTAF) to enable the convergence of final neuron states because of its non-linear properties. According to the value of Eq (22), Eq (23) is the generated final neuron states.

The initial state and the updated state of the neuron is represented as ${H_i}$ and ${H_i}(t)$ as shown in Eqs (22) and (23). $W_{\, i}^{\, (1)}$, $W_{\, ij}^{\, (2)}$, and $W_{\, ijk}^{\, (3)}$ represent the synaptic weights of the first-order, second-order and third-order in DHNN, respectively. The equation for HTAF is shown in Eq (24) by Mansor ^[35]:

and the neuron states are updated by Eq (25) below:

When the evaluation of the cost function primarily depends on the stable state of the neurons, its minimum corresponds to the optimal solution for the given structure. An analysis conducted by Ma et al. ^[36] demonstrated that the Lyapunov energy is finite and can attain a minimum energy value, further supporting the convergence of the system to an optimal state. The Lyapunov function is formulated as follows:

After the calculation of all the solution states for the ${L_{\, W}}$, the result can be concluded that the minimum of all the ${L_{\, W}}$ can be obtained by the number of clauses. The minimum value can be calculated as follows:

where $u$ and $v$ show the number of first-order and third-order logic. Equation (27) demonstrates that finding the minimum value of the Lyapunov function does not require the use of ES; instead, it can be determined based on the number of first- and third-order clauses. This equation provides a simpler method for calculating the minimum value. Furthermore, the optimal synaptic weights can be derived from the coefficients of the functions when ${E_{\, {W_{\, r3SAT}}}} = 0$ or $\frac{1}{2}(1 \pm {S_{{A_{\, i}}}}) = 0$ per clause.

Otherwise, the synaptic weights would be obtained randomly. When $L_{\, W}^{\, \min }$ is the expected global minimum energy solution of the DHNN, the final state will also be converged to the lowest energy of the neuron states. The converges of the final neuron state are calculated using Eq (28):

where $Tol$ is a predetermined value of tolerance. Although the smaller the tolerance value the better, it can always be set as $Tol = 0.001$ ^[23].

Algorithm 1 illustrates the implementation of Weighted Random 3 Satisfiability into DHNN. Equation (28) determines the condition that can be used to detect the behavior of satisfying ${W_{r3SAT}}$. Figure 2 illustrates the schematic diagram showing the implementation of ${W_{r3SAT}}$ in DHNN. The main block represented shows the first-order and third-order clauses after adding the ratio of negative literals. Inside the higher-order logic block, the blue, yellow, and pink lines indicate the connections between the neurons. The diagram consists of the learning phase and the retrieval phase. The clauses are selected randomly during the learning phase for ${W_{r3SAT}}$ and then converted into Boolean algebra representation.

Figure 3 demonstrates the flowchart of weighted r3SAT in DHNN. The presentation illustrates how weighted r3SAT is progressed obtaining the synaptic weight in the learning phase. It also indicates the retrieval phase of r3SAT in DHNN. The analysis presented herein confirms the objective of weighted r3SAT is to determine the neuron states that meet the requirements satisfied by the cost function given in Eq (20). Note that optimal global solution is a strong assumption for a general unconstrained optimization problem. However, the primary advantage of using neural networks is their powerful representational capabilities. Therefore, the global minimal solution is a reasonable assumption in DHNN. After selecting the negative literals for each clause, the optimal synaptic weight can be obtained by optimizing the neurons. The acquired synaptic weights are then used to access the desired information during the retrieval phase.

5.   Experimental setup

The following work proposed various performance metrics to evaluate the logic output across all phases. The purpose is to judge the effectiveness of adding the ratio into weighted r3SAT.

5.1.   Simulation platform

All simulations were conducted using Python 3 on a 64-bit Windows 11 operating system. All experiments were performed on the same computer with the same compiler. The neuron states were generated randomly by the neuron ${S_i} \in \left\{ { - 1, 1} \right\}$. In the learning phase, the ratios of negative literals are generated by the algorithm ES. Although ES may require significant computational time, it ensures the optimal number of negative neurons. The primary objective of ES is to determine the correct number of negative literals. Third-order clauses in DHNN are crucial for capturing non-linear relationships among variables, thereby extending two-way interactions. These clauses facilitate the handling of complex data patterns, thereby enhancing the accuracy of the models ^[37]. Several simulations were conducted to evaluate the effectiveness of the proposed satisfiability in both the learning and retrieval phases. Three distinct simulations are discussed in this study, which vary in terms of the number of logic combinations, neurons, and clauses. Additionally, different ratios of negative literals were also tested. Error analysis and neuron similarity analysis are utilized to estimate third-order clause performance in DHNN.

5.2.   Parameters assignments

The proposed model will undergo three parts: the logic phase, the learning phase, and the retrieval phase. Each specific detail of the simulations is as below:

a) Different numbers of literals and clauses. The objective of the logic phase is to obtain the right or at least the optimal weighted value of $\eta$ in the subsequent section. The effectiveness of different logic orders is assessed using various metrics during the learning phase.

b) Various ranges of ratios. Different ratio means different numbers of negative literals which can be used to evaluate the impact on ${W_{r3SAT}}$. The performance metrics at every phase can be evaluated and the effects of parameter alterations on ${W_{r3SAT}}$ can be determined.

c) Different logic structures. The proposed approach shows several existing well-known logic rules that are used to compare with ${W_{r3SAT}}$ in terms of the flexibility analysis of logic rules. Comparisons were performed using different ranges of neuron numbers as well as different metrics for the mentioned logic rules.

In the logic phase, ES is implemented to obtain the right number of negative literals. To avoid the early convergence of the algorithm, the size of the neuron population is set to 100. The higher number of neurons will lead to a high risk of gaining optimal global solutions ^[38]. When the ratio of negative literals during the logical phase is determined, the number of negative literals is also confirmed. Tables 1–3 show the list of parameters that are used in each phase of weighted r3SAT. The proposed model includes three components: the learning phase, retrieval phase, and testing phase. Table 1 shows the parameters involved in the logic phase. Tables 2 and 3 present the parameters used in the learning phase and the testing phase. ES will be implemented in the logic phase to obtain the correct number of negative literals. The number of $\eta$ can be obtained by the rounding function of $\left[{rN} \right]$ while the literals mentioned in the retrieval and testing phase are both generated randomly.

The ratios of value $r$ have been preset between 0.1 and 0.9, and the step size is set to $\Delta r = 0.2$. The research by Sidik et al. ^[39] has shown that the selection of the step size plays a crucial role in the process. Additionally, the alignment of the step size with the overall method is essential for ensuring optimal performance. While $r = 0$ means there are no negative literals in the logic structure, $r = 1$ determines all the literals are negative in the logic phase in ${W_{r3SAT}}$. The activation function used in the testing phase is the non-linear HTAF because of its discriminability and compatibility with bipolar entries.

5.3.   Performance metrics

For each phase of this work, various metrics are utilized to value the performance of the model. The performances of all these models are evaluated by variables of metrics that are well known. As each phase has a different objective, each objective will be illustrated for the actual performance of the proposed model. To obtain the correct synaptic weights of ${W_{r3SAT}}$, the cost function must be minimized. The quality of the final states can be evaluated by Eq (28) in the case of the quantity of global minimal solutions. Note that, the smaller the cost function, the better the logic structure. Additionally, the bigger the ratio of global solutions, the better. The optimal value of best fitness in the logic phase is related to the structure of the logic ${W_{r3SAT}}$. Mean absolute error (MAE), root means square error (RMSE), and mean absolute percentage error (MAPE) are well known and are widely used to assess the measurement of the performance of logic models ^[40].

These metrics can measure the effect of the performance of the models and systems. MAE seems a more direct measurement of error. MAE is less affected by outliers and is relatively simple to calculate. As MAE is less sensitive to extreme values, it can provide a more stable evaluation of model performance, especially when the data has uneven distribution or contains noise ^[41]. MAPE presents the percentage value of the proposed model ^[42]. RMSE is more sensitive to larger errors. RMSE has several mathematically advantageous properties. For example, it is expressed in the same units as the data, making the scale of the error more intuitively apparent in practical applications. RMSE is one of the standard methods for evaluating many statistical and machine learning models, thus, it has general applicability in model comparison and selection. The lower the value of these errors, the better the learning and retrieval phase. The effectiveness of the logic phase can be investigated in Eqs (29)–(31) as below:

where ${f_N}$ represents the best fitness of the logic phase and ${f_C}$ represents the current fitness. ${N_i}$ denotes the current number of learning trials. Equation (29) evaluates the effectiveness of the average of the performance for literals which can achieve the situation of ${f_N} = 0$. The discussions regarding the error are RMSE and MAPE, which are shown in Eqs (30) and (31). Both formulations can be used to evaluate the effectiveness of the fitness.

The metric MAE is a standard error based on the average difference between the computed fitness values and the expected fitness of the model we have mentioned. One of our aims is to focus on the effectiveness of ${W_{r3SAT}}$ logic model by different ratios of negative literals for total. To highlight the variations across different phases of the logic, the performance of each evaluation metric was analyzed. RMSE and MAPE, an extended version of MAE that incorporates percentage values, were discussed across the learning, training, and testing phases.

In the learning phase, the best fitness will be gained when Eq (16) is equal to zero. Additionally, the lower the value of such metrics mentioned above, the better the logic model will be. When the effectiveness of the logic phase is evaluated by Eqs (29)–(31), we can also measure the fitness of neuron states and the clauses that generate the optimal synaptic weights. These metrics are all used to assess the performance of the training phase to evaluate the errors in synaptic weight analysis. Equations (32)–(34) shows the parameters used in the training phase as below:

${W_i}$ is the synaptic weight obtained currently and $W$ is the synaptic weight obtained by the WA method. ${N_W}$ is the number of synaptic weights obtained during the retrieval phase each time. ${N_{TW}}$ is the total number of synaptic weights. The energy analysis is used to determine the efficiency of the weighted r3SAT in the testing phase, and the RMSE, MAE, and Global ratio are all used to evaluate the errors. Worth mentioning that, the smaller error leads to better logic and also a higher global optimization. When the signal ${N_{GS}}$ denotes the number of global solutions and ${N_{LS}}$ denotes the number of local solutions, the logic model is considered satisfied when the error values in Eqs (29) to (34) are equal to zero. In addition, the metrics determined in Eqs (35)–(37) showed the effectiveness of the parameters used in the retrieval phase.

Here, ${N_T}$ is the number of testing trails. ${N_T} \times {N_C}$ is the total number of solutions. ${N_{GS}}$ shows the number of solutions that can achieve the global minimum energy in the logic model. To make certain that the final neuron state can be satisfied, the global minimal solutions must be noticed. Guo et al. ^[25] have proposed a metric named Global Minimum Ratio (GR) to evaluate the energy of the network in recent work. The higher the value of GR and the lower value of the parameters mentioned in Eqs (35) and (36) mean a better satisfiability of the model and also an optimal model.

5.4.   Similarity analysis

Analyzing the similarity index can evaluate the quality of the final states of the model. The Jaccard Index (JAC) is a ratio of similarity between two distinct data sets. The calculation of the JAC is straightforward and computationally efficient, making it suitable for large-scale datasets. Fletcher et al. ^[43] used the JAC to measure the similarity between sets of patterns by converting each pattern into a single element within the set. The measurement of the work of ^[43] focuses on providing conceptual simplicity, computational simplicity, interpretability, and wide applicability. JAC reflects the ratio of the intersection to the union of the two sets, making it intuitive and easy to understand in practical applications ^[44]. It is used to evaluate the global solutions in the proposed approach. The similarity index JAC is defined in Eq (38) as follows.

Note that $\alpha$ is the number of states where $\left({S_{\, i}^{\, \max }, {S_{\, i}}} \right)$ when both the elements are equal to 1. $\beta$ is the number of the set when ${S_{\, i}} = - 1$ and $S_{\, i}^{\, \max } = 1$. $\gamma$ is the number of $\left({S_{\, i}^{\, \max }, {S_{\, i}}} \right)$ when ${S_{\, i}} = - 1$ and $S_{\, i}^{\, \max } = 1$. Since the similarity index serves as a measure for evaluating the quality of the final states retrieved, the proposed model will utilize this formulation of the similarity index ^[45] to assess the final neuron states. As previously mentioned, the Jaccard (JAC) similarity analysis will be employed to compare the neuron states, as outlined in Table 4.

5.5.   Baseline logics

To evaluate the performance of the proposed logic, the formulated logic model is compared with the baseline SAT models mentioned below:

(a) Random 2 Satisfiability (RAN2SAT) ^[22]: This study was first proposed by Sathasivam et al. and was mentioned as the earliest work on the non-systematic logic rule. RAN2SAT consists of both first and second-order logic clauses. This approach broadens the potential to transition from systematic to non-systematic logic, with the latter being more applicable for real-world implementation. Consequently, no work was performed on the measurement of negative literals, nor was there a logic phase involved.

(b) Random 3 Satisfiability (RAN3SAT) ^[23]: RAN3SAT is an expanding work of RAN2SAT which concluded third-order logic in the model. It consisted of three components: The first, second, and third-order clauses. The work by Karim et al. ^[23] noticed three types of logic combinations with different $k$. The literals were generated randomly while the clauses could be defined by the user. This approach significantly enhanced RAN2SAT by incorporating 3SAT, thereby making the structure more flexible. The researchers addressed the limitations of RAN2SAT in existing non-systematic logic, but it did not account for measuring the number of negative literals in the logical model.

(c) G-type Random $k$ Satisfiability (GRAN3SAT) ^[26]: GRAN3SAT was first put forward by Gao et al. ^[26] as a flexible logical rule by combining the structure of both systematic and non-systematic logic. It was enumerated randomly with a higher order logical structure including the first-order, second-order, and third-order of clauses. The present research GRAN3SAT can be reduced to systematic logic 2SAT, 3SAT. We focused on the flexibility of the logical structure but overlooked the consideration of the weight of the negative literals.

(d) Weighted Random $k$ Satisfiability (r2SAT) ^[28]: The work proposed by Zamri et al. considered a class of non-systematic satisfiability logic with the inclusive ratio of negative literals. The approach emphasizes the distribution of negative literals using GA to generate the logic phase. However, this paper only considers the weighting of the structure in first- and second-order clauses. Notably, third-order logic has not been considered in the proposed framework.

6.   Results and discussion

The proposed logic model was compared with state-of-the-art logic models in various aspects during the following work. Each phase of the proposed model will be assessed. Including the learning phase (logic phase), training phase, and the testing phase. ES is embedded into the logic phase to obtain the best value and the correct number of negative literals. It will be used to calculate the right number $\eta$ with different ratio $r$, respectfully. The ratio of negative literals mentioned herein results different cases to compare with each other. For example, when $r = 0.3$, it means the negative literals have a percentage of 30% while the other 70% are positive. After the logic phase ${W_{r3SAT}}$ was generated, ${W_{r3SAT}}$ was able to be trained to minimize the cost function embedded in DHNN. The metrics MAE, RMSE, and MAPE have been introduced in Eqs (29)–(31) and were used to evaluate the effectiveness of the logic model. To avoid the complexity of the comparison, the number of literals was set up to 100. This drawback of the number controlling is tolerant of a new logic that was first proposed. After the generation of the logic phase, the obtained logic was trained in DHNN. Equations (32)–(34) can measure the effectiveness of the synaptic weight. Equations (35)–(37) was used to evaluate the quality of the proposed model in reaching the global minimum solutions. The comparison between the proposed model and exit SAT models in DHNN was analyzed. Moreover, the similarity of the benchmark station and the final neuron station is measured in terms of the JAC similarity index.

6.1.   Analysis of logic phase

The purpose of the logic phase is to find a correct logic structure that has a strong ability for consistent interpretation. The proposed approach utilized ES to gain the number of negative literals for different ratio sets. The learning phase of the proposed model is specifically designed to achieve the desired number of negative literals, as formulated in Eq (15). It has been noted that r2SAT is unable to retrieve the correct structure when the number of neurons exceeds 45 for any negative ratio ^[28]. Due to the utilization of ES, the proposed work can obtain an optimal structure for all $r$. Herein, the performance metrics $MA{E_l}$, $RMS{E_{\, l}}$, and $MAP{E_{\, l}}$ shown in Eqs (29)–(31) are computed to measure the effectiveness of the clause fitness.

As the performance metrics were used to analyze the fitness of neuron states, the value of these metrics can also be referred to as the fitness error. Note that the main object of the learning phase is to search for the optimal ratio of negative literals that can achieve optimal fitness. Figure 4 illustrates the comparative effectiveness of different ratios in the learning phase in the logic ${W_{r3SAT}}$. The proposed value for a consistent step of ratio is determined as the symbol of $\Delta r$. To better observe the overall situation, we set the step of ratio equal to 0.2. Figure 4 was generated based on the data calculated under this condition shows that it can be concluded that the ratio $r = 0.7$ plays a good view in the proposed model while the step of ratio $\Delta r = 0.2$. It can achieve a smaller error to get the best fitness. Generally, when the number of neurons is below 20, the errors increase linearly as the number of neurons grows, except for certain values of $r$. However, when the number of neurons is between 20 and 40, the errors decrease. As the number of neurons continues to increase, the fitness error also increases. This trend has been observed and is consistent with the findings of Karim et al. ^[23] and Zamri et al. ^[28].

Figure 4(a) represents the mean absolute error (MAE) during the learning phase, while Figure 4(b) illustrates the root mean squared error (RMSE) and Figure 4(c) demonstrates the mean absolute error (MAPE) during the learning phase with different value of the ratio. Although the differences based on different ratios has shown quite diminish when the number of neurons increased, it can be demonstrated that the ratio of negative literals plays a great part in the contribution of logics. It is worth mentioning that when the ratio gets its extreme value in the range of $\left[{0.1, \, 0.9} \right]$, the solutions of the combination would not be very multiple. For example, when $r = 0.1$ or $r = 0.9$ and the number of literals is set to 15, the number of negative literals should be counted. The number of negative literals would equal to 1 under the condition of $r = 0.1$ and equal to 13 when the ratio $r = 0.9$. With no ordering arrangement of the literals where the negative ones should be, the possible combinations of ${W_{r3SAT}}$ is $\left(\begin{array}{c}15\\ 1\end{array}\right) = \frac{15}{1!} = 15$ or $\left(\begin{array}{c}15\\ 13\end{array}\right) = \frac{15\times 14\times 13\times \cdots \times 3}{13!} = 105$ respectively. When the value $r = 0.5$, the number of negative literals would be calculated and is equal to 7, while the number of possible ${W_{r3SAT}}$ would increase to $\left({\begin{array}{*{20}{c}} {15} \\ 7 \end{array}} \right) = 6435$. As the number of literals increased, the combinations would increase sharply. These differences in the structure are based on the division of negative literals. A huge amount of partition of literals would cause a huge number of states that must be noticed. In the study of this work, the set of combinations is set up randomly, and the states of the logic are selected randomly also. The evolutionary algorithm ES works effectively when the number of literals is small. More iterations are needed in the learning phase to generate the right logic when the size of neurons is large. It can be noticed that when the work was focused on the specific and exact direction where the negative literals are, there would be a large probability leading to a mismatch in the approximating process. In the work of Zamri et al. ^[28], once the process is unable to minimize the near-optimal solution, the generated logic fails to proceed to the training phase of DHNN. The proposed model did a good work of solving this confusing by selecting both the logic structure and negative literals randomly. After calculating the number of negative literals, the direction of the negative ones was distributed randomly and as the number of first or third-order clauses. In addition to the analysis of the learning phase, which aims to achieve a correct logic phase that leads to a zero-cost function, it can be concluded that the model is capable of generating the optimal structure of ${W_{r3SAT}}$ with each value of $r$ mentioned in the present research.

Table 5 shows the list of MAE for the proposed logic model ${W_{r3SAT}}$ and other models that are used to compete in the learning phase. The following work shows that all the logics mentioned are non-systematic logics such as RAN2SAT, RAN3SAT, GRANSAT, and r2SAT. RAN2SAT included the following situations of clauses with first-order and second-order. RAN3SAT set an implementation of third-order clauses into RAN2SAT, which is higher ordered. GRAN3SAT did the work that the logic model has clauses set up with first-order, second-order, or both of these two orders. In the case of added weight which is the ratio of negative literals in the logic rules included first and second-order clauses. ${W_{r3SAT}}$ has the capability to contain the clauses both first and third-order. It has both positive and negative literals also. Although the ratio of the negative literals was first determined in the logic named r2SAT ^[28], it was the first time adding third-order clauses in the structure. Furthermore, the direction of negative literals was selected randomly. This random selection would make a more flexible model which can be better utilized in real-life implications. The comparison with these non-systematic models shows the drawback of the roles first-order clauses played in the logic rules.

The marked values which are in highlighted values represent the best solution of the learning phase which has the lowest errors under the condition of different numbers of neurons. As the performance metric $MA{E_l}$ can utilized to evaluate the different fitness of the neuron state, the value of $MA{E_{\, l}}$ refers to fitness error. Table 5 demonstrates the value $MA{E_{\, l}}$ of the proposed model ${W_{r3SAT}}$ and the comparison with other non-systematic logic rules. Especially for the condition of r2SAT, the value in the table shows only under the condition of $r = 0.6$. The following tables are the same as above. The value of $MA{E_{\, l}}$ increased when the number of neurons increased. That means it is more difficult to obtain the optimal solutions satisfied ${E_{{W_{r3SAT}}}} = 0$ with a high value of N. When the number of neurons is smaller than 20, under the condition of $r = 0.1$ can generate the best model of logic which means it can obtain the optimal fitness obviously.

Figure 5 shows the value of different metrics for each ratio of ${W_{r3SAT}}$ and the average value of these metrics for r2SAT in the cases of all the ratios during the learning phase. While higher MAPE indicates a lower degree of accuracy, the smaller value of MAPE can obtain a better forecast ^[46]. The errors for each ratio of ${W_{r3SAT}}$ are relatively close due to the similar structure of first and third-order logic. When the number of neurons is below 30, all the logics have an instability veritable MAPE. When the number of literals increases, the error tends to be more linearly increasing no matter with the structure of the models. Putting this matter of adding ES aside, all the logics can achieve a lower error when the number is bigger than 60 which can be concluded to emphasize a better synaptic weight management. Although r2SAT performs well in incorporating the ratio of negative literals during the learning phase, it achieves a MAPE value close to 1. A MAPE value of 1 indicates a significant difference between the maximum fitness and the current fitness ^[23]. Achieving a smaller MAPE percentage is a key objective during the learning phase for all logic, particularly for the proposed model ${W_{r3SAT}}$.

6.2.   Analysis of learning phase

The analysis of the average errors for each metric indicated that the proposed model did better work than most of the existing models. Figure 6 Displayed the average value of each error metrics. There is only one model named RAN3SAT that is competitive with these ${W_{r3SAT}}$ models. It is noticeable that the logic RAN3SAT is more consistent than the other non-systematic models shown above except the ${W_{r3SAT}}$ with ratio $r = 0.2$, 0.4, 0.6 and 0.8 in Figure 5. The phenomenon is interesting that when the ratio of negative literals is set to be $r = 0.3$, 0.5, 0.7 and 0.9 the average value seems the same with RAN3SAT. In the work of ^[47], it was emphasized that a constrained number of learning iterations does not improve the training phase to achieve higher fitness of neuron states. Therefore, the proposed logic is expected to benefit this study.

6.3.   Analysis of retrieval phase

Our goal of this section is to discuss and evaluate the effectiveness of the proposed model and different models during the retrieval phase of DHNN. The results of the analysis during the retrieval phase are presented in the next part. Four metrics are presented to evaluate the performance of retrieval errors upon the proposed logic rule and other non-systematic rules. The value of global minimum solutions (GR) can achieve totally near 1 with a low value of N. This is because the lower number of clauses tends to be satisfied in the learning phase. As the number of literals increases, RAN2SAT and r2SAT are diminished straightly in terms of the literals selecting are random and both of the logic have difficulties to be satisfied. Furthermore, the value of GR quickly reduced to almost zero when the number of neurons increased to more than 50. As mentioned in the work of Guo et al. ^[25], the value of GR would crash because of the first-order clauses generated randomly. This shows more learning iterations are acquired to satisfy the interpretation of this logic. Figure 7 depicts the values of MAE (a), RMSE (b) and MAPE (c) for all logic models mentioned here. These metrics are utilized to analyze the effectiveness of different ratios of ${W_{r3SAT}}$ and all the other models. ES was also used in the learning phase to search for fitness. It is worth mentioning that ES was performed on a trial-and-error basis. Based on Figure 7, the conclusion is that when the ratio of ${W_{r3SAT}}$ is $r = 0.2$, 0.4, 0.6, and 0.8, the errors are almost zero when $N \leqslant 20$. On the other hand, the errors are not very high when the number of neurons is not bigger than 40. Additionally, the fluctuation waved strongly when N increased. As the number of neurons increases, the capability of DHNN leads to the global optimal solutions decrease. This causes DHNN to store the synaptic weight randomly which will lead the final states to local minimum solutions. As the results are shown in Figure 7, the errors are almost nearly zero when $N \leqslant 40$ for all the corresponding ${W_{r3SAT}}$ models. Besides, RAN3SAT obtained the right synaptic weights take contrast with RAN2SAT and r2SAT. GRAN3SAT also expressed a good performance in that situation. There is an interesting observation that all the values of GR are within a high level when the number of neurons is no more than 50 except RAN2SAT and r2SAT. For the proposed model ${W_{r3SAT}}$, the value is at a high level even though the number has increased to 70. These can be seen in Table 6. Where Table 7 shows the values of MAPE for different logic structures. Overall, the graph in Figure 8 and the values shown in Table 8 demonstrate that after adding the ratio of negative literals better GR can be achieved compared to other logics. It is noteworthy that the values in the table for r2SAT are only expressed when the ratio is $r = 0.6$ while the figure shows the average value of all the ratio contributions.

In the simulations for the proposed model, both the number of possible combinations and the distribution of negative literals were randomly selected. As the number of neuron states increases, achieving a zero-cost function that yields the optimal solutions becomes increasingly challenging. The proposed model can be improved by a high number of learning and then achieve the maximum fitness of neuron states.

6.4.   Analysis of testing phase

The following section demonstrates how the proposed model will be evaluated based on the management of synaptic weights. The efficiency of the testing phase will be indicated whether ${W_{r3SAT}}$ can successfully attain the optimal synaptic weights and produce global minimum solutions with the final neuron states. As shown in Table 6, note that the proposed logic model added a negative ratio first, and the value $\eta$ can be calculated using Eq (14), the proposed model demonstrates the same training method. ${W_{r3SAT}}$ can attain minimum errors of nearly zero for the ratio $r = 0.2$, 0.4, 0.6 and 0.8 when the number of neurons is less than 50. The same circumstances have happened in the learning phase. Table 7 shows that the proposed logic has the lowest error value of MAPE when the number of neurons increases not less than 60. The error fluctuation occurred when the number of literals is continually increased but not because of the differences in the ratio. The results shown in Figure 9 indicate that the MAE errors for RAN2SAT and r2SAT are at a very high level. This is because both logic structures have first and second-order clauses randomly. When the number of neurons increases, especially more than 50, the error can reach 1, so it appears that the global minimum solutions in the final states cannot be obtained. Strictly speaking, the model can obtain global minimum energy based on the optimal synaptic weight. Based on the values shown in Figure 10, RAN2SAT has the worst average value of GR. The reasons for this depend on the low capability of obtaining a consistent interpretation of non-systematic models. Research by Huang et al. ^[48] suggested that the implementation of improved differential evolution (IDE) can do some advantages in convergence speed and reduce optimization time for global searching in the testing phase. No matter how many neuron literals there are, MAE for testing seems similar between RAN3SAT and ${W_{r3SAT}}$. This interesting phenomenon occurs due to the structure of both RAN3SAT and ${W_{r3SAT}}$ are selected randomly and they both have the fortune to generate high potential of third-order clauses. Moreover, ${W_{r3SAT}}$ can sometimes be generated as an extremely special item of RAN3SAT. Besides, first-order logic which is one of the important structures in the proposed logic plays a great role in disturbing the process of retrieving and leading to a high level of testing error.

There is another interesting point that has been underlined by Karim et al. ^[23], which is that RAN3SAT with first and third-order clauses has reached the highest error value compared with other logics. However, it is quite different after adding the ratio of negative literals to the proposed logic model. As shown in Figure 9, although ${W_{r3SAT}}$ has components with first and third-order logic, it does have a lower value of error when the number of neurons grows larger than 30.

6.5.   Similarity analysis

The Jaccard similarity index (JAC) can be utilized to check the similarity between different data sets. It has been used to evaluate the global solutions in the work by Bag, S. et al. ^[49]. It was also suggested to assess the performance of logic with DHNN by the work ^[50]. JAC is a similarity ratio that measures the quality of final states, making it a standard parameter for indexing. In Figure 11, it can be observed that r2SAT has the highest value when $25 < N < 35$ and $65 < N < 75$. Sharp decline happened when the number of neurons increased by more than 40. After the number reached 60, large fluctuations occurred for the logic r2SAT. This fluctuation does not only bechance in r2SAT but also took place in RAN2SAT and GRAN3SAT for different ranges of neurons. Compared with the existing model, ratio-adding logic ${W_{r3SAT}}$ has a gentler variation with each mentioned ratio in the proposed model. RAN3SAT has a similar wave compared with the proposed model. The waves of the JAC values for each ratio of the Weighted Random Satisfiability show parallel lines in Figure 11. The oscillations for RAN2SAT, GRAN3SAT, and r2SAT are zero at $N = 70$. Even though ${W_{r3SAT}}$ is satisfied, the proposed model would fail to achieve the optimal states. As the local solutions increase in terms of the neuron increase, the ratio plays an important role in distributing the effectiveness. The simulation stops at $N = 70$ is enough for other baseline logic, it is not abundant for the logic ${W_{r3SAT}}$. Thus, the number is settled to 100. It is noteworthy that as the ratio of negative literals increases, the JAC similarity value changes in parallel. The higher the value of the ratio, the larger the similarity index.

Figure 11 presented the total trend is in the range of $\left[{0.50, \, 0.65} \right]$ when the ratio $r \in \left[{0.1, \, 0.4} \right]$. Under the situation $r \in \left[{0.5, \, 0.8} \right]$, the result reduces to $\left[{0.27, \, 0.47} \right]$. Furthermore, the value is declined by almost 0.1 when the ratio is up to 0.9. Despite the influence based on the diversification of the ratio in ${W_{r3SAT}}$, oscillations happened when the number of neurons is below 30. It needs to be noticed that the frequent oscillations occur as the neurons increase under the condition $r = 0.7$. Moreover, $r = 0.5$ indicates the gentlest diversification. As the metric JAC is aimed at neuron variation produced by ${W_{r3SAT}}$, additional similarity index should be invited. The logic rule that achieves the highest global optimum is also the most effective for logic mining. The final neuron state can be converted into the maximum induced logic ^[23]. The benefits of the proposed logic offer significant advantages for real-life applications.

From the discussion mentioned above, the definition of minimizing the cost function, generating the optimal synaptic weigh, and the variation of final neuron states of ${W_{r3SAT}}$ were all determined. Adding the ratio of negative literals presents better in obtaining higher global minimum solutions and affirming final neuron states. We outline that when adding the ratio $r = 0.7$ present the best logic model with producing lower errors and higher global minimum solutions. On the other hand, due to the learning phase of algorithm ES, this study limits it to 100 neurons. Overall, we can improve this limitation of the proposed model by adding other metaheuristics such as GA and EA (Election Algorithm). Despite the flexibility of ${W_{r3SAT}}$ in DHNN, additional modifications can be done to improve the effectiveness of solutions. This means to not only increase the number of iterations but also decrease the errors and more global solutions.

7.   Open problems

In the field of artificial intelligence, SAT has long been a foundation of theoretical research. Different logic structures and algorithms for SAT have been proposed and extensively discussed ^[18,20,35]. However, while higher-order logic has been broadly explored in computer science ^[20,23,35], higher-order logic adding a negative ratio remains relatively under-explored. Specifically, how to effectively express this negative ratio during the training phase is an unsolved issue. One key problem is how to design new activation functions that can accommodate the property of higher-order logic within DHNN.

Higher-order logic, which contains higher-order clauses, often involves more complex computing power. How to balance the effectiveness of benefit and computation cost remains a difficult task. Current neural network architectures, such as CNN ^[51], RNN ^[52], have limitations when the logical structure becomes more complex ^[53], especially for higher-order logic in high-dimensional spaces. In DHNN which has been discussed herein, the same problem must be solved and the computing time consumption also needs to be considered. Higher-order logic adding a negative ratio would make the network gradients unstable, which could lead to overfitting or gradient vanishing. How to design a model that can keep the properties of the logic in higher order while maintaining stability is a great challenge for further work.

The treatment of Weighted Random Satisfiability with higher-order logic in DHNN is an open problem that leaves a lot of strategies unexplored. The solving of the problems would push the applications of AI in complex reasoning and decision-making systems.

8.   Conclusions

We explore a novel logical model named ${W_{r3SAT}}$, which was proposed with a different logical structure adding a ratio of negative literals as a new instruction in DHNN. The proposed logic rules provide different ratios and the generated negative literals distributed in each clause are randomly. A logic phase will be introduced to aid the generation of the correct Weighted Random $k$ SAT according to the initiated $r$ before being trained in DHNN. Random literals of Weighted Random $k$ SAT will be generated according to the value of the input ratio $r$. The value of $\eta$ will be evaluated according to the value of weight in each clause. Although there are many algorithms for finding the number of negative literals, the minimization will be executed using ES in the logic phase. As the outperformance of the logic has shown that the differences are not quite clear, the step of the ratio chooses the length of $\Delta r = 0.1$. It is worth mentioning that further work can focus on a smaller step and make a more detailed work on the effectiveness of ratio. The results in the training and testing phase portrayed the capability of ${W_{r3SAT}}$ in producing optimal solutions for all values of $r$. The optimal model obtains a lower cost function, less error, and higher satisfaction in each step of the model. On the other hand, more diversity was obtained, which is more effective in real-life research. The comparisons between ${W_{r3SAT}}$ and other non-systematic logic models (RAN2SAT, RAN3SAT, GRAN3SAT, and r2SAT) were quite distinct. The results showed that the proposed model ${W_{r3SAT}}$ can obtain a better performance which indicates that adding a ratio will take advantage of global minimum generations.

For future work, logic mining should be discussed based on the distribution of negative literals which can be set up to the prescribed order of clauses. Researchers have explored log-linear analysis combined with higher-order logic mining during the pre-processing phase ^[54,55]. Additionally, reverse-based analysis logic mining has been investigated, leading to an increase in storage capacity ^[56,57]. Furthermore, logic mining and optimization of the learning phase provided improvement for the next step in research, and many other metaheuristic algorithms could be added in the learning phase to improve obtaining the optimal synaptic weight. Additionally, this can generate global minimum solutions with various final neuron states. Moreover, this can be utilized in many real-life problems.

Author contributions

Conceptualization, methodology, software, data curation and writing-original draft, X.L.; resources and formal analysis, Y.G.; validation and investigation, N.E.Z.; supervision and funding acquisition, M.S.M.K.; writing-review and editing, S.A.; visualization, Y.C.; project administration, M.S.M.K. All authors have read and agreed to the published version of the manuscript.

Use of Generative-AI tools declaration

The authors declare they have not used Artificial Intelligence (AI) tools in the creation of this article.

Acknowledgments

The authors express special thanks to all researchers in the Artificial Intelligence Research Development Group (AIRDG) for their supports and advises sincerely. This research is funded and supported by Universiti Sains Malaysia, Short Term Grant, 304/PMATHS/6315655.

Conflict of interest

The authors declare no conflicts of interest.