Figure 1.
Final individuals obtained by four MaOEAs on DTLZ1 with regular PF and DTLZ5 with irregular PF.
Citation: Herbert F. Jelinek, Jemal H. Abawajy, Andrei V. Kelarev, Morshed U. Chowdhury, Andrew Stranieri. Decision trees and multi-level ensemble classifiers for neurological diagnostics[J]. AIMS Medical Science, 2014, 1(1): 1-12. doi: 10.3934/medsci.2014.1.1
| [1] | Jiawei Yuan . A constraint handling technique using compound distance for solving constrained multi-objective optimization problems. AIMS Mathematics, 2021, 6(6): 6220-6241. doi: 10.3934/math.2021365 |
| [2] | N. Pazhaniraja, Shakila Basheer, Kalaipriyan Thirugnanasambandam, Rajakumar Ramalingam, Mamoon Rashid, J. Kalaivani . Multi-objective Boolean grey wolf optimization based decomposition algorithm for high-frequency and high-utility itemset mining. AIMS Mathematics, 2023, 8(8): 18111-18140. doi: 10.3934/math.2023920 |
| [3] | Guangjian Li, Guangjun He, Mingfa Zheng, Aoyu Zheng . Uncertain multi-objective dynamic weapon-target allocation problem based on uncertainty theory. AIMS Mathematics, 2023, 8(3): 5639-5669. doi: 10.3934/math.2023284 |
| [4] | Ying Sun, Yuelin Gao . An improved composite particle swarm optimization algorithm for solving constrained optimization problems and its engineering applications. AIMS Mathematics, 2024, 9(4): 7917-7944. doi: 10.3934/math.2024385 |
| [5] | Keyu Zhong, Qifang Luo, Yongquan Zhou, Ming Jiang . TLMPA: Teaching-learning-based Marine Predators algorithm. AIMS Mathematics, 2021, 6(2): 1395-1442. doi: 10.3934/math.2021087 |
| [6] | Xinfeng Zhang, Zhibin Zhu, Chongqi Zhang . Multi-stage differential evolution algorithm for constrained D-optimal design. AIMS Mathematics, 2021, 6(3): 2956-2969. doi: 10.3934/math.2021179 |
| [7] | Haowen Gong, Huijun Xiang, Yifei Wang, Huaijin Gao, Xinzhu Meng . Strategy evolution of a novel cooperative game model induced by reward feedback and a time delay. AIMS Mathematics, 2024, 9(11): 33161-33184. doi: 10.3934/math.20241583 |
| [8] | Charles Audet, Jean Bigeon, Romain Couderc, Michael Kokkolaras . Sequential stochastic blackbox optimization with zeroth-order gradient estimators. AIMS Mathematics, 2023, 8(11): 25922-25956. doi: 10.3934/math.20231321 |
| [9] | Peng Wang, Jiajun Huang, Weijia He, Jingqi Zhang, Fan Guo . Maximum likelihood DOA estimation based on improved invasive weed optimization algorithm and application of MEMS vector hydrophone array. AIMS Mathematics, 2022, 7(7): 12342-12363. doi: 10.3934/math.2022685 |
| [10] | Zhuolin Yan, Xiaowei Jiang, Siyao Wang . Objective penalty function method for nonlinear programming with inequality constraints. AIMS Mathematics, 2024, 9(12): 33572-33590. doi: 10.3934/math.20241602 |
In reality, many optimization problems (e.g., cloud scheduling [1], skin cancer detection [2], and recommendation system [3]) can be modeled as a kind of many-objective optimization problem (MaOP) that needs to optimize more than three conflicting objective functions simultaneously. A MaOP has the following general formulation:
| $ minimizeF(x)=(f1(x),f2(x),…,fm(x))subject tox∈Ω⊆Rn, $ | (1.1) |
where $ f_1(x), f_2(x), \cdots, f_m(x) $ denotes $ m $ conflicting objectives, and $ x = \left(x_1, x_2, \cdots, x_n\right) $ represents a $ n $-dimensional candidate solution. Note that when $ m $ is equal to 2 or 3, (1.1) is called a multi-objective optimization problem (MOP). Due to the conflict among objectives in (1.1), a solution cannot optimize all objectives simultaneously. On the contrary, a set of solutions is required to trade off different objectives [4]. We call the trade-off solution set the Pareto set (PS) and its projection in the objective space the Pareto front (PF) [5].
Evolutionary algorithms are capable of obtaining a solution set in a single run, making them popular for many-objective optimization. As a result, evolutionary algorithms have been widely used in different problems, such as prince prediction [6,7], plant classification [8], software defects prediction [9], and engineering optimization [10]. In recent years, many evolutionary algorithms have been proposed to solve MaOPs. Such evolutionary algorithms are known as many-objective evolutionary algorithms (MaOEAs). Existing MaOEAs mainly include dominance relation based MaOEAs, decomposition based MaOEAs, and indicator based MaOEAs.
Among MaOEAs, decomposition based MaOEAs have shown some advantages in dealing with MaOPs. The underlying reason for this is that the reference vectors can guide the population individuals to converge toward the PF from diverse directions, and thus such algorithms can obtain individuals with good convergence and diversity. However, most MaOEAs in this category have a low generality, since most are sensitive to PF shapes. To be specific, these predefined reference vector based MaOEAs cannot perform well on MaOPs with irregular PF shapes, as pointed out in [11,12]. This is because the distribution of predefined reference vectors cannot match that of the PFs of MaOPs with irregular PF shapes. As for the adaptive reference vector based MaOEAs, they may not work well on MaOPs with regular PF shapes. To intuitively present the fact above, we plot the individual distribution situations, where individuals obtained by four state-of-the-art MaOEAs (i.e., RVEA [11], $ \theta $-DEA [13], MaOEA/D-AWA [14], and MOEA/D-UR [15]) for DTLZ1 and DTLZ5 in Figure 1. Note that 1) RVEA and $ \theta $-DEA adopt the predefined reference vectors, while MaOEA/D-AWA and MOEA/D-UR adopt the adaptive reference vector; and 2) DTLZ1 has the regular PF, while DTLZ5 has the irregular PF. From this figure, we can see that RVEA and $ \theta $-DEA perform only well on DTLZ1 and MaOEA/D-AWA, and MOEA/D-UR perform well for only DTLZ5, and therefore verify the truth of the above fact. In addition, most decomposition based MaOEAs consider only the global diversity, while ignoring the local diversity. This may cause most of them to not achieve a satisfactory performance on the global and local diversities at the same time. In order to inherit the advantages of decomposition based MaOEAs and avoid the limitations above in decomposition based MaOEAs, a self-organizing decomposition based evolutionary algorithm with cooperative diversity measure for many-objective optimization is proposed in this work. The major contributions of this paper are as follows:
$ 1) $ A self-organizing decomposition manner is developed to automatically divide the objective space into different sub-regions instead of using the predefined reference points. In such a way, the proposed SDEA can adapt to different Pareto front shapes.
$ 2) $ A cooperative diversity measure is designed, which considers the global and local diversities. Such design is beneficial to improve the global diversity of the population and ensure the diversity among individuals.
$ 3) $ Based on the strategies outlined above, SDEA is proposed. SDEA cannot only inherit the advantages of the decomposition method, but avoid the limitations of the decomposition method. Therefore, the overall performance of SDEA is superior to that of seven state-of-the-art MaOEAs in solving different MaOPs.
In the remaining, Section 2 presents the preliminaries. In Section 3, SDEA is proposed. Section IV describes the experimental setups and results for three benchmark test suites. In Section 5, SDEA is used to solve two practical problems. Finally, a comprehensive conclusion and future works are presented in Section 6.
In section, we present the related work. First, the literature review is given, and then the calculation of two distances is presented.
Existing MaOEAs mostly include the following categories: The first category is the dominance relation-based MaOEAs. In light of the fact that Pareto dominance in NSGA-II cannot provide the sufficient selection pressure to make the population converge to the PF, researchers have proposed many new dominance relations [16]. For instance, some researchers modified the original objective function to improve the comparability among individuals, such as $ \alpha $-dominance [17], C$ \alpha $-dominance [18], and GPO-dominance [19]. Some researchers attempted to employ the fuzzy logic to design new dominance relations, such as (1-k)-dominance [20] and fuzzy-dominance [21]. Given that the evenly reference vectors are beneficial to manage the diversity, some researchers proposed some reference vector based dominance relation, such as RPS-dominance [22], and DR-dominance [23]. Recently, some researchers fully utilized the advantage of the niche technology in maintaining the population diversity to design the dominance relations. Two typical representatives are the strengthened dominance relation (SDR) [24] and controlled strengthened dominance relation (CSDR) [25].
The second category is the indicator-based MaOEAs. Its basic working principle is to utilize the indicator to select elite individuals for the next generation, and thus drive the population evolution. For example, Zitzler et al. [26] developed an indicator based selection framework for MOPs, where the proposed $ I_{\varepsilon} $ indicator acts as the selection criterion. The hypervolume (HV) indicator is characterized by the Pareto-compliant property, so that some HV based MaOEAs have been proposed, such as SMS-EMOA [27], and HyPE [28]. These algorithms are capable of better solving MaOPs, but their computational complexities increase exponentially with the increasing number of of objectives. In [29], the R2 indicator based two-stage MaOEA was proposed, which acts as the selection criterion of the first stage. There are similar algorithms, such as R2HCA-EMOA [30] and MaOEA-FEGL [31]. In [32], the IGD indicator based MaOEA (MaOEA-IGD) was developed, where the IGD indicator cooperates with the efficient dominance comparison method to select individuals.
The third category is the decomposition-based MaOEAs, which decomposes a MaOP into some sub-problems and then solves them cooperatively. The classic representative of this class is MOEA/D [33], which can perform well on MOPs/MaOPs. Therefore, based on the framework of MOEA/D, some MaOEAs have been developed. For example, [34] designed a collaborative resource allocation strategy in the framework of MOEA/D for MaOPs. There are some similar algorithms, such as MOEA/D-PaS [35], and MOEA/D-LWS [36]. In [11], a reference vector guided evolutionary algorithm (RVEA) was developed, where an angle penalty distance was designed to evaluate the comprehensive performance of individuals. In [37], a reference vector based density estimator was proposed, where the reference vectors partition the objective space into some subspaces for evaluating the diversity. [38] proposed a dynamical decomposition-based method, where individuals act as the reference vectors instead of the predefined ones. The above algorithms have shown some advantages in solving MaOPs. However, they seem to become invalidated, when dealing with MaOPs with irregular PFs. This is because the used reference vectors are predefined. Therefore, some adaptive reference vectors based MaOEAs have been developed, such as AdAW [12], RVEA-iGNG [39], MOEA/D-UR [15], and SPEA/ARP [40].
Apart from the aforementioned three classes, there are some other MaOEAs. For instance, some researchers utilized the advantages of the angle vector in the high-dimensional space to develop some MaOEAs, such as MaOEA-CSS [41], AnD [42], UIMaOTO [43], and 3DEA [44]. In addition, some MaOEAs assisted by the machine learning, reinforcement learning, and deep learning have been proposed, such as MaOEA-DPP [45], RL-RVEA [46], and MOCELA [47].
This selection aims to introduce the calculation of two distances, including the projection distance ($ d_1 $) of an individual on a reference vector and the perpendicular distance ($ d_2 $) between an individual and a reference vector, as presented in Figure 2.
Before calculating the $ d_1 $ and $ d_2 $, the objective values of an individual $ x $ are normalized by:
| $ fi(x)=fi(x)−zminizmax−zmin,i=1,2,⋯,m $ | (2.1) |
where $ z_i^{\min } = \min _{x \in P}\left(f_i(x)\right) $ and $ z_i^{\max } = \max _{x \in P}\left(f_i(x)\right) $. Note that, in order to avoid the denominator in formula (2.1) becoming zero, is set to zero when $ z_{i}^{\max }-z_{i}^{\min } < 1e-6 $.
After normalizing, the projection distance ($ d_1 $) and the perpendicular distance ($ d_2(x) $) of an individual $ x $ can be calculated by:
| $ d1(x)=‖f(x)Tr‖‖r‖ $ | (2.2) |
| $ d2(x)=‖f(x)−d1(x)r‖r‖‖ $ | (2.3) |
where $ r = (r_1, r_2, \cdots, r_3) $ is a reference vector.
In this section, we describe the proposed SDEA. The framework of SDEA is given, and then the main components of SDEA are presented. Finally, the computational complexity of SDEA is analyzed.
Algorithm 1 presents the pseudo code of the main framework of SDEA. To be specific, SDEA randomly generates $ N $ individuals as the initial population $ P $, and then employs the developed self-organizing decomposition to generate $ N $ reference vectors. Subsequently, SDEA enters the iteration procedure until the termination condition is met, where the termination condition refers to the maximum function evaluations ($ maxFEs $). In the procedure, SDEA first implements the genetic operator, including the random mating selection, simulated binary crossover (SBX) [48], and polynomial mutation (PM) [49] to generate the offspring population $ O $ with $ N $ individuals. Then, the populations $ O $ and $ P $ are combined to form the combined population $ U $. Next, the environmental selection of SDEA is implemented to pick out $ N $ elite individuals for the next generation. Finally, the developed self-organizing decomposition is employed to generate $ N $ reference vectors.
| Algorithm 1 Framework of SDEA |
|
Require: $ N $ (population size), parameter $ \delta $
Ensure: $ P $ (final population) 1: $ P\leftarrow $ Initialize the population ($ N $); 2: $ R\leftarrow $ Implement the self-organizing decomposition ($ P $, $ N $); 3: while $ FEs\leq maxFEs $ do 4: $ M\leftarrow $ Random mating selection ($ P $); 5: $ O\leftarrow $ Crossover and mutation ($ M $); 6: $ U\leftarrow \; P \cup O $; 7: $ P\leftarrow $ Implement the environmental selection ($ U $, $ R $, $ N $); 8: $ R\leftarrow $ Implement the the self-organizing decomposition ($ P $, $ N $); 9: end while 10: Return: $ P $. |
The striking feature of decomposition based MaOEAs is to use the reference vectors to guide the evolution. However, most decomposition based MaOEAs more or less depend on the predefined reference vectors. This makes most of them are sensitive to the PF shapes, as shown in [35]. Therefore, a self-organizing decomposition is developed to overcome the drawback above in this work. The process of the developed self-organizing decomposition is presented in Algorithm 2 and Figure 3.
First, the population $ P $ with $ N $ individuals is normalized by formula (2.1). Moreover, the Euclidean distance $ d $ value of any two individuals in $ P $ is calculated, and the distance value of each individual and its closest individual is stored in the distance archive (DA). Then, the minimum distance value $ d_{\mathrm{min}} $ in DA is initialized to 0. Next, these individuals with poor diversity are deleted from $ P $ via the following steps (lines 5–13 of Algorithm 2):
Step 1: According to the DA, find these two individuals with the minimum $ d_{\mathrm{min}} $ value and mark them as individuals A and B, where $ d_{\mathrm{min}} = \mathrm{min} (\mathrm{DA}) $;
Step 2: According to the DA, delete the individual with poor diversity in individuals A and B;
Step 3: After finishing step 2, update the DA;
Step 4: Repeat the three steps above until the minimum distance value $ d_{\mathrm{min}} $ in DA is greater than the threshold $ \delta $.
| Algorithm 2 Self-organizing decomposition |
|
Require: $ P $ (current population), $ N $ (population size), Parameter $ \delta $
Ensure: $ R $ (generated reference vectors) 1: $ P \leftarrow $ Normalize the objective values of individuals via the formula (2.1) ($ P $); 2: $ d_{\mathrm{min}} \; \leftarrow $ Calculate the minimum distance of each individual ($ P $); 3: DA $ \leftarrow $ Store the $ d_{\mathrm{min}} $ value of each individual ($ P $, $ d_{\mathrm{min}} $); 4: $ d_{\mathrm{min}} $ = 0; 5: while $ d_{\mathrm{min}}\leq \delta $ do 6: (A, B) $ \leftarrow $ Find these two individuals meeting the $ d_{\mathrm{min}} = \mathrm{min} (\mathrm{DA}) $ condition (DA); 7: if $ d_{\mathrm{min}}(\mathrm{A}) \leq d_{\mathrm{min}}(\mathrm{B}) $ then 8: Delete individual ($ \mathrm{A} $); 9: else 10: Delete individual ($ \mathrm{B} $); 11: end if 12: $ \mathrm{DA} \leftarrow $ Update the DA; 13: end while 14: while $ |P| \leq N $ do 15: (C, D) $ \leftarrow $ Find these two individuals meeting the $ d_{\mathrm{min}} = \mathrm{max} (\mathrm{DA}) $ condition (DA); 16: E $ \leftarrow $ Generate a new individual in the midpoint of individuals C and D; 17: $ P \leftarrow $ Add the generated individual E into population $ P $; 18: $ \mathrm{DA} \leftarrow $ Update the DA; 19: end while 20: for each individual $ x $ in $ P $ do 21: $ R = \frac{f(x)}{\|f(x)\|} $; 22: end for 23: Return: $ R $ |
After deleting these individuals with poor diversity, some new individuals with good diversity are generated (lines 14–22 of Algorithm 2). Specifically, according to the current DA, these two individuals with maximum $ d_{\mathrm{min}} $ value are found and marked as individuals C and D. Then, a new individual E is generated in the midpoint between individuals C and D. Next, the new individual E is added into the population $ P $, and the DA is updated. The above three steps are repeated until the number of population $ P $ reaches $ N $. Finally, $ N $ individuals in $ P $ with good diversity are transformed into $ N $ reference vectors.
Existing diversity measures (i.e., crowding degree) used in MaOEAs mostly include the crowding distance [4], $ k $th nearest distance/angle [41], and reference vector [32]. The first two utilize the nearest neighbors to evaluate the crowding degree of individuals. Therefore, the first two emphasize the local diversity. As for the latter, reference vectors guide individuals to converge toward the PF from various directions. As a result, reference vector emphasizes the global diversity.
Based on the above analysis, one can know that existing diversity measures cannot take local and global diversities into account simultaneously. To tackle the problem, we propose a cooperative diversity measure. To be specific, according to the minimum Euclidean distance between individuals and each reference vector, each reference vector is assigned several individuals that are closest to each reference vector. The assignment process is presented in lines 2–17 of Algorithm 3. After the assignment above, each reference vector is assigned at least an individual. These individuals assigned to each reference vector form a cluster ($ C_i, i \in 1, 2, \cdots, N $). Based on the formed cluster, the mathematical description of the designed cooperative diversity measure is as follows:
| $ D(x)=λGD(x)+LD(x) $ | (3.1) |
where $ GD(x) $ is the global diversity measure, $ LD(x) $ refers to the local diversity measure, and $ \lambda $ denotes the adaptive penalty factor. When individual $ x $ has a smaller $ D(x) $ value, this means that individual $ x $ has a better diversity.
| $ GD(x)=d2(x) $ | (3.2) |
| $ LD(x)=miny∈Ci,x≠yd(x,y) $ | (3.3) |
| $ x,y∈Ci $ | (3.4) |
where $ d_2(x) $ is the perpendicular distance between individual $ x $ in $ C_i $ and its closest reference vector, and $ \min_{y \in C_i, x \neq y} d(x, y) $ denotes the minimum distance between individual $ x $ in $ C_i $ and the remaining individuals in $ C_i $.
To achieve the cooperation of $ GD(x) $ and $ LD(x) $, an adaptive penalty factor $ \lambda $ is proposed, whose mathematical description is as follows:
| $ λ=|Ci|2 $ | (3.5) |
where $ |C_i| $ is the number of individuals in $ C_i $.
The reason for such a design is to adaptively adjust the priority of global and local diversities based on the number of individuals in $ C_i $. In such a way, the diversity of individuals can be better measured from global and local angles. To be specific, when the $ C_i $ has more individuals, $ \lambda $ becomes greater. In this case, the diversity of individual $ x $ mainly depends on its local diversity. On the contrary, the diversity of individual $ x $ is determined by its global diversity.
| Algorithm 3 Environmental selection process of SDEA |
|
Require: $ R $ (reference vectors), $ N $ (population size)
Ensure: $ P $ (population for the next generation) 1: $ U \leftarrow $ Normalize the objective values of individuals via the formula (2.1) ($ U $); 2: for each $ x\in U $ do 3: for $ r\in R $ do 4: Compute the $ d_2 $ value of individual $ x $ according to the formula (2.3); 5: end for 6: Assign $ \pi(x) = r: \arg \min _{r \in R} d_2 (x, r) $ 7: end for 8: $ \left\{C_{1}, C_{2}, \ldots, C_{N}\right\} = \{\phi, \phi, \cdots, \phi\} $ 9: for $ i = 1:|R| $ do 10: $ C_{i} = C_{i} \cup\left\{x \mid \pi(x) = r_{i}\right\} $ 11: end for 12: for each $ C_i $ do 13: if isempty $ C_i $ then 14: Assign $ \pi(x) = r: \arg \min _{r \in R} d_2 (x, r) $; 15: $ C_{i} = C_{i} \cup\left\{x \mid \pi(x) = r_{i}\right\} $; 16: end if 17: end for 18: for $ i = 1:N $ do 19: for each $ x \in C_i $ do 20: Calculate the $ F(x) $ value of each individual in $ C_i $ based on the formula (3.7); 21: Ascending sort $ F(x) $ values of these individual in $ C_i $ as $ F_1, F_2, \cdots F_{|C_i|} $; 22: end for 23: end for 24: $ \left\{F_{1}, F_{2}, \cdots, F_{l}\right\} = \{\phi, \phi, \cdots, \phi\}, l = \max \left\{\left|C_{1}\right|, \left|C_{2}\right|, \ldots, \left|C_{N}\right|\right\} $; 25: for $ i = 1:l $ do 26: for $ j = 1:N $ do 27: $ F_{i} = F_{i} \cup C_{j}[i] $; 28: end for 29: end for 30: for $ i = 1:l $ do 31: if $ \left|F_{i}\right| = \min \left\{\left|F_{i}\right|, N-\left|P\right|\right\} $; then 32: $ P = P \cup F_{i} $; 33: else 34: Ascending sort ($ F_{i} $, $ F(x\in F_{i}) $); 35: $ \left.P = P \cup F_{i}[1: N-| P \mid\right] $; 36: end if 37: end for |
Convergence measure plays an important role in developing a new MaOEA. In this work, we adopt the projection distance of each individual on its closest reference vector to evaluate the convergence of individuals, which has been widely used in MaOEAs. For individual $ x $, its convergence is evaluated by:
| $ C(x)=d1(x) $ | (3.6) |
where $ d_1(x) $ is calculated by the formula (2.2).
Based on the above cooperative diversity measure and convergence measure, the new fitness evaluation of individual $ x $ is calculate by:
| $ F(x)=C(x)+D(x) $ | (3.7) |
Note that when individual $ x $ has a smaller fitness evaluation value, this means that individual $ x $ has the better overall performance (i.e., convergence and diversity).
The environmental selection in a MaOEA aims at picking out $ N $ promising individuals to enter the next generation. The environmental selection process of SDEA is presented in Algorithm 3. To be specific, the objective values of each individual in $ U $ are first normalized by the formula (2.1) (line 1). Moreover, according to the minimum $ d_2 $ value in formula (2.3), each individual in $ U $ is assigned to its closest reference vector, and then examine whether each reference vector is assigned at least an individual (lines 2–17). When a reference vector is not assigned any individual, the individual that is closest to the reference vector will be assigned to the reference vector (lines 12–17). These individuals that are assigned to each reference vector form a cluster. After the assignment operator above, the fitness evaluation value $ F(x) $ of each individual in $ U $ is calculated for each cluster ($ C_i, i \in 1, 2, \cdots N $). Next, these individuals in each cluster are sorted in the ascending order based on their $ F(x) $ values, and thus form different ranking layers (i.e., $ F_1, F_2, \cdots, F_{|C_i|} $) (lines 18–23). Finally, promising individuals are selected layer by layer from each cluster for the next generation until the size of next generation population reaches $ N $ (lines 24–37). When individuals in the final layers of all clusters make the number of next generation population just exceed the $ N $, these individuals in the final layers with small $ F(x) $ values are picked out for the next generation.
According the framework of SDEA, we analyzed the computational complexity of SDEA in detail. To be specific, the genetic operators require $ O(ND) $, where $ N $ is the population size and $ D $ is the number of decision variables. To implement the self-organizing decomposition, it requires $ O\left(N^2\right) $ costs. To implement the environmental selection, the computational complexity of normalizing the objective values is $ O(mN) $, where $ m $ is the number of objectives; the computational complexity of assigning individuals for each reference vector is $ O\left(m N^2\right) $; the computational complexity of calculating the fitness value of individuals is $ O(mN) $; and the computational complexity of selecting $ N $ individuals for the next generation is $ O\left(m N^2\right) $.Therefore, SDEA executes one, which requires $ O\left(m N^2\right) $ costs.
Since the proposed SDEA belongs the decomposition based MaOEAs, some discussions about SDEA and traditional decomposition MaOEAs are necessary. Note that we select the RVEA adopting the predefined reference vectors and MOEA/D-AWA adopting the adaptive reference vectors as the representative of traditional decomposition MaOEAs.
(1) Similarities among SDEA, RVEA, and MOEA/D-AWA
● All are the decomposition based MaOEAs.
● Both SDEA and RVEA adopt the fitness evaluation that combines the convergence and diversity measures to evaluate the quality of individuals.
● They make individual population approaches to the PF from diverse search directions, thereby balancing convergence and diversity of population.
(2) Differences between SDEA and RVEA
● SDEA transforms individuals with good diversity into reference vectors, so that SDEA is not sensitive to the PF shapes.
● RVEA needs to preset the evenly distributed reference vectors. Therefore, RVEA performs unsatisfactorily in solving the problems with irregular PFs.
● In SDEA, the cooperative diversity measure is adopted to evaluate the crowded degree (i.e., diversity), which is beneficial to better measure the population diversity. Conversely, RVEA utilizes only the angle value between each individual and its closest reference vector to measure the crowded degree.
(3) Differences between SDEA and MOEA/D-AWA
● MOEA/D-AWA also needs to preset the evenly reference vectors. However, it continuously adjusts the reference vectors (i.e., adaptive reference vectors). To be specific, it first evaluates the degree of sparsity of each individual. Then, according to the obtained sparsity degree, it deletes the crowded reference vector, and generates the new reference vectors by utilizing the non-dominated individuals with the greatest sparsity degree. Although such algorithms show certain superiority in solving problems with irregular PFs, their performance in solving problems with regular PFs becomes bad.
● MOEA/D-AWA adopts the Chebyshev aggregation function as the quality evaluation of individuals. On the contrary, SDEA adopts the fitness evaluation that combines the convergence and cooperative diversity measures.
This section conducts a series of experiments to test the ability of SDEA. The experiments mostly include: 1) Comparison of SDEA with six advanced MaOEAs; 2) effectiveness validation of each main component in SDEA; and 3) parameter sensitivity analysis in SDEA. These experiments are all conducted on the matlab platform PlatEMO [45].
In this work, we select WFG1-9 from the WFG benchmark test suite [50], MaF1-10 from the MaF benchmark test suite [31], and IMOP1-8 from the IMOP benchmark test suite to test the performance of SDEA. For WFG1-9, the number of decision variable denoted as $ D $, is set to $ m+9 $, where m denotes the number of objectives. For MaF1-6 and MaF10, $ D $ is set to $ m+9 $. For MaF7, $ D $ is set to $ m+19 $. For MaF8-9, $ D $ is set to 2. As for IMOP1-8, all scenarios have 10 decision variables.
To visually show the performance difference of SIEA and its six competitors, IGD [32] and HV [27]) metrics are utilized for evaluating the algorithm performance, which are capable of evaluating the convergence and diversity of algorithms simultaneously.
$ 1) $ IGD metric
Its mathematical definition is as follows:
| $ IGD(P∗,P,)=∑X∗∈P∗minX∈P‖X∗−X‖|P∗| $ | (4.1) |
where $ P $ refers to solutions obtained by algorithm, and $ P^* $ denotes the 10000 evenly sampling points from the true PF. A smaller the IGD value signifies a better comprehensive performance.
$ 2) $ HV metric
Its mathematical definition is as follows:
| $ HV(P)=VOL(Uz∈P[f1(x),zr1]×⋯[fm(x),zrm]) $ | (4.2) |
where $ VOL(\cdot) $ is the Lebesgue measure,
| $ Z^r = \left(z_1^r, \cdots, z_m^r\right)^T $ |
is set to $ (1.1, 1.1, \cdots, 1.1) $. A greater HV value denotes a better performance for approximating the true PF.
Six state-of-the-art MaOEAs are selected to compare with SDEA, where these six MaOEAs include RVEA [6], MOEA/D-AWA [9], RPS-NSGA-II [17], MOEA/D-UR [10], MaOEA-PDS [48], and HEA [49]. Their core working principles are as follows:
$ 1) $ RVEA: Belongs to the decomposition-based MaOEAs, which divides the objective space into some sub-spaces, and then the angle penalty distance (APD) is used in each sub-space to select individuals.
$ 2) $ MOEA/D-AWA: First evaluates the sparsity degree of each individual. Then, according to the obtained sparsity degree, it deletes the crowded reference vector, and generates the new reference vectors by utilizing the non-dominated individuals with the greatest sparsity degree.
$ 3) $ RPS-NSGA-II: Utilizes the proposed RPS-dominance to distinguish these individuals that Pareto dominance cannot distinguish. In the RPS-dominance, the sum of objectives and the number of individuals in each reference points act as the convergence and diversity measures, respectively.
$ 4) $ MOEA/D-UR: First uses an improvement metric to detect the performance of reference vectors. When the the performance of reference vectors have the convergence trend, the adaptive process of reference vectors are implemented
$ 5) $ MaOEA-PDS: Partitions the objective space into different sub-space, and then deletes these individuals with poor convergence from each sub-space. The deletion process in different sub-spaces is repeated until the the number of individuals entering the next generation reaches $ N $.
$ 6) $ HEA: Uses the cooperation of the proposed hyper-dominance degree and the improved reference vectors-based diversity preservation mechanism to select parents for the next generation. The hyper-dominance degree emphasizes the convergence, which can dynamically adjust the selection pressure.
$ 1) $ Population size and termination criterion: The setting of population size $ N $ for each algorithm follows the Table 1, as suggested in [11]. Each algorithm is independently run 20 times on each test instance. The termination criterion of each run is the $ maxFEs $, which is set to 100000. Such setting of $ maxFEs $ has been widely used [51,52].
| $ m $ | Division($ H_{1} $, $ H_{2} $) | $ N $ |
| 5 | (6, 0) | 210 |
| 10 | (3, 2) | 275 |
| 15 | (2, 1) | 135 |
DownLoad:
CSV
$ 2) $ Crossover and mutation: Seven MaOEAs all use the SBX and PM to reproduce the offspring, where the parameters in SBX and PM are set based on [6].
$ 3) $ Parameters of compared algorithms: To ensure a fair comparison, the parameters of each comparison algorithm remain consistent with their original literature.
$ 4) $ Significant test: The Wilcoxon rank-sum test, with a significant level of 0.05 is used to compare the significance of differences between SDEA and each comparison algorithms, where "+", "-", and "$ \approx $" mean that the comparison algorithm is better than, worse than, and similar to the proposed SDEA, respectively.
Here, we present and analyze the experimental results achieved by seven MaOEAs on three benchmark test suites. Tables 2–9 give the IGD and HV results obtained by SDEA and its six competitors. Note that the optimal result on each test case is marked as the gray. From these tables, we know that SDEA has a significant advantage over its competitors on each benchmark test suite. Below, we analyze the comparison results in detail.
| Problem | $ m $ | $ D $ | RVEA | MOEA/D-AWA | RPS-NSGA-II | MOEA/D-UR | MaOEA-PDS | HEA | SDEA |
| WFG1 | 5 | 14 | 5.4635e-1 (7.46e-2) $ - $ | 3.9587e-1 (1.26e-2) $ - $ | 7.2685e-1 (4.79e-2) $ - $ | 7.0831e-1 (4.08e-2) $ - $ | 3.8544e-1 (1.66e-2) $ \approx $ | 4.4033e-1 (1.03e-2) $ - $ | 3.7929e-1 (1.06e-2) |
| 10 | 19 | 1.4450e+0 (9.21e-2) $ - $ | 1.0525e+0 (2.06e-2) $ - $ | 1.5323e+0 (4.55e-2) $ - $ | 1.8904e+0 (3.14e-2) $ - $ | 1.5440e+0 (1.27e-1) $ - $ | 1.0905e+0 (5.08e-2) $ - $ | 9.4664e-1 (1.55e-2) | |
| 15 | 24 | 1.9111e+0 (1.67e-1) $ \approx $ | 1.9978e+0 (6.68e-2) $ - $ | 2.1808e+0 (3.71e-2) $ - $ | 2.4509e+0 (4.71e-2) $ - $ | 2.4542e+0 (5.92e-1) $ - $ | 1.7976e+0 (3.28e-2) $ - $ | 1.6338e+0 (4.70e-2) | |
| WFG2 | 5 | 14 | 3.8976e-1 (2.07e-3) $ \approx $ | 4.5517e-1 (1.33e-2) $ - $ | 6.0492e-1 (4.93e-2) $ - $ | 6.6631e-1 (7.09e-2) $ - $ | 4.0538e-1 (1.17e-2) $ - $ | 4.5076e-1 (7.40e-3) $ - $ | 3.8297e-1 (1.04e-2) |
| 10 | 19 | 1.0615e+0 (8.53e-2) $ \approx $ | 1.1886e+0 (3.23e-2) $ - $ | 1.7249e+0 (6.11e-2) $ - $ | 1.7981e+0 (5.37e-2) $ - $ | 1.0226e+0 (2.03e-2) $ + $ | 1.0919e+0 (2.49e-2) $ \approx $ | 1.1072e+0 (3.42e-2) | |
| 15 | 24 | 2.0018e+0 (9.86e-2) $ - $ | 2.4181e+0 (6.59e-2) $ - $ | 2.4740e+0 (4.28e-2) $ - $ | 1.7691e+0 (4.42e-2) $ \approx $ | 2.0263e+0 (2.09e-1) $ - $ | 1.8323e+0 (7.73e-2) $ - $ | 1.6338e+0 (4.70e-2) | |
| WFG3 | 5 | 14 | 4.4053e-1 (4.97e-2) $ \approx $ | 3.4144e-1 (3.25e-2) $ + $ | 1.8024e+0 (2.61e-1) $ - $ | 1.2584e+0 (1.48e-1) $ - $ | 5.0335e-1 (1.06e-1) $ \approx $ | 3.0801e-1 (5.42e-2) $ + $ | 4.5416e-1 (2.63e-2) |
| 10 | 19 | 2.0550e+0 (1.94e-1) $ + $ | 1.6373e+0 (2.49e-1) $ + $ | 6.6493e+0 (7.25e-1) $ - $ | 5.1583e+0 (2.80e-1) $ - $ | 1.8029e+0 (1.88e-1) $ + $ | 1.3715e+0 (4.29e-1) $ + $ | 3.4318e+0 (9.29e-1) | |
| 15 | 24 | 3.5356e+0 (1.39e+0) $ + $ | 3.5804e+0 (1.58e+0) $ + $ | 1.3448e+1 (5.47e-1) $ - $ | 9.3436e+0 (6.32e-1) $ \approx $ | 4.8727e+0 (3.73e-1) $ + $ | 3.1511e+0 (1.07e+0) $ + $ | 8.5894e+0 (2.28e+0) | |
| WFG4 | 5 | 14 | 9.6578e-1 (1.33e-3) $ \approx $ | 1.0927e+0 (1.07e-2) $ - $ | 1.3665e+0 (6.93e-2) $ - $ | 1.4823e+0 (1.09e-1) $ - $ | 9.6670e-1 (1.50e-3) $ - $ | 1.1283e+0 (2.19e-2) $ - $ | 9.6503e-1 (1.29e-3) |
| 10 | 19 | 4.5176e+0 (1.11e-2) $ - $ | 4.5238e+0 (8.79e-2) $ - $ | 5.9297e+0 (1.33e-1) $ - $ | 5.9074e+0 (1.31e-1) $ - $ | 4.5598e+0 (2.09e-2) $ - $ | 4.8097e+0 (2.22e-2) $ - $ | 4.3489e+0 (5.08e-2) | |
| 15 | 24 | 9.4038e+0 (3.78e-2) $ - $ | 9.6090e+0 (2.38e-1) $ - $ | 1.1576e+1 (3.12e-1) $ - $ | 1.1622e+1 (3.11e-1) $ - $ | 9.4306e+0 (2.12e-2) $ - $ | 1.0005e+1 (1.06e-1) $ - $ | 9.3085e+0 (1.21e-1) | |
| WFG5 | 5 | 14 | 9.5715e-1 (6.34e-4) $ \approx $ | 1.0975e+0 (1.38e-2) $ - $ | 1.2890e+0 (6.34e-2) $ - $ | 1.3457e+0 (7.76e-2) $ - $ | 9.5725e-1 (1.32e-3) $ \approx $ | 1.1162e+0 (2.48e-2) $ - $ | 9.5719e-1 (1.01e-3) |
| 10 | 19 | 4.5065e+0 (1.33e-2) $ - $ | 4.9460e+0 (6.56e-2) $ - $ | 5.7480e+0 (1.62e-1) $ - $ | 5.7359e+0 (2.36e-1) $ - $ | 4.5447e+0 (9.44e-3) $ - $ | 4.7827e+0 (3.58e-2) $ - $ | 4.3479e+0 (2.75e-2) | |
| 15 | 24 | 9.2848e+0 (6.48e-3) $ - $ | 1.0669e+1 (1.74e-1) $ - $ | 1.1357e+1 (4.98e-1) $ - $ | 1.1187e+1 (2.09e-1) $ - $ | 9.3812e+0 (1.09e-2) $ - $ | 9.9388e+0 (1.70e-1) $ - $ | 9.2056e+0 (6.68e-2) | |
| WFG6 | 5 | 14 | 9.6252e-1 (1.14e-3) $ - $ | 1.0948e+0 (1.85e-2) $ - $ | 1.5668e+0 (1.11e-1) $ - $ | 1.7897e+0 (1.18e-1) $ - $ | 9.5952e-1 (9.96e-4) $ + $ | 1.1153e+0 (1.30e-2) $ - $ | 9.6094e-1 (8.47e-4) |
| 10 | 19 | 4.5765e+0 (1.85e-2) $ - $ | 4.5781e+0 (1.62e-1) $ - $ | 6.8764e+0 (2.00e-1) $ - $ | 6.6838e+0 (1.58e-1) $ - $ | 4.5748e+0 (1.36e-2) $ - $ | 4.8668e+0 (3.52e-2) $ - $ | 4.2328e+0 (4.95e-2) | |
| 15 | 24 | 9.4004e+0 (1.63e-1) $ + $ | 1.0520e+1 (5.68e-1) $ - $ | 1.2902e+1 (4.00e-1) $ - $ | 1.2160e+1 (2.80e-1) $ - $ | 9.3998e+0 (1.63e-2) $ + $ | 1.0004e+1 (1.66e-1) $ \approx $ | 9.8458e+0 (3.07e-1) | |
| WFG7 | 5 | 14 | 9.6656e-1 (6.58e-4) $ \approx $ | 1.0954e+0 (1.10e-2) $ - $ | 1.5246e+0 (1.34e-1) $ - $ | 1.8936e+0 (1.74e-1) $ - $ | 9.6620e-1 (5.91e-4) $ + $ | 1.1273e+0 (1.38e-2) $ - $ | 9.6704e-1 (7.35e-4) |
| 10 | 19 | 4.5568e+0 (5.79e-2) $ - $ | 4.5835e+0 (5.31e-2) $ - $ | 5.9259e+0 (1.69e-1) $ - $ | 6.1533e+0 (2.17e-1) $ - $ | 4.5483e+0 (2.31e-2) $ - $ | 4.8166e+0 (1.48e-2) $ - $ | 4.3796e+0 (4.55e-2) | |
| 15 | 24 | 9.3673e+0 (7.65e-2) $ \approx $ | 9.6379e+0 (1.09e-1) $ - $ | 1.1123e+1 (4.45e-1) $ - $ | 1.1392e+1 (5.91e-1) $ - $ | 9.3975e+0 (2.34e-2) $ - $ | 1.0018e+1 (1.48e-1) $ - $ | 9.3218e+0 (8.84e-2) | |
| WFG8 | 5 | 14 | 9.9498e-1 (7.53e-3) $ \approx $ | 1.1109e+0 (9.54e-3) $ - $ | 1.5536e+0 (1.08e-1) $ - $ | 1.5334e+0 (1.01e-1) $ - $ | 9.9122e-1 (1.92e-3) $ \approx $ | 1.1370e+0 (1.90e-2) $ - $ | 9.8996e-1 (1.47e-3) |
| 10 | 19 | 4.6331e+0 (2.82e-1) $ - $ | 4.4239e+0 (6.29e-2) $ - $ | 6.3655e+0 (3.95e-1) $ - $ | 6.4510e+0 (3.48e-1) $ - $ | 4.6307e+0 (3.14e-2) $ - $ | 4.8988e+0 (2.70e-2) $ - $ | 4.2874e+0 (8.91e-2) | |
| 15 | 24 | 1.0290e+1 (4.26e-1) $ - $ | 1.1028e+1 (6.27e-1) $ - $ | 1.1822e+1 (4.75e-1) $ - $ | 9.3252e+0 (3.36e-2) $ \approx $ | 9.7060e+0 (7.39e-2) $ - $ | 9.3963e+0 (3.55e-1) $ - $ | 9.1878e+0 (1.93e-1) | |
| WFG9 | 5 | 14 | 9.3407e-1 (2.99e-3) $ + $ | 1.0364e+0 (2.11e-2) $ - $ | 1.2761e+0 (3.72e-2) $ - $ | 1.4277e+0 (9.88e-2) $ - $ | 9.3237e-1 (3.36e-3) $ + $ | 1.0890e+0 (1.53e-2) $ - $ | 9.4329e-1 (1.94e-3) |
| 10 | 19 | 4.3721e+0 (4.86e-2) $ - $ | 4.4745e+0 (1.14e-1) $ - $ | 5.4624e+0 (1.45e-1) $ - $ | 5.5268e+0 (2.36e-1) $ - $ | 4.4653e+0 (3.26e-2) $ - $ | 4.7361e+0 (2.93e-2) $ - $ | 4.2489e+0 (5.03e-2) | |
| 15 | 24 | 8.7261e+0 (1.09e-1) $ + $ | 9.6567e+0 (2.62e-1) $ - $ | 1.0325e+1 (2.92e-1) $ - $ | 1.0114e+1 (4.07e-1) $ - $ | 8.8524e+0 (1.04e-1) $ + $ | 9.5723e+0 (9.41e-2) $ - $ | 9.1838e+0 (1.46e-1) | |
| $ +/-/\approx $ | 5/13/9 | 3/24/0 | 0/27/0 | 0/24/3 | 6/18/3 | 3/22/2 | |||
DownLoad:
CSV
| Problem | $ m $ | $ D $ | RVEA | MOEA/D-AWA | RPS-NSGA-II | MOEA/D-UR | MaOEA-PDS | HEA | SDEA |
| WFG1 | 5 | 14 | 8.3053e-1 (2.90e-2) $ - $ | 9.7421e-1 (2.46e-2) $ - $ | 8.5218e-1 (4.63e-2) $ - $ | 9.7917e-1 (2.42e-2) $ - $ | 9.8167e-1 (1.93e-2) $ - $ | 9.8866e-1 (1.93e-2) $ \approx $ | 9.9801e-1 (3.79e-4) |
| 10 | 19 | 7.0216e-1 (7.03e-2) $ - $ | 9.9014e-1 (1.37e-2) $ - $ | 9.0923e-1 (5.62e-2) $ - $ | 9.9337e-1 (1.75e-3) $ - $ | 9.9065e-1 (1.89e-3) $ - $ | 9.9715e-1 (1.56e-3) $ - $ | 9.9963e-1 (3.80e-4) | |
| 15 | 24 | 9.9970e-1 (2.81e-4) $ - $ | 9.9908e-1 (5.68e-4) $ - $ | 9.9719e-1 (7.74e-4) $ - $ | 9.9822e-1 (9.05e-4) $ - $ | 9.1853e-1 (9.60e-2) $ - $ | 9.9713e-1 (6.41e-4) $ - $ | 9.9998e-1 (2.41e-5) | |
| WFG2 | 5 | 14 | 9.9206e-1 (1.59e-3) $ - $ | 9.7930e-1 (7.83e-3) $ - $ | 9.5050e-1 (1.24e-2) $ - $ | 9.7737e-1 (6.58e-3) $ - $ | 9.8845e-1 (2.47e-3) $ - $ | 9.9039e-1 (1.66e-3) $ - $ | 9.9366e-1 (7.60e-4) |
| 10 | 19 | 9.9048e-1 (4.84e-3) $ \approx $ | 9.9382e-1 (1.38e-3) $ + $ | 9.7457e-1 (6.38e-3) $ - $ | 9.8970e-1 (1.93e-3) $ - $ | 9.8425e-1 (3.78e-3) $ - $ | 9.8200e-1 (3.18e-3) $ - $ | 9.9221e-1 (9.41e-4) | |
| 15 | 24 | 9.9283e-1 (2.47e-3) $ - $ | 9.9575e-1 (9.90e-4) $ \approx $ | 9.7657e-1 (4.57e-3) $ - $ | 9.9191e-1 (2.00e-3) $ - $ | 9.8856e-1 (4.47e-3) $ - $ | 9.7026e-1 (1.01e-2) $ - $ | 9.9681e-1 (1.72e-3) | |
| WFG3 | 5 | 14 | 1.6003e-1 (1.76e-2) $ \approx $ | 2.1443e-1 (2.02e-2) $ + $ | 0.0000e+0 (0.00e+0) $ - $ | 6.2622e-2 (1.19e-2) $ - $ | 2.0853e-1 (1.16e-2) $ + $ | 1.5886e-1 (1.06e-2) $ \approx $ | 1.4825e-1 (3.29e-2) |
| 10 | 19 | 0.0000e+0 (0.00e+0) $ \approx $ | 3.7907e-3 (5.95e-3) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 3.5069e-2 (2.21e-2) $ + $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) | |
| 15 | 24 | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) | |
| WFG4 | 5 | 14 | 7.9689e-1 (1.75e-3) $ - $ | 7.8109e-1 (1.54e-3) $ - $ | 6.0579e-1 (1.99e-2) $ - $ | 6.8128e-1 (1.21e-2) $ - $ | 7.8801e-1 (3.28e-3) $ - $ | 7.9944e-1 (1.74e-3) $ - $ | 8.0615e-1 (1.22e-3) |
| 10 | 19 | 9.2669e-1 (4.49e-3) $ - $ | 9.4142e-1 (2.19e-3) $ - $ | 6.1073e-1 (2.34e-2) $ - $ | 7.9287e-1 (6.16e-3) $ - $ | 9.2540e-1 (3.47e-3) $ - $ | 9.2109e-1 (5.45e-3) $ - $ | 9.4963e-1 (2.90e-3) | |
| 15 | 24 | 9.8122e-1 (1.93e-3) $ \approx $ | 9.4296e-1 (3.86e-3) $ - $ | 4.7679e-1 (3.71e-2) $ - $ | 8.3205e-1 (1.21e-2) $ - $ | 9.7710e-1 (1.66e-3) $ - $ | 9.7070e-1 (4.84e-3) $ - $ | 9.8303e-1 (1.78e-3) | |
| WFG5 | 5 | 14 | 7.5959e-1 (5.56e-4) $ - $ | 7.3861e-1 (1.22e-3) $ - $ | 6.0160e-1 (1.92e-2) $ - $ | 6.5696e-1 (8.53e-3) $ - $ | 7.4441e-1 (3.15e-3) $ - $ | 7.5920e-1 (8.25e-4) $ - $ | 7.6022e-1 (4.13e-4) |
| 10 | 19 | 8.9239e-1 (1.01e-3) $ - $ | 8.8790e-1 (1.36e-3) $ - $ | 6.2494e-1 (2.15e-2) $ - $ | 7.5820e-1 (1.48e-2) $ - $ | 8.8046e-1 (3.92e-3) $ - $ | 8.8853e-1 (1.92e-3) $ - $ | 8.9832e-1 (6.00e-4) | |
| 15 | 24 | 9.1403e-1 (3.10e-4) $ - $ | 8.6622e-1 (5.15e-3) $ - $ | 5.3440e-1 (5.33e-2) $ - $ | 7.8067e-1 (1.10e-2) $ - $ | 9.0666e-1 (2.57e-3) $ - $ | 9.1592e-1 (2.40e-4) $ + $ | 9.1455e-1 (4.77e-4) | |
| WFG6 | 5 | 14 | 7.2850e-1 (8.39e-3) $ - $ | 7.2679e-1 (1.51e-2) $ - $ | 5.2228e-1 (1.93e-2) $ - $ | 5.7880e-1 (1.52e-2) $ - $ | 7.2883e-1 (1.18e-2) $ - $ | 7.3955e-1 (1.05e-2) $ \approx $ | 7.4043e-1 (1.41e-2) |
| 10 | 19 | 8.6206e-1 (1.81e-2) $ - $ | 8.5783e-1 (1.57e-2) $ - $ | 4.6979e-1 (2.58e-2) $ - $ | 6.4730e-1 (3.00e-2) $ - $ | 8.5984e-1 (1.60e-2) $ - $ | 8.4746e-1 (9.01e-3) $ - $ | 8.6998e-1 (1.36e-2) | |
| 15 | 24 | 8.8396e-1 (3.18e-2) $ - $ | 8.2787e-1 (3.14e-2) $ - $ | 3.0173e-1 (2.60e-2) $ - $ | 6.6774e-1 (3.40e-2) $ - $ | 8.8606e-1 (3.44e-2) $ \approx $ | 6.9644e-1 (9.03e-2) $ - $ | 8.8649e-1 (1.31e-2) | |
| WFG7 | 5 | 14 | 8.0157e-1 (1.42e-3) $ - $ | 7.9170e-1 (1.66e-3) $ - $ | 6.0729e-1 (3.24e-2) $ - $ | 6.3656e-1 (1.59e-2) $ - $ | 7.9105e-1 (2.35e-3) $ - $ | 8.0340e-1 (7.00e-4) $ - $ | 8.0758e-1 (2.98e-4) |
| 10 | 19 | 9.4331e-1 (3.60e-3) $ - $ | 9.5594e-1 (1.10e-3) $ - $ | 5.6192e-1 (3.11e-2) $ - $ | 7.8347e-1 (1.37e-2) $ - $ | 9.4214e-1 (1.66e-3) $ - $ | 9.3428e-1 (2.94e-3) $ - $ | 9.5879e-1 (1.82e-3) | |
| 15 | 24 | 9.7887e-1 (4.38e-3) $ - $ | 9.5899e-1 (3.22e-3) $ - $ | 4.4289e-1 (4.06e-2) $ - $ | 8.2889e-1 (2.07e-2) $ - $ | 9.7911e-1 (1.70e-3) $ - $ | 8.9832e-1 (1.20e-1) $ - $ | 9.8483e-1 (1.88e-3) | |
| WFG8 | 5 | 14 | 6.8616e-1 (3.90e-3) $ - $ | 6.5493e-1 (4.43e-3) $ - $ | 4.2548e-1 (2.67e-2) $ - $ | 5.4135e-1 (2.03e-2) $ - $ | 6.8318e-1 (3.65e-3) $ - $ | 6.9016e-1 (1.71e-3) $ - $ | 6.9502e-1 (8.48e-4) |
| 10 | 19 | 8.4110e-1 (8.57e-3) $ - $ | 8.5124e-1 (3.00e-2) $ \approx $ | 2.9935e-1 (6.35e-2) $ - $ | 5.1359e-1 (1.07e-1) $ - $ | 8.3601e-1 (1.18e-2) $ - $ | 7.1634e-1 (7.48e-2) $ - $ | 8.6059e-1 (5.74e-3) | |
| 15 | 24 | 9.2249e-1 (1.86e-2) $ \approx $ | 8.3888e-1 (2.18e-2) $ - $ | 3.1650e-1 (1.76e-1) $ - $ | 4.7231e-1 (1.04e-1) $ - $ | 9.0555e-1 (3.02e-3) $ - $ | 6.3933e-1 (7.66e-2) $ - $ | 9.1724e-1 (5.33e-3) | |
| WFG9 | 5 | 14 | 7.4953e-1 (5.75e-3) $ - $ | 7.5078e-1 (3.47e-3) $ - $ | 6.1318e-1 (9.20e-3) $ - $ | 6.4374e-1 (9.46e-3) $ - $ | 7.4210e-1 (3.51e-3) $ - $ | 7.6033e-1 (2.41e-3) $ \approx $ | 7.6045e-1 (3.47e-3) |
| 10 | 19 | 8.4481e-1 (5.63e-2) $ \approx $ | 8.8742e-1 (5.34e-3) $ + $ | 6.7655e-1 (2.62e-2) $ - $ | 7.2650e-1 (5.38e-2) $ - $ | 8.5557e-1 (1.25e-2) $ - $ | 8.5015e-1 (2.13e-2) $ - $ | 8.7926e-1 (6.82e-3) | |
| 15 | 24 | 8.7216e-1 (7.24e-2) $ + $ | 8.6629e-1 (1.39e-2) $ \approx $ | 5.9362e-1 (3.55e-2) $ - $ | 6.9505e-1 (6.75e-2) $ - $ | 8.9247e-1 (7.23e-3) $ + $ | 8.0237e-1 (6.23e-2) $ - $ | 8.6638e-1 (1.92e-2) | |
| $ +/-/\approx $ | 1/19/7 | 3/19/5 | 0/25/2 | 0/25/2 | 3/22/2 | 1/20/6 | |||
DownLoad:
CSV
| SDEA vs | IGD | HV | |||||
| $ R^{+} $ | $ R^{-} $ | $ p $-value | $ R^{+} $ | $ R^{-} $ | $ p $-value | ||
| RVEA | 313 | 65.0 | 0.00278 | 327.5 | 50.5 | 0.000381 | |
| MOEA/D-AWA | 308.0 | 70.0 | 0.00259 | 330.5 | 47.5 | 0.000512 | |
| RPS-NSGA-II | 311.0 | 67.0 | 0.002105 | 376 | 2 | 0.000058 | |
| MORA/D-UR | 378.0 | 0.0 | 0.000005 | 376 | 2 | 0.000058 | |
| MaOEA-PDS | 378.0 | 0.0 | 0.000005 | 301.5 | 76.5 | 0.004471 | |
| HEA | 320.0 | 58.0 | 0.001021 | 351.5 | 26.5 | 0.000074 | |
DownLoad:
CSV
| Problem | $ m $ | $ D $ | RVEA | MOEA/D-AWA | RPS-NSGA-II | MOEA/D-UR | MaOEA-PDS | HEA | SDEA |
| MaF1 | 5 | 14 | 1.8093e-1 (9.11e-3) $ - $ | 2.7675e-1 (1.65e-2) $ - $ | 1.9100e-1 (8.70e-3) $ - $ | 1.0217e-1 (7.92e-4) $ + $ | 3.0799e-1 (1.99e-2) $ - $ | 1.5632e-1 (2.59e-3) $ - $ | 1.0332e-1 (5.15e-4) |
| 10 | 19 | 2.8503e-1 (1.00e-2) $ - $ | 5.7593e-1 (5.61e-2) $ - $ | 2.8201e-1 (9.95e-3) $ - $ | 2.0642e-1 (8.54e-4) $ + $ | 3.4548e-1 (1.93e-2) $ - $ | 3.3494e-1 (2.12e-2) $ - $ | 2.1234e-1 (1.28e-3) | |
| 15 | 24 | 3.3595e-1 (4.33e-3) $ - $ | 7.5949e-1 (1.01e-1) $ - $ | 3.8999e-1 (1.11e-1) $ - $ | 3.0240e-1 (2.34e-2) $ \approx $ | 4.2697e-1 (1.92e-2) $ - $ | 4.0789e-1 (8.02e-3) $ - $ | 2.9088e-1 (3.59e-3) | |
| MaF2 | 5 | 14 | 1.1175e-1 (2.75e-3) $ - $ | 1.1591e-1 (1.02e-3) $ - $ | 1.4819e-1 (3.49e-3) $ - $ | 8.0600e-2 (1.10e-3) $ + $ | 1.2648e-1 (2.17e-3) $ - $ | 1.1666e-1 (7.76e-4) $ - $ | 9.3169e-2 (1.42e-3) |
| 10 | 19 | 2.0786e-1 (1.27e-2) $ - $ | 2.5117e-1 (1.81e-2) $ - $ | 2.4176e-1 (3.76e-3) $ - $ | 1.9495e-1 (7.87e-3) $ - $ | 1.8387e-1 (1.94e-3) $ - $ | 1.7124e-1 (3.52e-3) $ - $ | 1.6260e-1 (2.20e-3) | |
| 15 | 24 | 2.5667e-1 (2.82e-2) $ - $ | 7.0357e-1 (1.09e-1) $ - $ | 2.1510e-1 (3.51e-3) $ \approx $ | 3.0015e-1 (2.46e-2) $ - $ | 2.1522e-1 (3.18e-3) $ \approx $ | 2.1796e-1 (7.13e-3) $ - $ | 1.9880e-1 (2.52e-3) | |
| MaF3 | 5 | 14 | 7.7658e-2 (5.48e-3) $ + $ | 2.0514e-1 (1.28e-1) $ - $ | 1.4994e-1 (2.37e-2) $ - $ | 2.2080e-1 (1.66e-1) $ - $ | 1.0687e-1 (2.28e-2) $ \approx $ | 1.2120e-1 (3.98e-2) $ - $ | 9.4125e-2 (5.81e-3) |
| 10 | 19 | 1.8372e+2 (2.28e+2) $ - $ | 8.9908e-1 (5.78e-1) $ - $ | 3.9731e-1 (1.94e-1) $ - $ | 1.5877e+1 (9.42e+0) $ - $ | 1.2335e-1 (8.51e-3) $ \approx $ | 2.8923e-1 (6.64e-2) $ - $ | 2.0168e-1 (2.46e-1) | |
| 15 | 24 | 4.5781e+1 (1.08e+2) $ - $ | 1.0100e-1 (4.80e-3) $ + $ | 2.3712e-1 (2.12e-1) $ - $ | 1.5856e-1 (7.32e-2) $ - $ | 1.8609e-1 (2.83e-2) $ - $ | 7.1590e+5 (9.20e+5) $ - $ | 1.0980e-1 (4.30e-3) | |
| MaF4 | 5 | 14 | 2.8205e+0 (7.22e-1) $ - $ | 3.5731e+0 (8.15e-1) $ - $ | 3.1007e+0 (4.89e-1) $ - $ | 3.1097e+0 (5.18e-1) $ - $ | 2.8769e+0 (9.80e-2) $ - $ | 2.8891e+0 (2.62e-1) $ - $ | 2.0313e+0 (7.54e-2) |
| 10 | 19 | 9.7435e+1 (5.35e+0) $ - $ | 1.8558e+2 (2.52e+1) $ - $ | 1.5111e+2 (2.27e+1) $ - $ | 1.2357e+2 (1.86e+1) $ - $ | 8.6259e+1 (7.06e+0) $ - $ | 1.1181e+2 (9.77e+0) $ - $ | 5.3091e+1 (2.05e+0) | |
| 15 | 24 | 4.3936e+3 (2.26e+2) $ - $ | 7.8711e+3 (2.62e+3) $ - $ | 1.5273e+4 (8.13e+3) $ - $ | 8.8258e+3 (2.68e+3) $ - $ | 4.9481e+3 (4.93e+2) $ - $ | 3.6663e+3 (3.62e+2) $ - $ | 2.2801e+3 (1.82e+2) | |
| MaF5 | 5 | 14 | 1.9704e+0 (2.65e-3) $ - $ | 1.9719e+0 (2.23e-3) $ - $ | 1.9269e+0 (2.30e-2) $ - $ | 3.7179e+0 (4.57e-1) $ - $ | 2.2132e+0 (4.51e-2) $ - $ | 1.9609e+0 (9.42e-3) $ - $ | 1.8264e+0 (2.57e-2) |
| 10 | 19 | 7.7358e+1 (6.82e-1) $ - $ | 9.4917e+1 (8.78e+0) $ - $ | 5.6693e+1 (4.40e+0) $ \approx $ | 2.5553e+2 (2.13e+1) $ - $ | 8.1962e+1 (1.38e+0) $ - $ | 6.8867e+1 (2.53e+0) $ - $ | 5.2805e+1 (2.15e+0) | |
| 15 | 24 | 2.9732e+3 (3.91e+2) $ \approx $ | 3.4405e+3 (3.17e+2) $ - $ | 7.1810e+3 (2.06e+2) $ - $ | 3.3465e+3 (6.39e+2) $ \approx $ | 2.7237e+3 (4.67e+2) $ \approx $ | 2.7973e+3 (3.82e+2) $ - $ | 1.9311e+3 (9.37e+1) | |
| MaF6 | 5 | 14 | 1.7091e-2 (4.80e-3) $ - $ | 6.5710e-2 (8.06e-3) $ - $ | 3.7627e-1 (1.75e-1) $ - $ | 1.5278e-1 (2.57e-2) $ - $ | 1.4969e-1 (3.67e-2) $ - $ | 9.3489e-2 (1.20e-2) $ - $ | 2.1829e-3 (1.22e-4) |
| 10 | 19 | 5.1392e-1 (1.37e-1) $ - $ | 9.6871e-2 (2.37e-2) $ + $ | 2.6096e-1 (8.98e-2) $ \approx $ | 7.5239e-1 (2.55e-1) $ - $ | 2.6798e+0 (5.12e+0) $ - $ | 1.6405e-1 (5.96e-2) $ + $ | 3.1961e-1 (6.90e-2) | |
| 15 | 24 | 3.1400e-1 (7.06e-2) $ \approx $ | 1.7163e-1 (6.01e-7) $ + $ | 8.6088e+0 (9.50e+0) $ - $ | 1.5886e-1 (3.06e-2) $ + $ | 1.2430e+1 (3.08e+1) $ - $ | 3.8032e-1 (1.63e-1) $ \approx $ | 3.1966e-1 (4.75e-2) | |
| MaF7 | 5 | 24 | 2.8163e-1 (8.99e-3) $ - $ | 7.8828e-1 (1.03e-1) $ - $ | 3.1363e-1 (7.14e-3) $ - $ | 3.0032e-1 (2.29e-2) $ - $ | 3.7629e-1 (5.08e-2) $ - $ | 3.0510e-1 (3.46e-3) $ - $ | 2.3753e-1 (3.46e-3) |
| 10 | 29 | 1.1300e+0 (1.44e-1) $ \approx $ | 3.0577e+0 (1.63e+0) $ - $ | 2.0908e+0 (5.39e-1) $ - $ | 2.0597e+0 (4.40e-1) $ - $ | 1.3901e+0 (1.29e-2) $ - $ | 1.6103e+0 (9.52e-2) $ - $ | 1.0949e+0 (1.03e-1) | |
| 15 | 34 | 7.6126e+0 (1.26e+0) $ - $ | 4.1200e+0 (1.36e+0) $ - $ | 6.6959e+0 (1.04e+0) $ - $ | 5.1948e+0 (4.46e-1) $ - $ | 6.7687e+0 (1.79e-1) $ - $ | 8.2024e+0 (3.78e-1) $ - $ | 2.4257e+0 (3.02e-1) | |
| MaF8 | 5 | 2 | 1.5616e-1 (1.13e-2) $ - $ | 3.3402e-1 (2.84e-2) $ - $ | 3.0683e-1 (3.86e-2) $ - $ | 8.3643e-2 (1.62e-3) $ \approx $ | 2.2278e-1 (4.07e-2) $ - $ | 1.4864e-1 (6.43e-3) $ - $ | 8.2350e-2 (6.26e-3) |
| 10 | 2 | 3.5278e-1 (1.01e-1) $ - $ | 9.9749e-1 (1.29e-1) $ - $ | 6.9682e-1 (8.72e-2) $ - $ | 1.2152e-1 (3.27e-3) $ \approx $ | 2.9131e-1 (7.31e-2) $ - $ | 2.9153e-1 (1.09e-1) $ - $ | 1.2209e-1 (4.52e-3) | |
| 15 | 2 | 4.1765e-1 (3.93e-2) $ - $ | 1.3049e+0 (2.48e-1) $ - $ | 9.9097e-1 (1.58e-1) $ - $ | 2.1885e-1 (9.41e-3) $ + $ | 5.1606e-1 (6.52e-2) $ - $ | 4.5377e-1 (2.97e-2) $ - $ | 2.7222e-1 (3.96e-2) | |
| MaF9 | 5 | 2 | 3.2052e-1 (6.83e-2) $ - $ | 2.8137e-1 (3.16e-2) $ - $ | 2.3289e-1 (4.14e-2) $ - $ | 1.5154e-1 (4.65e-3) $ - $ | 1.5336e-1 (1.93e-2) $ - $ | 1.8296e-1 (3.19e-2) $ - $ | 7.5546e-2 (2.24e-3) |
| 10 | 2 | 5.3758e-1 (1.52e-1) $ - $ | 7.7406e-1 (2.37e-1) $ - $ | 4.3598e-1 (9.37e-2) $ - $ | 2.2451e-1 (6.70e-2) $ - $ | 4.8400e-1 (1.10e-1) $ - $ | 3.4440e-1 (4.80e-2) $ - $ | 1.2820e-1 (8.32e-3) | |
| 15 | 2 | 3.6908e+0 (5.49e+0) $ - $ | 1.8150e+0 (3.80e-1) $ - $ | 7.8709e-1 (1.69e-1) $ - $ | 3.3958e-1 (8.91e-2) $ - $ | 7.3567e+0 (5.42e+0) $ - $ | 3.7792e+0 (5.43e+0) $ - $ | 1.8485e-1 (4.08e-3) | |
| MaF10 | 5 | 14 | 4.9673e-1 (2.12e-2) $ - $ | 3.7616e-1 (1.17e-2) $ - $ | 4.0004e-1 (1.44e-2) $ - $ | 7.3363e-1 (5.32e-2) $ - $ | 4.4329e-1 (9.09e-3) $ - $ | 3.7721e-1 (9.23e-3) $ - $ | 3.6411e-1 (4.85e-3) |
| 10 | 19 | 1.4464e+0 (1.32e-1) $ - $ | 1.0918e+0 (3.13e-2) $ - $ | 1.0352e+0 (3.16e-2) $ - $ | 1.5495e+0 (4.82e-2) $ - $ | 1.6057e+0 (2.35e-1) $ - $ | 9.5575e-1 (3.25e-2) $ \approx $ | 9.6470e-1 (1.40e-2) | |
| 15 | 24 | 1.7889e+0 (1.15e-1) $ - $ | 1.7714e+0 (3.23e-2) $ - $ | 1.9862e+0 (8.38e-2) $ - $ | 2.2074e+0 (5.11e-2) $ - $ | 2.8502e+0 (7.26e-1) $ - $ | 1.6574e+0 (6.36e-2) $ \approx $ | 1.6561e+0 (2.52e-2) | |
| $ +/-/\approx $ | 1/26/3 | 3/27/0 | 0/27/3 | 5/21/4 | 0/26/4 | 1/26/3 | |||
DownLoad:
CSV
| Problem | $ m $ | $ D $ | RVEA | MOEA/D-AWA | RPS-NSGA-II | MOEA/D-UR | MaOEA-PDS | HEA | SDEA |
| MaF1 | 5 | 14 | 6.5160e-3 (3.49e-4) $ - $ | 3.3381e-3 (2.76e-4) $ - $ | 7.2824e-3 (3.71e-4) $ - $ | 1.2931e-2 (6.69e-5) $ - $ | 2.4605e-3 (3.33e-4) $ - $ | 8.5443e-3 (1.78e-4) $ - $ | 1.3086e-2 (1.73e-4) |
| 10 | 19 | 4.6265e-7 (3.10e-8) $ \approx $ | 9.6417e-9 (3.87e-9) $ - $ | 5.4721e-7 (4.89e-8) $ \approx $ | 2.1973e-7 (3.51e-7) $ - $ | 2.4711e-7 (9.06e-8) $ - $ | 6.7236e-7 (3.24e-7) $ - $ | 8.9684e-7 (9.44e-8) | |
| 15 | 24 | 2.8020e-12 (2.45e-13) $ + $ | 2.3083e-14 (1.51e-14) $ + $ | 3.3293e-12 (1.68e-12) $ + $ | 2.8918e-11 (7.54e-11) $ \approx $ | 4.2472e-13 (2.14e-13) $ + $ | 1.2032e-12 (1.92e-13) $ + $ | 0.0000e+0 (0.00e+0) | |
| MaF2 | 5 | 14 | 1.8617e-1 (1.96e-3) $ - $ | 1.7705e-1 (1.71e-3) $ - $ | 1.8712e-1 (3.16e-3) $ - $ | 1.8890e-1 (2.04e-3) $ - $ | 1.7784e-1 (2.41e-3) $ - $ | 1.7737e-1 (1.39e-3) $ - $ | 2.0025e-1 (1.94e-3) |
| 10 | 19 | 2.1840e-1 (3.84e-3) $ - $ | 1.7054e-1 (2.67e-3) $ - $ | 2.0546e-1 (4.52e-3) $ - $ | 2.0151e-1 (5.21e-3) $ - $ | 1.9617e-1 (2.69e-3) $ - $ | 2.0335e-1 (1.82e-3) $ - $ | 2.2616e-1 (2.74e-3) | |
| 15 | 24 | 1.6970e-1 (9.83e-3) $ - $ | 8.8735e-2 (2.55e-2) $ - $ | 1.6401e-1 (4.65e-3) $ - $ | 1.6881e-1 (3.69e-3) $ - $ | 1.5509e-1 (3.46e-3) $ - $ | 1.5267e-1 (2.55e-3) $ - $ | 2.0144e-1 (3.56e-3) | |
| MaF3 | 5 | 14 | 9.9871e-1 (6.32e-4) $ \approx $ | 9.2138e-1 (1.60e-1) $ - $ | 9.7611e-1 (1.31e-2) $ - $ | 8.3051e-1 (2.34e-1) $ - $ | 9.9660e-1 (1.86e-3) $ - $ | 9.9315e-1 (7.94e-3) $ - $ | 9.9834e-1 (3.46e-4) |
| 10 | 19 | 1.4286e-1 (3.78e-1) $ - $ | 3.2615e-1 (4.53e-1) $ - $ | 7.0304e-1 (2.30e-1) $ - $ | 0.0000e+0 (0.00e+0) $ - $ | 9.9806e-1 (1.36e-3) $ \approx $ | 8.4629e-1 (1.01e-1) $ - $ | 9.0368e-1 (2.54e-1) | |
| 15 | 24 | 4.7947e-1 (4.64e-1) $ - $ | 9.9884e-1 (6.82e-4) $ \approx $ | 8.7677e-1 (2.48e-1) $ - $ | 9.7772e-1 (5.51e-2) $ \approx $ | 9.5232e-1 (3.53e-2) $ - $ | 4.7688e-1 (4.54e-1) $ - $ | 9.9932e-1 (1.02e-3) | |
| MaF4 | 5 | 14 | 7.0518e-2 (1.00e-2) $ - $ | 3.1046e-2 (1.42e-2) $ - $ | 1.0284e-1 (4.64e-3) $ - $ | 7.9981e-2 (1.28e-2) $ - $ | 8.4317e-2 (5.99e-3) $ - $ | 7.2699e-2 (2.38e-3) $ - $ | 1.3276e-1 (1.86e-3) |
| 10 | 19 | 2.1562e-4 (1.72e-5) $ - $ | 6.8904e-8 (1.59e-8) $ - $ | 4.0569e-5 (2.07e-5) $ - $ | 2.9345e-6 (2.36e-6) $ - $ | 2.0362e-4 (2.78e-5) $ - $ | 1.5428e-4 (2.68e-5) $ - $ | 3.6693e-4 (2.87e-5) | |
| 15 | 24 | 2.1729e-7 (3.19e-8) $ + $ | 2.0133e-12 (4.88e-12) $ - $ | 8.7797e-9 (1.37e-8) $ - $ | 1.1794e-11 (1.86e-11) $ - $ | 6.5427e-8 (1.42e-8) $ - $ | 2.7130e-8 (3.86e-8) $ - $ | 1.0839e-7 (3.45e-8) | |
| MaF5 | 5 | 14 | 8.1195e-1 (3.14e-4) $ + $ | 8.1213e-1 (3.04e-4) $ + $ | 7.9326e-1 (1.88e-3) $ - $ | 6.3853e-1 (1.69e-2) $ - $ | 7.9520e-1 (1.80e-3) $ - $ | 8.0824e-1 (1.59e-3) $ \approx $ | 8.0826e-1 (8.30e-4) |
| 10 | 19 | 9.6929e-1 (1.69e-4) $ - $ | 9.4729e-1 (2.30e-3) $ - $ | 9.5845e-1 (2.82e-3) $ - $ | 6.8320e-1 (3.73e-2) $ - $ | 9.5502e-1 (2.18e-3) $ - $ | 9.7005e-1 (1.56e-3) $ - $ | 9.7215e-1 (4.14e-4) | |
| 15 | 24 | 9.8923e-1 (3.61e-3) $ + $ | 9.1661e-1 (2.82e-2) $ - $ | 9.6073e-1 (4.97e-3) $ - $ | 5.6169e-1 (7.22e-2) $ - $ | 9.7546e-1 (6.23e-3) $ - $ | 9.8372e-1 (2.64e-3) $ - $ | 9.8844e-1 (1.64e-3) | |
| MaF6 | 5 | 14 | 1.2347e-1 (1.42e-3) $ - $ | 1.1722e-1 (2.29e-3) $ - $ | 9.3332e-2 (2.73e-2) $ - $ | 9.8929e-3 (5.99e-3) $ - $ | 1.0144e-1 (7.60e-3) $ - $ | 1.1860e-1 (4.22e-3) $ - $ | 1.2996e-1 (2.56e-4) |
| 10 | 19 | 6.5015e-4 (1.72e-3) $ - $ | 9.4780e-2 (6.13e-4) $ + $ | 6.3933e-2 (2.35e-2) $ \approx $ | 0.0000e+0 (0.00e+0) $ - $ | 1.3009e-13 (2.91e-13) $ - $ | 9.6725e-2 (1.66e-3) $ + $ | 3.8061e-2 (3.31e-2) | |
| 15 | 24 | 5.4888e-2 (2.95e-2) $ - $ | 9.1740e-2 (2.74e-4) $ + $ | 0.0000e+0 (0.00e+0) $ - $ | 5.8975e-4 (1.04e-3) $ - $ | 3.8785e-3 (6.00e-3) $ - $ | 8.9102e-2 (2.03e-3) $ + $ | 7.5767e-2 (7.28e-3) | |
| MaF7 | 5 | 24 | 2.5925e-1 (3.34e-3) $ - $ | 1.0563e-1 (1.96e-2) $ - $ | 2.5670e-1 (4.79e-3) $ - $ | 1.7534e-1 (1.92e-2) $ - $ | 2.4798e-1 (4.14e-3) $ - $ | 2.5353e-1 (2.11e-3) $ - $ | 2.7150e-1 (1.58e-3) |
| 10 | 29 | 1.7289e-1 (7.15e-3) $ - $ | 2.0858e-4 (2.64e-4) $ - $ | 1.8403e-1 (3.88e-3) $ \approx $ | 2.9045e-4 (1.31e-4) $ - $ | 1.7472e-1 (3.20e-3) $ - $ | 1.7483e-1 (7.17e-3) $ - $ | 1.8090e-1 (4.24e-3) | |
| 15 | 34 | 1.3662e-1 (2.01e-2) $ \approx $ | 1.5746e-5 (2.06e-5) $ - $ | 1.5022e-1 (7.15e-3) $ \approx $ | 0.0000e+0 (0.00e+0) $ - $ | 1.2905e-1 (1.31e-2) $ - $ | 1.4487e-1 (6.11e-3) $ \approx $ | 1.4421e-1 (2.17e-3) | |
| MaF8 | 5 | 2 | 1.1023e-1 (2.19e-3) $ - $ | 7.5719e-2 (3.12e-3) $ - $ | 8.6853e-2 (4.74e-3) $ - $ | 1.2532e-1 (3.04e-4) $ - $ | 1.0468e-1 (3.16e-3) $ - $ | 1.1021e-1 (1.30e-3) $ - $ | 1.2574e-1 (2.10e-4) |
| 10 | 2 | 9.1903e-3 (3.22e-4) $ - $ | 3.3919e-3 (7.76e-4) $ - $ | 5.0007e-3 (9.34e-4) $ - $ | 1.0950e-2 (5.36e-5) $ - $ | 8.0310e-3 (9.54e-4) $ - $ | 8.3357e-3 (2.99e-4) $ - $ | 1.1111e-2 (3.84e-5) | |
| 15 | 2 | 3.6538e-4 (3.32e-5) $ - $ | 1.3700e-4 (5.54e-5) $ - $ | 2.0735e-4 (3.08e-5) $ - $ | 5.7711e-4 (1.10e-5) $ - $ | 2.1292e-4 (4.32e-5) $ - $ | 1.6319e-4 (6.33e-5) $ - $ | 5.9484e-4 (1.15e-5) | |
| MaF9 | 5 | 2 | 2.2823e-1 (2.05e-2) $ - $ | 2.2671e-1 (1.30e-2) $ - $ | 2.4771e-1 (1.51e-2) $ - $ | 2.7906e-1 (3.33e-3) $ - $ | 2.9014e-1 (8.68e-3) $ - $ | 2.7870e-1 (1.23e-2) $ - $ | 3.1991e-1 (2.13e-3) |
| 10 | 2 | 9.4947e-3 (1.80e-3) $ - $ | 5.8386e-3 (1.68e-3) $ - $ | 1.0978e-2 (1.56e-3) $ - $ | 1.4818e-2 (1.35e-3) $ - $ | 7.6859e-3 (2.37e-3) $ - $ | 1.0519e-2 (1.43e-3) $ - $ | 1.7500e-2 (3.84e-4) | |
| 15 | 2 | 4.4083e-4 (3.29e-4) $ - $ | 1.1750e-4 (6.45e-5) $ - $ | 3.0851e-4 (7.06e-5) $ - $ | 8.6941e-4 (1.13e-4) $ - $ | 8.2382e-5 (2.01e-4) $ - $ | 3.1479e-4 (2.46e-4) $ - $ | 1.1605e-3 (2.45e-5) | |
| MaF10 | 5 | 14 | 8.7077e-1 (2.14e-2) $ - $ | 9.9743e-1 (8.94e-4) $ + $ | 9.8892e-1 (2.84e-3) $ - $ | 8.6314e-1 (2.11e-2) $ - $ | 9.9311e-1 (4.81e-3) $ - $ | 9.9815e-1 (4.06e-4) $ + $ | 9.9680e-1 (6.63e-4) |
| 10 | 19 | 6.7584e-1 (5.61e-2) $ - $ | 9.9732e-1 (9.19e-4) $ - $ | 9.8156e-1 (3.23e-2) $ - $ | 8.9473e-1 (5.33e-2) $ - $ | 9.8488e-1 (5.26e-3) $ - $ | 9.9958e-1 (3.37e-4) $ \approx $ | 9.9962e-1 (2.54e-4) | |
| 15 | 24 | 9.9936e-1 (4.69e-4) $ - $ | 9.9716e-1 (3.23e-4) $ - $ | 9.9931e-1 (6.81e-4) $ - $ | 9.9600e-1 (8.78e-4) $ - $ | 9.6931e-1 (5.50e-2) $ - $ | 9.9999e-1 (7.04e-6) $ + $ | 9.9996e-1 (7.72e-6) | |
| $ +/-/\approx $ | 4/23/3 | 5/24/1 | 1/25/4 | 0/28/2 | 1/28/1 | 5/22/3 | |||
DownLoad:
CSV
| SDEA vs | IGD | HV | |||||
| $ R^{+} $ | $ R^{-} $ | $ p $-value | $ R^{+} $ | $ R^{-} $ | $ p $-value | ||
| RVEA | 461.0 | 4.0 | 0.000002 | 434.0 | 31.0 | 0.000026 | |
| MOEA/D-AWA | 441.0 | 24.0 | 0.000017 | 412.5 | 52.5 | 0.000198 | |
| RPS-NSGA-II | 430.0 | 35.0 | 0.000047 | 425.0 | 40.0 | 0.000069 | |
| MORA/D-UR | 402.0 | 33.0 | 0.000063 | 461.0 | 4.0 | 0.000002 | |
| MaOEA-PDS | 456.5 | 8.5 | 0.000004 | 434.0 | 31.0 | 0.000031 | |
| HEA | 384.0 | 51.0 | 0.000305 | 365.5 | 69.5 | 0.001283 | |
DownLoad:
CSV
| Problem | $ m $ | $ D $ | RVEA | MOEA/D-AWA | RPS-NSGA-II | MOEA/D-UR | MaOEA-PDS | HEA | SDEA |
| IMOP1 | 2 | 10 | 1.3871e-1 (6.02e-3) $ - $ | 1.0191e-1 (5.20e-3) $ \approx $ | 1.3700e-1 (3.71e-2) $ - $ | 3.7121e-1 (4.93e-2) $ - $ | 2.5130e-1 (2.32e-2) $ - $ | 3.5259e-1 (1.02e-2) $ - $ | 9.0222e-2 (1.27e-2) |
| IMOP2 | 2 | 10 | 7.3070e-1 (1.44e-1) $ - $ | 1.2196e-2 (4.62e-3) $ - $ | 5.3323e-1 (3.55e-1) $ - $ | 5.2281e-1 (8.74e-2) $ - $ | 7.8495e-1 (5.80e-5) $ - $ | 1.5455e-1 (2.23e-2) $ - $ | 4.6425e-3 (7.53e-5) |
| IMOP3 | 2 | 10 | 8.1834e-3 (1.42e-3) $ - $ | 6.2844e-3 (9.36e-4) $ \approx $ | 8.5002e-3 (6.34e-4) $ - $ | 4.7468e-1 (1.05e-1) $ - $ | 1.2144e-2 (1.60e-3) $ - $ | 4.2638e-2 (1.51e-2) $ - $ | 5.3531e-3 (5.38e-4) |
| IMOP4 | 3 | 10 | 4.4279e-2 (6.47e-3) $ - $ | 1.6830e-2 (1.43e-3) $ - $ | 6.5909e-2 (1.91e-2) $ - $ | 2.8222e-2 (5.84e-3) $ - $ | 9.7195e-3 (1.11e-3) $ - $ | 3.6545e-2 (5.73e-3) $ - $ | 7.4608e-3 (3.43e-4) |
| IMOP5 | 3 | 10 | 6.6704e-2 (2.13e-3) $ - $ | 4.8526e-2 (1.75e-3) $ - $ | 6.8496e-2 (7.75e-3) $ - $ | 5.1696e-2 (4.83e-3) $ - $ | 3.6481e-2 (1.62e-3) $ - $ | 6.2797e-2 (3.36e-3) $ - $ | 3.3979e-2 (1.08e-3) |
| IMOP6 | 3 | 10 | 1.9018e-1 (2.16e-1) $ - $ | 1.1559e-1 (1.91e-1) $ - $ | 5.6896e-2 (6.64e-3) $ - $ | 4.7488e-2 (2.04e-3) $ - $ | 4.7660e-1 (1.94e-1) $ - $ | 1.4029e-1 (1.82e-1) $ - $ | 3.2074e-2 (7.17e-4) |
| IMOP7 | 3 | 10 | 8.2949e-1 (2.87e-1) $ - $ | 6.4672e-1 (4.12e-1) $ - $ | 5.8980e-2 (2.87e-3) $ - $ | 1.3336e-1 (2.34e-1) $ - $ | 9.3555e-1 (1.67e-3) $ - $ | 9.0978e-1 (2.72e-2) $ - $ | 3.5658e-2 (6.98e-4) |
| IMOP8 | 3 | 10 | 1.3487e-1 (2.99e-3) $ - $ | 1.0245e-1 (1.11e-3) $ - $ | 9.8327e-2 (3.51e-3) $ - $ | 8.7488e-2 (1.62e-3) $ - $ | 1.3496e-1 (1.47e-2) $ - $ | 1.9142e-1 (1.53e-1) $ - $ | 7.5254e-2 (2.42e-3) |
| $ +/-/\approx $ | 0/8/0 | 0/6/2 | 0/8/0 | 0/8/0 | 0/8/0 | 0/8/0 | |||
DownLoad:
CSV
| Problem | $ m $ | $ D $ | RVEA | MOEA/D-AWA | RPS-NSGA-II | MOEA/D-UR | MaOEA-PDS | HEA | SDEA |
| IMOP1 | 2 | 10 | 9.8505e-1 (1.03e-4) $ \approx $ | 9.8642e-1 (8.82e-5) $ + $ | 9.8406e-1 (9.09e-4) $ - $ | 6.6970e-1 (1.08e-1) $ - $ | 9.8193e-1 (1.43e-3) $ - $ | 9.6919e-1 (2.54e-3) $ - $ | 9.8550e-1 (9.05e-4) |
| IMOP2 | 2 | 10 | 2.3134e-1 (6.03e-5) $ - $ | 1.0160e-1 (2.83e-2) $ - $ | 1.3071e-1 (6.80e-2) $ - $ | 9.3322e-2 (2.30e-3) $ - $ | 9.0909e-2 (9.62e-9) $ - $ | 2.1301e-1 (5.95e-3) $ - $ | 2.3168e-1 (3.22e-5) |
| IMOP3 | 2 | 10 | 6.5536e-1 (3.17e-4) $ - $ | 6.5257e-1 (8.66e-4) $ - $ | 6.5257e-1 (7.38e-4) $ - $ | 2.7852e-1 (6.65e-2) $ - $ | 6.5573e-1 (3.54e-3) $ \approx $ | 6.4070e-1 (1.50e-2) $ - $ | 6.5745e-1 (2.15e-4) |
| IMOP4 | 3 | 10 | 4.2570e-1 (2.64e-3) $ - $ | 4.1148e-1 (2.27e-3) $ - $ | 4.0835e-1 (3.48e-3) $ - $ | 4.2179e-1 (2.96e-3) $ - $ | 4.3319e-1 (5.31e-4) $ - $ | 4.1798e-1 (2.77e-3) $ - $ | 4.3436e-1 (1.79e-4) |
| IMOP5 | 3 | 10 | 4.6123e-1 (5.43e-3) $ - $ | 4.3088e-1 (1.08e-2) $ - $ | 4.7988e-1 (1.88e-2) $ \approx $ | 5.0180e-1 (2.24e-2) $ + $ | 4.9271e-1 (1.40e-2) $ + $ | 4.7985e-1 (1.05e-2) $ \approx $ | 4.7432e-1 (6.32e-3) |
| IMOP6 | 3 | 10 | 4.8716e-1 (6.81e-2) $ - $ | 3.8249e-1 (1.87e-1) $ - $ | 5.0737e-1 (3.38e-3) $ - $ | 5.1478e-1 (2.26e-3) $ - $ | 1.3785e-1 (1.68e-1) $ - $ | 4.6991e-1 (6.48e-2) $ - $ | 5.2810e-1 (4.61e-4) |
| IMOP7 | 3 | 10 | 2.1513e-1 (2.05e-1) $ - $ | 1.3956e-1 (1.29e-1) $ - $ | 4.9775e-1 (3.87e-3) $ - $ | 4.5807e-1 (1.41e-1) $ - $ | 9.0920e-2 (1.23e-5) $ - $ | 9.3341e-2 (2.27e-3) $ - $ | 5.2769e-1 (5.24e-4) |
| IMOP8 | 3 | 10 | 4.8617e-1 (1.65e-3) $ - $ | 4.6607e-1 (7.19e-3) $ - $ | 5.3480e-1 (3.13e-2) $ \approx $ | 5.4614e-1 (2.99e-2) $ \approx $ | 4.8785e-1 (1.18e-2) $ - $ | 4.5786e-1 (3.35e-2) $ - $ | 5.5282e-1 (3.41e-2) |
| $ +/-/\approx $ | 0/7/1 | 1/7/0 | 0/6/2 | 1/6/1 | 1/6/1 | 0/7/1 | |||
DownLoad:
CSV
IGD and HV results achieved by seven MaOEAs on the WFG test suite are summarized in Table 2 and Table 3, respectively. As shown in these tables, SDEA achieves 17 best IGD results and 19 HV results out of 27 test instances. As for six compared algorithms (RVEA, MOEA/D-AWA, RPS-NSGA-II, MOEA/D-UR, MaOEA-PDS, and HEA), they achieve only 2, 3, 0, 0, 5, and 0 best IGD results of 27 test instances, and achieve only 1, 3, 0, 0, 2, and 1 best HV results of 27 test instances.
WFG1, WFG2, and WFG3 are characterized by the irregular PF shapes, which have the mixed PF, disconnected PF, and degenerate PF in turn. Such characteristics make MaOEAs are difficult to solve. Even so, SDEA defeats all its competitors on most of test instances of WFG1 and WFG2. This is because the self-organizing decomposition manner are not sensitive to the PF shapes. On the contrary, the remaining algorithms more or less depend on the predefined reference vectors, and thus they do not perform satisfactorily. As for WFG3, HEA and SDEA are two top algorithms.This may be because HEA first emphasizes the convergence and thus makes more individuals converge to the degenerate PF.
The PF shapes of WFG4-9 are all concave. WFG4 and WFG5 are used to test whether an algorithm is capable of escape from the local optima. SDEA exceeds the other MaOEAs on most test cases, since the developed self-organizing decomposition manner and designed cooperative diversity measure are beneficial to improve the diversity. For WFG6 with non-separable variables, MaOEA-PDS performs best on this test problem, indicating that MaOEA-PDS is suitable to solve the problem. WFG7-9 all cover some biases, which make it difficult for algorithms to manage diversity. The best results are achieved by SDEA in terms of IGD and HV indicators. The reasons for this are the same as that of performing better on the WFG4-5.
In order to verify the significance difference between SDEA and its each competitor statistically, we conduct the Wilcoxon rank-sum tests on the IGD and HV values of all WFG test instances. Table 4 gives the corresponding results. From this table, we see the following two points: 1) The $ R^+ $ values of SDEA vs each competitor are much greater than the $ R^- $ values of SDEA vs each competitor; 2) each $ p $ value of SDEA vs each competitor is less than 0.05. According to such results, SDEA is significantly better than these six comparison methods statistically.
Figure 4 gives the parallel coordinate plots of final individuals obtained by seven MaOEAs for 15-objective WFG1. As shown in this figure, the plot of SDEA is most similar to that of the true PF in comparison with the other MaOEAs. For RVEA, RPS-NSGA-II, and MaOEA-PDS, the individuals obtained by them mainly distribute on the PF boundary or the PF center. As for the remaining two MaOEAs, partial objective values of them are less than the actual values. This indicates that MOEA/D-AWA, MOEA/D-UR, and HEA cannot make individuals cover the whole PF. In addition, Figure 5 gives the convergence curves of HV achieved by seven MaOEAs on WFG1 and WFG6. From this figure, one can observe that the convergence spped of SDEA is inferior to that of MOEA/D-AWA and RPS-NSGA-II. This may be because the PF distribution of WFG1 is similar to that of predefined reference vectors. Even so, SDEA obtains the great HV value on WFG1. As for the WFG6, the convergence speed and HV value of SDEA is significantly superior to that of its competitors.
Tables 5 and 6 present IGD and HV results achieved by seven MaOEAs on the MaF test suite in turn. For the IGD results, SDEA is significantly better than or similar to its six competitors on 29, 27, 30, 25, 30, and 29 out of 30 test instances in turn. Moreover, SDEA achieves 18 optimal HV results of 30 test instances, while its six competitors achieve only 3, 2, 2, 1, 1, and 3 best results of 39 test instances, respectively.
The difficulty of MaF1-3 for a MaOEA is to make the population converge to the true PF. SDEA performs best on most test instances of MaF1 and MaF2, while SDEA ranks only second to the optimal algorithm on each test instance. This indicates that SDEA is capable of making the population converge to the true PF, as shown in Figure 6(f). For MaF4-5 with the badly scaled PF, SDEA is the top algorithm, which means that SDEA is suited to deal with such problems. MaF6 is a more difficult problem, as it has the degenerate PF.
Nonetheless, SDEA performs better than its competitors for most test instances. This may be because SDEA utilizes the individuals with good diversity to guide the population evolution, which can make more individuals fall on the PF. MaF7 is a multi-modal problem and has discrete PF. SDEA is the optimal algorithm for most test instances. This confirms the motivation rationality of this work. MaF8-9 all have the huge search space, which is used to test the convergence of algorithms. SDEA is significantly better than the other algorithms in terms of IGD values and HV values. MaF10 is similar to WFG1. SDEA achieves the 2 best IGD results and the 1 best HV results in turn. The remaining optimal IGD and HV results are obtained by the HEA.
To verify the significant difference between SDEA and each competitor statistically, we conduct the Wilcoxon rank-sum tests for the IGD and HV values of all MaF test instances. Table 7 gives the corresponding results. From this table, we see the following two points: 1) The $ R^+ $ values of SDEA vs each competitor are much greater than the $ R^- $ values of SDEA vs each competitor; and 2) each $ p $ value of SDEA vs each competitor is less than 0.05. According to such results, SDEA is significantly better than these six comparison methods statistically.
To further test the performance of SDEA in dealing with problems with irregular PF shapes, we compare SDEA with its six competitors on the IMOP test suit. In the experiment, the population size $ N $ is set to 105, while other settings follow Sections 4.1–4.3. Tables 8 and Tables 9 summarize the IGD and HV results achieved by these seven algorithms, respectively. The proposed SDEA has a clear advantage over these six state-of-the-art MaOEAs. To be specific, SIEA achieves the 8 best IGD results and 6 best HV results, respectively. Such results demonstrate that SDEA has the high competitiveness in solving problems with irregular PFs again.
Figure 7 gives the distribution plots of final individuals obtained by seven MaOEAs on 3-objective IMOP8. According to this figure, we know that individuals obtained by the SDEA can completely cover all segments of the true PF in comparison with the other MaOEAs. As for the remaining MaOEAs, they fail to make individuals converge to some PF segments.
SDEA mostly includes two core contents: Self-organizing decomposition manner and cooperative diversity measure. In this subsection, we aim to investigate the effect of these two core contents. Therefore, three variants of SDEA are designed to compare with the original SDEA on the WFG test suite, whose details are as follows:
1) SDEA1: The developed self-organizing decomposition manner is replaced with the predefined reference vectors.
2) SDEA2: The local diversity in the designed cooperative diversity measure is retained, while the global diversity in the designed cooperative diversity measure is removed.
3) SDEA3: The local diversity in the designed cooperative diversity measure is removed, while the global diversity in the designed cooperative diversity measure is retained.
4) SDEA4: The diversity of individuals is evaluated by the crowded distance in NSGA-II. The remaining components stay the same as SDEA.
5) SDEA5: The diversity of individuals is evaluated by the angle value between each individual and its closest individual. The remaining components stay the same as SDEA.
The HV results obtained by three variants of SDEA and SDEA for the WFG test suite are given in Table 10. SDEA1 is significantly superior or similar to SDEA on 4 of 27 test problems. This indicates that the overall performance of SDEA is better than that of SDEA1. This is because that the self-organizing decomposition manner make SDEA have the high generality. Compared with SDEA2, SDEA shows superiority for 14 test instances, inferiority for 3 test instances, and similarity for 10 test instance. For SDEA3, it is only significantly better than SDEA for 2 test instances.
| Problem | $ m $ | $ D $ | SDEA1 | SDEA2 | SDEA3 | SDEA4 | SDEA5 | SDEA |
| WFG1 | 5 | 14 | 7.2318e-1 (6.27e-3) $ - $ | 9.4742e-1 (6.19e-3) $ - $ | 9.3968e-1 (6.35e-4) $ - $ | 6.7091e-1 (4.29e-3) $ - $ | 8.9150e-1 (4.27e-4) $ - $ | 9.9801e-1 (3.79e-4) |
| 10 | 19 | 8.9180e-1 (2.25e-4) $ - $ | 9.9752e-1 (3.69e-3) $ \approx $ | 9.5308e-1 (1.86e-3) $ - $ | 8.5984e-1 (5.27e-3) $ - $ | 6.9721e-1 (4.81e-4) $ - $ | 9.9963e-1 (3.80e-4) | |
| 15 | 24 | 8.7286e-1 (5.95e-3) $ - $ | 9.8711e-1 (3.02e-2) $ - $ | 9.2462e-1 (6.65e-4) $ \approx $ | 8.3671e-1 (4.22e-3) $ - $ | 9.1062e-1 (3.71e-4) $ - $ | 9.9998e-1 (2.41e-5) | |
| WFG2 | 5 | 14 | 9.9527e-1 (5.95e-4) $ \approx $ | 9.5968e-1 (2.08e-3) $ - $ | 9.9052e-1 (2.71e-3) $ \approx $ | 8.5132e-1 (2.94e-3) $ - $ | 7.0914e-1 (5.22e-4) $ - $ | 9.9366e-1 (7.60e-4) |
| 10 | 19 | 9.2610e-1 (2.83e-3) $ - $ | 9.9485e-1 (1.88e-3) $ \approx $ | 9.6025e-1 (3.73e-3) $ - $ | 7.5916e-1 (5.21e-3) $ - $ | 8.3706e-1 (3.11e-4) $ - $ | 9.9221e-1 (9.41e-4) | |
| 15 | 24 | 9.1029e-1 (3.59e-3) $ - $ | 9.9317e-1 (3.77e-3) $ \approx $ | 9.9051e-1 (4.57e-3) $ \approx $ | 7.8204e-1 (3.95e-3) $ - $ | 8.7618e-1 (2.56e-4) $ - $ | 9.9681e-1 (1.72e-3) | |
| WFG3 | 5 | 14 | 5.2384e-2 (2.57e-2) $ \approx $ | 7.3816e-2 (4.76e-2) $ - $ | 6.1034e-2 (1.52e-2) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 1.4825e-1 (3.29e-2) |
| 10 | 19 | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) | |
| 15 | 24 | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) | |
| WFG4 | 5 | 14 | 7.8652e-1 (4.19e-3) $ - $ | 8.0753e-1 (2.71e-3) $ \approx $ | 7.6037e-1 (1.02e-2) $ - $ | 7.6052e-1 (3.96e-3) $ - $ | 6.2513e-1 (2.41e-3) $ - $ | 8.0615e-1 (1.22e-3) |
| 10 | 19 | 8.7023e-1 (3.95e-3) $ - $ | 9.3051e-1 (4.15e-3) $ - $ | 8.5127e-1 (3.72e-2) $ - $ | 7.7108e-1 (1.95e-3) $ - $ | 8.4058e-1 (2.11e-3) $ - $ | 9.4963e-1 (2.90e-3) | |
| 15 | 24 | 7.8157e-1 (2.74e-3) $ - $ | 9.8016e-1 (2.43e-3) $ \approx $ | 8.6352e-1 (4.91e-2) $ - $ | 8.8205e-1 (2.33e-3) $ - $ | 8.6073e-1 (1.56e-4) $ - $ | 9.8303e-1 (1.78e-3) | |
| WFG5 | 5 | 14 | 8.5082e-1 (1.43e-3) $ + $ | 7.8365e-1 (1.24e-3) $ - $ | 7.6019e-1 (5.31e-3) $ - $ | 5.3601e-1 (4.28e-3) $ - $ | 4.7201e-1 (2.17e-4) $ - $ | 7.6022e-1 (4.13e-4) |
| 10 | 19 | 8.9681e-1 (1.78e-3) $ \approx $ | 9.0254e-1 (1.38e-3) $ + $ | 8.5597e-1 (5.40e-3) $ - $ | 6.2801e-1 (5.28e-3) $ - $ | 7.0921e-1 (4.71e-4) $ - $ | 8.9832e-1 (6.00e-4) | |
| 15 | 24 | 8.9087e-1 (6.15e-4) $ - $ | 8.7194e-1 (2.83e-3) $ - $ | 8.5701e-1 (4.22e-3) $ - $ | 8.2071e-1 (3.45e-3) $ - $ | 9.0427e-1 (3.18e-4) $ \approx $ | 9.1455e-1 (4.77e-4) | |
| WFG6 | 5 | 14 | 7.2679e-1 (1.37e-2) $ - $ | 7.4503e-1 (1.43e-2) $ \approx $ | 7.6805e-1 (1.47e-2) $ + $ | 4.2131e-1 (4.37e-2) $ - $ | 7.3638e-1 (2.32e-2) $ \approx $ | 7.4043e-1 (1.41e-2) |
| 10 | 19 | 8.6787e-1 (1.69e-2) $ \approx $ | 8.3508e-1 (1.28e-2) $ - $ | 6.8542e-1 (3.64e-2) $ - $ | 6.9415e-1 (3.28e-3) $ - $ | 6.7155e-1 (2.66e-2) $ - $ | 8.6998e-1 (1.36e-2) | |
| 15 | 24 | 8.9551e-1 (1.21e-2) $ \approx $ | 8.8418e-1 (2.80e-2) $ \approx $ | 6.5042e-1 (1.26e-1) $ - $ | 7.3942e-1 (5.28e-1) $ - $ | 6.5218e-1 (4.01e-2) $ - $ | 8.8649e-1 (1.31e-2) | |
| WFG7 | 5 | 14 | 7.2351e-1 (2.51e-3) $ - $ | 8.0052e-1 (2.23e-3) $ - $ | 8.0617e-1 (5.39e-3) $ \approx $ | 6.5035e-1 (4.81e-3) $ - $ | 7.1468e-1 (3.26e-4) $ - $ | 8.0758e-1 (2.98e-4) |
| 10 | 19 | 9.6087e-1 (2.53e-3) $ - $ | 9.6095e-1 (2.04e-3) $ - $ | 8.6218e-1 (2.76e-2) $ - $ | 7.8901e-1 (3.27e-3) $ - $ | 8.9122e-1 (2.16e-3) $ - $ | 9.5879e-1 (1.82e-3) | |
| 15 | 24 | 9.8611e-1 (9.20e-4) $ + $ | 9.8672e-1 (7.25e-4) $ + $ | 9.1340e-1 (2.51e-2) $ - $ | 7.5011e-1 (3.28e-3) $ - $ | 8.9041e-1 (2.75e-4) $ - $ | 9.8483e-1 (1.88e-3) | |
| WFG8 | 5 | 14 | 6.6585e-1 (4.70e-3) $ - $ | 6.6738e-1 (3.69e-3) $ - $ | 6.9475e-1 (6.81e-4) $ \approx $ | 3.7602e-1 (2.81e-3) $ - $ | 4.9130e-1 (4.26e-4) $ - $ | 6.9502e-1 (8.48e-4) |
| 10 | 19 | 8.5285e-1 (1.97e-2) $ - $ | 8.5119e-1 (2.81e-2) $ - $ | 7.7624e-1 (3.74e-2) $ - $ | 5.7165e-1 (3.25e-3) $ - $ | 6.3022e-1 (5.41e-3) $ - $ | 8.6059e-1 (5.74e-3) | |
| 15 | 24 | 9.2307e-1 (3.57e-3) $ + $ | 9.1868e-1 (4.57e-3) $ \approx $ | 9.1706e-1 (5.20e-3) $ \approx $ | 7.2511e-1 (3.70e-3) $ - $ | 9.2908e-1 (4.71e-3) $ - $ | 9.1724e-1 (5.33e-3) | |
| WFG9 | 5 | 14 | 6.5026e-1 (3.14e-3) $ - $ | 7.3908e-1 (2.57e-3) $ - $ | 7.2057e-1 (5.37e-3) $ - $ | 6.4742e-1 (5.96e-3) $ - $ | 6.3968e-1 (4.35e-3) $ - $ | 7.6045e-1 (3.47e-3) |
| 10 | 19 | 8.9267e-1 (5.22e-3) $ + $ | 8.9287e-1 (6.82e-3) $ + $ | 8.3925e-1 (4.68e-2) $ - $ | 6.4472e-1 (4.28e-3) $ - $ | 6.3082e-1 (5.37e-3) $ - $ | 8.7926e-1 (6.82e-3) | |
| 15 | 24 | 8.5781e-1 (6.30e-2) $ - $ | 8.5037e-1 (6.25e-2) $ - $ | 9.1051e-1 (8.25e-3) $ + $ | 4.7027e-1 (6.81e-3) $ - $ | 5.066e-1 (4.27e-2) $ - $ | 8.6638e-1 (1.92e-2) | |
| $ +/-/\approx $ | 3/17/7 | 3/14/10 | 2/16/9 | 0/24/3 | 0/22/5 | |||
DownLoad:
CSV
As for SDEA4 and SDEA5, they do not perform significantly better than SDEA for each test instance. This is because these diversity measures in SDEA4 and SDEA5 cannot match the framework of SDEA. Based on the above analysis, two core contents (self-organizing decomposition manner and cooperative diversity measure) in SDEA are important to the comprehensive performance of SDEA.
The proposed SDEA includes a parameter $ \delta $ that influences the generation of reference vectors. In order to investigate its influences on the performance of SDEA, we conduct experiments to evaluate the performance of SDEA with different $ \delta $ values
| $ (i.e., \delta = {0.01,0.03,0.04,0.05,0.07,0.08, 0.1}). $ |
The other experimental settings follow Sections 4.1–4.3. HV results obtained by SDEA with different $ \delta $ values on WFG1-2 and MaF3 test problem are presented in Figure 8. From this figure, we can observe that SDEA obtains the different HV results under the different $ \delta $ values. This indicates that SDEA is sensitive to parameter $ \delta $. The reasons for this may be attributed to the following facts: 1) If $ \delta $ is very large, generated reference vectors cannot reflect the the PF distribution; and 2) if $ \delta $ is very small, generated reference vectors cannot reflect the the PF distribution as well. Moreover, we can observe that when $ \delta $ is set to 0.05, SDEA can achieve the best performance for most test instances. Therefore, in the work, we set $ \delta $ to 0.05.
According to the behavior studies of SDEA and comparative results, we can observe that most existing decomposition based evolutionary algorithms for many-objective optimization may not work effectively, especially in dealing with these problems with irregular PFs. This might be attributed to the fact that they depend on the predefined reference vectors to some degree, so that they are sensitive to the PF shapes. In our SDEA, we develop a self-organizing decomposition method that utilizes these individuals with good diversity as reference vectors, thereby adapting to different PFs. The effectiveness of the self-organizing decomposition method has been confirmed by the corresponding the behavior studies of SDEA (i.e., SDEA1). In existing decomposition based MaOEAs, most consider only the global diversity by utilizing the reference vectors. In contrast, SDEA takes global diversity and local diversity into account simultaneously. This can better evaluate the distributions of population individuals, which has also been confirmed by our ablation study (i.e., SDEA2-5).
However, our empirical results on the selected WFG test problems indicate that the proposed SDEA does not present a clear advantage over other algorithms, as shown in Tables 2 and 3. Instead, the algorithms that use predefined reference points perform better than SDEA, where these algorithms include MOEA/D-AWA, MaOEA-PDS, and HEA. As shown in Tables 4 and 5, SDEA becomes less effective on MaF6. In addition, when SDEA solves these problems with the degenerate PF, the developed self-organizing decomposition may mislead the population evolution. This is because the developed self-organizing decomposition prefers these individuals with good diversity and further influences that the population can converge to the true PF.
To testing the ability of SDEA in dealing with practical problems, SDEA is compared with its six competitors on two practical problems, which are the car side impact design problem (CSIDP), and water resources management problem (WRMP) [52]. More details of CSIDP and WRMP can be found in [53]. The CSIDP has three objectives, while the WRMP covers five objectives. In addition, the number of decision variables for CSIDP and WRMP is set to seven and three in turn. The experimental setups follow Sections 4.1–4.3. Note that the population size $ N $ for CSIDP is set to 105. For the constraints of CSIDP and WRMP, we adopt the constraint handling technology in [1]. To be specific, we combine the constraint handling technology in [54] with the fitness evaluation of seven MaOEAs to handle constraints. After combining the constraint handling technology in [1] with the fitness evaluation of SDEA, the mathematical description of fitness evaluation is as follows:
| $ F(x)=C(x)+D(x)+(1−fr)CV(x) $ | (5.1) |
where $ f_r $ is the feasible ratio of current population, and $ CV(x) $ denotes the constraint violation value of individual $ x $.
HV results achieved by these seven MaOEAs on CSIDP and WRMP are summarized in Table 11. As presented in this table, SDEA significantly outperforms its six competitors on WRMP. As for the CSIDP, SDEA is comparable to the optimal MaOEA (i.e., HEA). Such results can contribute to the fact that the collaboration of each component in SDEA makes SDEA obtain an individual set with good convergence and diversity. Moreover, the self-organizing decomposition can adapt different Pareto fronts. To intuitively present the superiority of SDEA, we plot the population distributions of individuals achieved by seven MaOEAs on WRMP in Figure 9. As shown in this figure, individuals obtained by SDEA have better convergence and diversity. Therefore, SDEA can be considered an effective method for practical problems.
| Problem | $ m $ | $ D $ | RVEA | MOEA/D-AWA | RPS-NSGA-II | MOEA/D-UR | MaOEA-PDS | HEA | SDEA |
| CSIDP | 3 | 7 | 5.7035e-1 (1.72e-3) $ - $ | 3.7053e-1 (2.52e-2) $ - $ | 5.0271e-1 (2.17e-2) $ - $ | 5.1019e-1 (1.34e-3) $ - $ | 4.5127e-1 (4.32e-2) $ - $ | 6.3021e-1 (2.85e-3) $ \approx $ | 6.2908e-1 (1.35e-3) |
| WRMP | 5 | 3 | 4.1208e-1 (2.53e-2) $ - $ | 5.23721e-1 (1.59e-2) $ - $ | 3.9806e-1 (1.75e-2) $ - $ | 4.3253e-1 (2.98e-3) $ - $ | 5.7025e-1 (2.35e-2) $ - $ | 5.3920e-1 (2.12e-2) $ - $ | 7.5019e-1 (1.95e-2) |
| $ +/-/\approx $ | 0/2/0 | 0/2/0 | 0/2/0 | 0/2/0 | 0/2/0 | 0/1/1 | |||
DownLoad:
CSV
In this paper, a novel MaOEA, called SDEA, is proposed to solve MaOPs. In SDEA, an adaptive collaborative mechanism is developed to better balance convergence and diversity of the population. In this mechanism, angle selection operation first identifies a pair of individuals with the minimum angle value, and then multi-criterion deletion operation deletes a poor one according to the population state information. In addition, given that most MaOEAs ignore the dimensional influences on algorithm performance, the dimensional ranking based convergence measure, and the dimensional difference based diversity measure are designed to better evaluate the convergence and diversity of individuals, respectively. Experimental results for 74 benchmark test instances, two combination optimization problems, and 2 real-world problems have demonstrated that SDEA is an effective method in dealing with MaOPs.
Although SDEA has shown superior performance in solving MaOPs to some extent, there are still some issues that need to be further studied. First, how to execute the self-organizing decomposition according to the requirements instead of each iteration is worth exploring. This can reduce the computational complexity of SDEA. Second, real-world problems generally have many decision vectors. Therefore, how to extend SDEA to solve the large-scale MaOPs has practical significance.
Siyuan Zhao: Conceptualization, Methodology, Software, Writing-original draft; Zichun Shao: Validation, Funding acquisition, Writing-review & editing; Yile Chen: Validation, Supervision, Writing-review & editing; Liang Zheng: Formal analysis, Supervision, Writing-review & editing; Junming Chen: Validation, Supervision, Writing-review & editing. All authors have read and approved the final version of the manuscript for publication.
The authors declare that they have not used Artificial Intelligence (AI) tools in the creation of this article.
The authors declare that they have no conflicts of interest.
| [1] | Colagiuri S, Colagiuri R, Ward J (1998) National diabetes strategy and implementation plan. Canberra: Paragon Printers. |
| [2] | Pop-Busui R, Evans GW, Gerstein HC, et al. (2010) The ACCORD Study Group. Effects of cardiac autonomic dysfunction on mortality risk in the Action to Control Cardiovascular Risk in Diabetes (ACCORD) Trial. Diab Care 33: 1578-84. |
| [3] |
Spallone V, Ziegler D, Freeman R, et al. (2011) Cardiovascular autonomic neuropathy in diabetes: clinical impact, assessment, diagnosis, and management. Diab Metab Res Rev 27:639-53. doi: 10.1002/dmrr.1239
|
| [4] | Jelinek HF, Imam HM, Al-Aubaidy H, et al. (2013) Association of cardiovascular risk using nonlinear heart rate variability measures with the Framingham risk score in a rural population. Front Physiol 4: 186. |
| [5] |
Ziegler D (1994) Diabetic cardiovascular autonomic neuropathy: prognosis, diagnosis and treatment. Diab Metabol Rev 10: 339-83. doi: 10.1002/dmr.5610100403
|
| [6] |
Gerritsen J, Dekker JM, TenVoorde BJ, et al. (2001) Impaired autonomic function is associated with increased mortality, especially in subjects with diabetes, hypertension or a history of cardiovascular disease. Diab Care 24: 1793-8. doi: 10.2337/diacare.24.10.1793
|
| [7] | Johnston SC, Easton JD (2003) Are patients with acutely recovered cerebral ischemia more unstable? Stroke 4: 24-46. |
| [8] |
Ko SH, Kwon HS, Lee JM, et al. (2006) Cardiovascular autonomic neuropathy in patients with type 2 diabetes mellitus. J Korean Diab Assoc 30: 226-35. doi: 10.4093/jkda.2006.30.3.226
|
| [9] |
Agelink MW, Malessa R, Baumann B, et al. (2001) Standardized tests of heart rate variability: normal ranges obtained from 309 healthy humans, and effects of age, gender and heart rate. Clin Auton Res 11: 99-108. doi: 10.1007/BF02322053
|
| [10] |
Ewing DJ, Martyn CN, Young RJ, et al. (1985) The value of cardiovascular autonomic functions tests: 10 years experience in diabetes. Diab Care 8: 491-8. doi: 10.2337/diacare.8.5.491
|
| [11] |
Pumprla J, Howorka K, Groves D, et al. (2002) Functional assessment of HRV variability: physiological basis and practical applications. Int J Cardiol 84: 1-14. doi: 10.1016/S0167-5273(02)00057-8
|
| [12] |
Stern S, Sclarowsky S (2009) The ECG in diabetes mellitus. Circulation 120: 1633-6. doi: 10.1161/CIRCULATIONAHA.109.897496
|
| [13] | Reilly RB, Lee TC (2010) Electrograms (ECG, EEG, EMG, EOG). Technol Heal Care 18:443-58. |
| [14] |
Baumert M, Schlaich MP, Nalivaiko E, et al. (2011) Relation between QT interval variability and cardiac sympathetic activity in hypertension. Am J Physiol Heart Circ Physiol 300: H1412-7. doi: 10.1152/ajpheart.01184.2010
|
| [15] | Kelarev AV, Dazeley R, Stranieri A, et al. (2012) Detection of CAN by ensemble classifiers based on ripple down rules. Lect Notes Artif Int 7457: 147-59. |
| [16] |
Fang ZY, Prins JB, Marwick TH (2004) Diabetic cardiomyopathy: evidence, mechanisms, and therapeutic implications. Endocrinol Rev 25: 543-67. doi: 10.1210/er.2003-0012
|
| [17] | Huda S, Jelinek HF, Ray B, et al. (2010) Exploring novel features and decision rules to identify cardiovascular autonomic neuropathy using a Hybrid of Wrapper-Filter based feature selection. Marusic S, Palaniswami M, Gubbi J, et al, editors. Intelligent sensors, sensor networks and information processing, ISSNIP 2010. Sydney: IEEE Press, 297-302. |
| [18] | Witten H, Frank E (2011) Data mining: practical machine learning tools and techniques with java implementations. 3Eds, Sydney: Morgan Kaufmann, 2011. |
| [19] |
Hall M, Frank E, Holmes G, et al. (2009). The WEKA data mining software: an update. SIGKDD Explor 11(1): 10-8. doi: 10.1145/1656274.1656278
|
| [20] | Freund Y, Mason L (1999) The alternating decision tree learning algorithm. Proceedings of the sixteenth international conference on machine learning, 124-33. |
| [21] | Kotsiantis SB (2007) Supervised machine learning: A review of classification techniques. Informatica 31: 249-68. |
| [22] | Kohavi R (1996) Scaling up the accuracy of Naive-Bayes classifiers: a decision-tree hybrid. Proceedings of the 2nd international conference on knowledge discovery and data mining,202-7. |
| [23] | Breiman L, Friedman JH, Olshen RA, et al. (1984) Classification and regression trees. California: Wadsworth International Group. |
| [24] | Dazeley R, Yearwood J, Kang B, et al. (2010) Consensus clustering and supervised classification for profiling phishing emails in internet commerce secu+rity. In: Kang BH, Richards D, editors. Knowledge management and acquisition for smart systems and services, PKAW 2010. Daegu: Springer Verlag, 235-46. |
| [25] | Yearwood J, Webb D, Ma L, et al. (2009) Applying clustering and ensemble clustering approaches to phishing profiling. Proceedings of the 8th Australasian data mining conference, AusDM 2009. Curr Res Prac Inf Technol 101: 25-34. |
| [26] | Kang B, Kelarev A, Sale A, et al. (2006) A new model for classifying DNA code inspired by neural networks and FSA. Advances in Knowledge Acquisition and Management, 19th Australian Joint Conference on Artificial Intelligence, AI06. Lect Notes Comp Sci 4303:187-98. |
| [27] | Breiman L (1996) Bagging predictors. Mach Learn 24: 123-40. |
| [28] | Freund Y, Schapire R (1996) Experiments with a new boosting algorithm. Proceedings of 13th International Conference of Machince Learning 148-56. |
| [29] |
Webb G (2000) Multiboosting: A technique for combining boosting and wagging. Mach Learn 40: 159-96. doi: 10.1023/A:1007659514849
|
| [30] |
Wolpert D (1992) Stacked generalization. Neural Networks 5: 241-59. doi: 10.1016/S0893-6080(05)80023-1
|
| [31] | Melville P, Mooney R (2005) Creating diversity in ensembles using artificial data. Inf Fusion6: 99-111. |
| [32] | Ting K, Witten I (1997) Stacking bagged and dagged models. Fourteenth International Conference Machine Learning, 367-75. |
| [33] | Seewald AK, Furnkranz J (2001) An evaluation of grading classifiers. Hoffmann F, Adams N, Fisher D, et al, editors. Advances in intelligent data analysis, IDA 2001. Heidelberg: Springer,115-24. |
| [34] | Kelarev A, Stranieri A, Yearwood J, et al. (2012) Empirical study of decision trees and ensemble classifiers for monitoring of diabetes patients in pervasive healthcare, 15th International Conference on Networked-Based Information System, NBiS-2012. Melbourne: CPS, 441-6. |
| [35] | Huehn J, Huellermeier E (2009) FURIA: An algorithm for unordered fuzzy rule induction. DMKD 19: 293-319. |
| [36] | Kelarev A, Stranieri A, Yearwood J, et al. (2012) Improving classifications for cardiac autonomic neuropathy using multi-level ensemble classifiers and feature selection based on random forest. In: Zhao Y, Li J, Kennedy PJ, et al, editors. Data mining and analytics, 11th Australasian Data Mining Conference, AusDM 2012. Sydney: CRPIT, 134: 93-102. |
Figures(4) / Tables(2)
Herbert F. Jelinek, Jemal H. Abawajy, Andrei V. Kelarev, Morshed U. Chowdhury, Andrew Stranieri. Decision trees and multi-level ensemble classifiers for neurological diagnostics[J]. AIMS Medical Science, 2014, 1(1): 1-12. doi: 10.3934/medsci.2014.1.1
| $ m $ | Division($ H_{1} $, $ H_{2} $) | $ N $ |
| 5 | (6, 0) | 210 |
| 10 | (3, 2) | 275 |
| 15 | (2, 1) | 135 |
DownLoad:
CSV
| Problem | $ m $ | $ D $ | RVEA | MOEA/D-AWA | RPS-NSGA-II | MOEA/D-UR | MaOEA-PDS | HEA | SDEA |
| WFG1 | 5 | 14 | 5.4635e-1 (7.46e-2) $ - $ | 3.9587e-1 (1.26e-2) $ - $ | 7.2685e-1 (4.79e-2) $ - $ | 7.0831e-1 (4.08e-2) $ - $ | 3.8544e-1 (1.66e-2) $ \approx $ | 4.4033e-1 (1.03e-2) $ - $ | 3.7929e-1 (1.06e-2) |
| 10 | 19 | 1.4450e+0 (9.21e-2) $ - $ | 1.0525e+0 (2.06e-2) $ - $ | 1.5323e+0 (4.55e-2) $ - $ | 1.8904e+0 (3.14e-2) $ - $ | 1.5440e+0 (1.27e-1) $ - $ | 1.0905e+0 (5.08e-2) $ - $ | 9.4664e-1 (1.55e-2) | |
| 15 | 24 | 1.9111e+0 (1.67e-1) $ \approx $ | 1.9978e+0 (6.68e-2) $ - $ | 2.1808e+0 (3.71e-2) $ - $ | 2.4509e+0 (4.71e-2) $ - $ | 2.4542e+0 (5.92e-1) $ - $ | 1.7976e+0 (3.28e-2) $ - $ | 1.6338e+0 (4.70e-2) | |
| WFG2 | 5 | 14 | 3.8976e-1 (2.07e-3) $ \approx $ | 4.5517e-1 (1.33e-2) $ - $ | 6.0492e-1 (4.93e-2) $ - $ | 6.6631e-1 (7.09e-2) $ - $ | 4.0538e-1 (1.17e-2) $ - $ | 4.5076e-1 (7.40e-3) $ - $ | 3.8297e-1 (1.04e-2) |
| 10 | 19 | 1.0615e+0 (8.53e-2) $ \approx $ | 1.1886e+0 (3.23e-2) $ - $ | 1.7249e+0 (6.11e-2) $ - $ | 1.7981e+0 (5.37e-2) $ - $ | 1.0226e+0 (2.03e-2) $ + $ | 1.0919e+0 (2.49e-2) $ \approx $ | 1.1072e+0 (3.42e-2) | |
| 15 | 24 | 2.0018e+0 (9.86e-2) $ - $ | 2.4181e+0 (6.59e-2) $ - $ | 2.4740e+0 (4.28e-2) $ - $ | 1.7691e+0 (4.42e-2) $ \approx $ | 2.0263e+0 (2.09e-1) $ - $ | 1.8323e+0 (7.73e-2) $ - $ | 1.6338e+0 (4.70e-2) | |
| WFG3 | 5 | 14 | 4.4053e-1 (4.97e-2) $ \approx $ | 3.4144e-1 (3.25e-2) $ + $ | 1.8024e+0 (2.61e-1) $ - $ | 1.2584e+0 (1.48e-1) $ - $ | 5.0335e-1 (1.06e-1) $ \approx $ | 3.0801e-1 (5.42e-2) $ + $ | 4.5416e-1 (2.63e-2) |
| 10 | 19 | 2.0550e+0 (1.94e-1) $ + $ | 1.6373e+0 (2.49e-1) $ + $ | 6.6493e+0 (7.25e-1) $ - $ | 5.1583e+0 (2.80e-1) $ - $ | 1.8029e+0 (1.88e-1) $ + $ | 1.3715e+0 (4.29e-1) $ + $ | 3.4318e+0 (9.29e-1) | |
| 15 | 24 | 3.5356e+0 (1.39e+0) $ + $ | 3.5804e+0 (1.58e+0) $ + $ | 1.3448e+1 (5.47e-1) $ - $ | 9.3436e+0 (6.32e-1) $ \approx $ | 4.8727e+0 (3.73e-1) $ + $ | 3.1511e+0 (1.07e+0) $ + $ | 8.5894e+0 (2.28e+0) | |
| WFG4 | 5 | 14 | 9.6578e-1 (1.33e-3) $ \approx $ | 1.0927e+0 (1.07e-2) $ - $ | 1.3665e+0 (6.93e-2) $ - $ | 1.4823e+0 (1.09e-1) $ - $ | 9.6670e-1 (1.50e-3) $ - $ | 1.1283e+0 (2.19e-2) $ - $ | 9.6503e-1 (1.29e-3) |
| 10 | 19 | 4.5176e+0 (1.11e-2) $ - $ | 4.5238e+0 (8.79e-2) $ - $ | 5.9297e+0 (1.33e-1) $ - $ | 5.9074e+0 (1.31e-1) $ - $ | 4.5598e+0 (2.09e-2) $ - $ | 4.8097e+0 (2.22e-2) $ - $ | 4.3489e+0 (5.08e-2) | |
| 15 | 24 | 9.4038e+0 (3.78e-2) $ - $ | 9.6090e+0 (2.38e-1) $ - $ | 1.1576e+1 (3.12e-1) $ - $ | 1.1622e+1 (3.11e-1) $ - $ | 9.4306e+0 (2.12e-2) $ - $ | 1.0005e+1 (1.06e-1) $ - $ | 9.3085e+0 (1.21e-1) | |
| WFG5 | 5 | 14 | 9.5715e-1 (6.34e-4) $ \approx $ | 1.0975e+0 (1.38e-2) $ - $ | 1.2890e+0 (6.34e-2) $ - $ | 1.3457e+0 (7.76e-2) $ - $ | 9.5725e-1 (1.32e-3) $ \approx $ | 1.1162e+0 (2.48e-2) $ - $ | 9.5719e-1 (1.01e-3) |
| 10 | 19 | 4.5065e+0 (1.33e-2) $ - $ | 4.9460e+0 (6.56e-2) $ - $ | 5.7480e+0 (1.62e-1) $ - $ | 5.7359e+0 (2.36e-1) $ - $ | 4.5447e+0 (9.44e-3) $ - $ | 4.7827e+0 (3.58e-2) $ - $ | 4.3479e+0 (2.75e-2) | |
| 15 | 24 | 9.2848e+0 (6.48e-3) $ - $ | 1.0669e+1 (1.74e-1) $ - $ | 1.1357e+1 (4.98e-1) $ - $ | 1.1187e+1 (2.09e-1) $ - $ | 9.3812e+0 (1.09e-2) $ - $ | 9.9388e+0 (1.70e-1) $ - $ | 9.2056e+0 (6.68e-2) | |
| WFG6 | 5 | 14 | 9.6252e-1 (1.14e-3) $ - $ | 1.0948e+0 (1.85e-2) $ - $ | 1.5668e+0 (1.11e-1) $ - $ | 1.7897e+0 (1.18e-1) $ - $ | 9.5952e-1 (9.96e-4) $ + $ | 1.1153e+0 (1.30e-2) $ - $ | 9.6094e-1 (8.47e-4) |
| 10 | 19 | 4.5765e+0 (1.85e-2) $ - $ | 4.5781e+0 (1.62e-1) $ - $ | 6.8764e+0 (2.00e-1) $ - $ | 6.6838e+0 (1.58e-1) $ - $ | 4.5748e+0 (1.36e-2) $ - $ | 4.8668e+0 (3.52e-2) $ - $ | 4.2328e+0 (4.95e-2) | |
| 15 | 24 | 9.4004e+0 (1.63e-1) $ + $ | 1.0520e+1 (5.68e-1) $ - $ | 1.2902e+1 (4.00e-1) $ - $ | 1.2160e+1 (2.80e-1) $ - $ | 9.3998e+0 (1.63e-2) $ + $ | 1.0004e+1 (1.66e-1) $ \approx $ | 9.8458e+0 (3.07e-1) | |
| WFG7 | 5 | 14 | 9.6656e-1 (6.58e-4) $ \approx $ | 1.0954e+0 (1.10e-2) $ - $ | 1.5246e+0 (1.34e-1) $ - $ | 1.8936e+0 (1.74e-1) $ - $ | 9.6620e-1 (5.91e-4) $ + $ | 1.1273e+0 (1.38e-2) $ - $ | 9.6704e-1 (7.35e-4) |
| 10 | 19 | 4.5568e+0 (5.79e-2) $ - $ | 4.5835e+0 (5.31e-2) $ - $ | 5.9259e+0 (1.69e-1) $ - $ | 6.1533e+0 (2.17e-1) $ - $ | 4.5483e+0 (2.31e-2) $ - $ | 4.8166e+0 (1.48e-2) $ - $ | 4.3796e+0 (4.55e-2) | |
| 15 | 24 | 9.3673e+0 (7.65e-2) $ \approx $ | 9.6379e+0 (1.09e-1) $ - $ | 1.1123e+1 (4.45e-1) $ - $ | 1.1392e+1 (5.91e-1) $ - $ | 9.3975e+0 (2.34e-2) $ - $ | 1.0018e+1 (1.48e-1) $ - $ | 9.3218e+0 (8.84e-2) | |
| WFG8 | 5 | 14 | 9.9498e-1 (7.53e-3) $ \approx $ | 1.1109e+0 (9.54e-3) $ - $ | 1.5536e+0 (1.08e-1) $ - $ | 1.5334e+0 (1.01e-1) $ - $ | 9.9122e-1 (1.92e-3) $ \approx $ | 1.1370e+0 (1.90e-2) $ - $ | 9.8996e-1 (1.47e-3) |
| 10 | 19 | 4.6331e+0 (2.82e-1) $ - $ | 4.4239e+0 (6.29e-2) $ - $ | 6.3655e+0 (3.95e-1) $ - $ | 6.4510e+0 (3.48e-1) $ - $ | 4.6307e+0 (3.14e-2) $ - $ | 4.8988e+0 (2.70e-2) $ - $ | 4.2874e+0 (8.91e-2) | |
| 15 | 24 | 1.0290e+1 (4.26e-1) $ - $ | 1.1028e+1 (6.27e-1) $ - $ | 1.1822e+1 (4.75e-1) $ - $ | 9.3252e+0 (3.36e-2) $ \approx $ | 9.7060e+0 (7.39e-2) $ - $ | 9.3963e+0 (3.55e-1) $ - $ | 9.1878e+0 (1.93e-1) | |
| WFG9 | 5 | 14 | 9.3407e-1 (2.99e-3) $ + $ | 1.0364e+0 (2.11e-2) $ - $ | 1.2761e+0 (3.72e-2) $ - $ | 1.4277e+0 (9.88e-2) $ - $ | 9.3237e-1 (3.36e-3) $ + $ | 1.0890e+0 (1.53e-2) $ - $ | 9.4329e-1 (1.94e-3) |
| 10 | 19 | 4.3721e+0 (4.86e-2) $ - $ | 4.4745e+0 (1.14e-1) $ - $ | 5.4624e+0 (1.45e-1) $ - $ | 5.5268e+0 (2.36e-1) $ - $ | 4.4653e+0 (3.26e-2) $ - $ | 4.7361e+0 (2.93e-2) $ - $ | 4.2489e+0 (5.03e-2) | |
| 15 | 24 | 8.7261e+0 (1.09e-1) $ + $ | 9.6567e+0 (2.62e-1) $ - $ | 1.0325e+1 (2.92e-1) $ - $ | 1.0114e+1 (4.07e-1) $ - $ | 8.8524e+0 (1.04e-1) $ + $ | 9.5723e+0 (9.41e-2) $ - $ | 9.1838e+0 (1.46e-1) | |
| $ +/-/\approx $ | 5/13/9 | 3/24/0 | 0/27/0 | 0/24/3 | 6/18/3 | 3/22/2 | |||
DownLoad:
CSV
| Problem | $ m $ | $ D $ | RVEA | MOEA/D-AWA | RPS-NSGA-II | MOEA/D-UR | MaOEA-PDS | HEA | SDEA |
| WFG1 | 5 | 14 | 8.3053e-1 (2.90e-2) $ - $ | 9.7421e-1 (2.46e-2) $ - $ | 8.5218e-1 (4.63e-2) $ - $ | 9.7917e-1 (2.42e-2) $ - $ | 9.8167e-1 (1.93e-2) $ - $ | 9.8866e-1 (1.93e-2) $ \approx $ | 9.9801e-1 (3.79e-4) |
| 10 | 19 | 7.0216e-1 (7.03e-2) $ - $ | 9.9014e-1 (1.37e-2) $ - $ | 9.0923e-1 (5.62e-2) $ - $ | 9.9337e-1 (1.75e-3) $ - $ | 9.9065e-1 (1.89e-3) $ - $ | 9.9715e-1 (1.56e-3) $ - $ | 9.9963e-1 (3.80e-4) | |
| 15 | 24 | 9.9970e-1 (2.81e-4) $ - $ | 9.9908e-1 (5.68e-4) $ - $ | 9.9719e-1 (7.74e-4) $ - $ | 9.9822e-1 (9.05e-4) $ - $ | 9.1853e-1 (9.60e-2) $ - $ | 9.9713e-1 (6.41e-4) $ - $ | 9.9998e-1 (2.41e-5) | |
| WFG2 | 5 | 14 | 9.9206e-1 (1.59e-3) $ - $ | 9.7930e-1 (7.83e-3) $ - $ | 9.5050e-1 (1.24e-2) $ - $ | 9.7737e-1 (6.58e-3) $ - $ | 9.8845e-1 (2.47e-3) $ - $ | 9.9039e-1 (1.66e-3) $ - $ | 9.9366e-1 (7.60e-4) |
| 10 | 19 | 9.9048e-1 (4.84e-3) $ \approx $ | 9.9382e-1 (1.38e-3) $ + $ | 9.7457e-1 (6.38e-3) $ - $ | 9.8970e-1 (1.93e-3) $ - $ | 9.8425e-1 (3.78e-3) $ - $ | 9.8200e-1 (3.18e-3) $ - $ | 9.9221e-1 (9.41e-4) | |
| 15 | 24 | 9.9283e-1 (2.47e-3) $ - $ | 9.9575e-1 (9.90e-4) $ \approx $ | 9.7657e-1 (4.57e-3) $ - $ | 9.9191e-1 (2.00e-3) $ - $ | 9.8856e-1 (4.47e-3) $ - $ | 9.7026e-1 (1.01e-2) $ - $ | 9.9681e-1 (1.72e-3) | |
| WFG3 | 5 | 14 | 1.6003e-1 (1.76e-2) $ \approx $ | 2.1443e-1 (2.02e-2) $ + $ | 0.0000e+0 (0.00e+0) $ - $ | 6.2622e-2 (1.19e-2) $ - $ | 2.0853e-1 (1.16e-2) $ + $ | 1.5886e-1 (1.06e-2) $ \approx $ | 1.4825e-1 (3.29e-2) |
| 10 | 19 | 0.0000e+0 (0.00e+0) $ \approx $ | 3.7907e-3 (5.95e-3) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 3.5069e-2 (2.21e-2) $ + $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) | |
| 15 | 24 | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) | |
| WFG4 | 5 | 14 | 7.9689e-1 (1.75e-3) $ - $ | 7.8109e-1 (1.54e-3) $ - $ | 6.0579e-1 (1.99e-2) $ - $ | 6.8128e-1 (1.21e-2) $ - $ | 7.8801e-1 (3.28e-3) $ - $ | 7.9944e-1 (1.74e-3) $ - $ | 8.0615e-1 (1.22e-3) |
| 10 | 19 | 9.2669e-1 (4.49e-3) $ - $ | 9.4142e-1 (2.19e-3) $ - $ | 6.1073e-1 (2.34e-2) $ - $ | 7.9287e-1 (6.16e-3) $ - $ | 9.2540e-1 (3.47e-3) $ - $ | 9.2109e-1 (5.45e-3) $ - $ | 9.4963e-1 (2.90e-3) | |
| 15 | 24 | 9.8122e-1 (1.93e-3) $ \approx $ | 9.4296e-1 (3.86e-3) $ - $ | 4.7679e-1 (3.71e-2) $ - $ | 8.3205e-1 (1.21e-2) $ - $ | 9.7710e-1 (1.66e-3) $ - $ | 9.7070e-1 (4.84e-3) $ - $ | 9.8303e-1 (1.78e-3) | |
| WFG5 | 5 | 14 | 7.5959e-1 (5.56e-4) $ - $ | 7.3861e-1 (1.22e-3) $ - $ | 6.0160e-1 (1.92e-2) $ - $ | 6.5696e-1 (8.53e-3) $ - $ | 7.4441e-1 (3.15e-3) $ - $ | 7.5920e-1 (8.25e-4) $ - $ | 7.6022e-1 (4.13e-4) |
| 10 | 19 | 8.9239e-1 (1.01e-3) $ - $ | 8.8790e-1 (1.36e-3) $ - $ | 6.2494e-1 (2.15e-2) $ - $ | 7.5820e-1 (1.48e-2) $ - $ | 8.8046e-1 (3.92e-3) $ - $ | 8.8853e-1 (1.92e-3) $ - $ | 8.9832e-1 (6.00e-4) | |
| 15 | 24 | 9.1403e-1 (3.10e-4) $ - $ | 8.6622e-1 (5.15e-3) $ - $ | 5.3440e-1 (5.33e-2) $ - $ | 7.8067e-1 (1.10e-2) $ - $ | 9.0666e-1 (2.57e-3) $ - $ | 9.1592e-1 (2.40e-4) $ + $ | 9.1455e-1 (4.77e-4) | |
| WFG6 | 5 | 14 | 7.2850e-1 (8.39e-3) $ - $ | 7.2679e-1 (1.51e-2) $ - $ | 5.2228e-1 (1.93e-2) $ - $ | 5.7880e-1 (1.52e-2) $ - $ | 7.2883e-1 (1.18e-2) $ - $ | 7.3955e-1 (1.05e-2) $ \approx $ | 7.4043e-1 (1.41e-2) |
| 10 | 19 | 8.6206e-1 (1.81e-2) $ - $ | 8.5783e-1 (1.57e-2) $ - $ | 4.6979e-1 (2.58e-2) $ - $ | 6.4730e-1 (3.00e-2) $ - $ | 8.5984e-1 (1.60e-2) $ - $ | 8.4746e-1 (9.01e-3) $ - $ | 8.6998e-1 (1.36e-2) | |
| 15 | 24 | 8.8396e-1 (3.18e-2) $ - $ | 8.2787e-1 (3.14e-2) $ - $ | 3.0173e-1 (2.60e-2) $ - $ | 6.6774e-1 (3.40e-2) $ - $ | 8.8606e-1 (3.44e-2) $ \approx $ | 6.9644e-1 (9.03e-2) $ - $ | 8.8649e-1 (1.31e-2) | |
| WFG7 | 5 | 14 | 8.0157e-1 (1.42e-3) $ - $ | 7.9170e-1 (1.66e-3) $ - $ | 6.0729e-1 (3.24e-2) $ - $ | 6.3656e-1 (1.59e-2) $ - $ | 7.9105e-1 (2.35e-3) $ - $ | 8.0340e-1 (7.00e-4) $ - $ | 8.0758e-1 (2.98e-4) |
| 10 | 19 | 9.4331e-1 (3.60e-3) $ - $ | 9.5594e-1 (1.10e-3) $ - $ | 5.6192e-1 (3.11e-2) $ - $ | 7.8347e-1 (1.37e-2) $ - $ | 9.4214e-1 (1.66e-3) $ - $ | 9.3428e-1 (2.94e-3) $ - $ | 9.5879e-1 (1.82e-3) | |
| 15 | 24 | 9.7887e-1 (4.38e-3) $ - $ | 9.5899e-1 (3.22e-3) $ - $ | 4.4289e-1 (4.06e-2) $ - $ | 8.2889e-1 (2.07e-2) $ - $ | 9.7911e-1 (1.70e-3) $ - $ | 8.9832e-1 (1.20e-1) $ - $ | 9.8483e-1 (1.88e-3) | |
| WFG8 | 5 | 14 | 6.8616e-1 (3.90e-3) $ - $ | 6.5493e-1 (4.43e-3) $ - $ | 4.2548e-1 (2.67e-2) $ - $ | 5.4135e-1 (2.03e-2) $ - $ | 6.8318e-1 (3.65e-3) $ - $ | 6.9016e-1 (1.71e-3) $ - $ | 6.9502e-1 (8.48e-4) |
| 10 | 19 | 8.4110e-1 (8.57e-3) $ - $ | 8.5124e-1 (3.00e-2) $ \approx $ | 2.9935e-1 (6.35e-2) $ - $ | 5.1359e-1 (1.07e-1) $ - $ | 8.3601e-1 (1.18e-2) $ - $ | 7.1634e-1 (7.48e-2) $ - $ | 8.6059e-1 (5.74e-3) | |
| 15 | 24 | 9.2249e-1 (1.86e-2) $ \approx $ | 8.3888e-1 (2.18e-2) $ - $ | 3.1650e-1 (1.76e-1) $ - $ | 4.7231e-1 (1.04e-1) $ - $ | 9.0555e-1 (3.02e-3) $ - $ | 6.3933e-1 (7.66e-2) $ - $ | 9.1724e-1 (5.33e-3) | |
| WFG9 | 5 | 14 | 7.4953e-1 (5.75e-3) $ - $ | 7.5078e-1 (3.47e-3) $ - $ | 6.1318e-1 (9.20e-3) $ - $ | 6.4374e-1 (9.46e-3) $ - $ | 7.4210e-1 (3.51e-3) $ - $ | 7.6033e-1 (2.41e-3) $ \approx $ | 7.6045e-1 (3.47e-3) |
| 10 | 19 | 8.4481e-1 (5.63e-2) $ \approx $ | 8.8742e-1 (5.34e-3) $ + $ | 6.7655e-1 (2.62e-2) $ - $ | 7.2650e-1 (5.38e-2) $ - $ | 8.5557e-1 (1.25e-2) $ - $ | 8.5015e-1 (2.13e-2) $ - $ | 8.7926e-1 (6.82e-3) | |
| 15 | 24 | 8.7216e-1 (7.24e-2) $ + $ | 8.6629e-1 (1.39e-2) $ \approx $ | 5.9362e-1 (3.55e-2) $ - $ | 6.9505e-1 (6.75e-2) $ - $ | 8.9247e-1 (7.23e-3) $ + $ | 8.0237e-1 (6.23e-2) $ - $ | 8.6638e-1 (1.92e-2) | |
| $ +/-/\approx $ | 1/19/7 | 3/19/5 | 0/25/2 | 0/25/2 | 3/22/2 | 1/20/6 | |||
DownLoad:
CSV
| SDEA vs | IGD | HV | |||||
| $ R^{+} $ | $ R^{-} $ | $ p $-value | $ R^{+} $ | $ R^{-} $ | $ p $-value | ||
| RVEA | 313 | 65.0 | 0.00278 | 327.5 | 50.5 | 0.000381 | |
| MOEA/D-AWA | 308.0 | 70.0 | 0.00259 | 330.5 | 47.5 | 0.000512 | |
| RPS-NSGA-II | 311.0 | 67.0 | 0.002105 | 376 | 2 | 0.000058 | |
| MORA/D-UR | 378.0 | 0.0 | 0.000005 | 376 | 2 | 0.000058 | |
| MaOEA-PDS | 378.0 | 0.0 | 0.000005 | 301.5 | 76.5 | 0.004471 | |
| HEA | 320.0 | 58.0 | 0.001021 | 351.5 | 26.5 | 0.000074 | |
DownLoad:
CSV
| Problem | $ m $ | $ D $ | RVEA | MOEA/D-AWA | RPS-NSGA-II | MOEA/D-UR | MaOEA-PDS | HEA | SDEA |
| MaF1 | 5 | 14 | 1.8093e-1 (9.11e-3) $ - $ | 2.7675e-1 (1.65e-2) $ - $ | 1.9100e-1 (8.70e-3) $ - $ | 1.0217e-1 (7.92e-4) $ + $ | 3.0799e-1 (1.99e-2) $ - $ | 1.5632e-1 (2.59e-3) $ - $ | 1.0332e-1 (5.15e-4) |
| 10 | 19 | 2.8503e-1 (1.00e-2) $ - $ | 5.7593e-1 (5.61e-2) $ - $ | 2.8201e-1 (9.95e-3) $ - $ | 2.0642e-1 (8.54e-4) $ + $ | 3.4548e-1 (1.93e-2) $ - $ | 3.3494e-1 (2.12e-2) $ - $ | 2.1234e-1 (1.28e-3) | |
| 15 | 24 | 3.3595e-1 (4.33e-3) $ - $ | 7.5949e-1 (1.01e-1) $ - $ | 3.8999e-1 (1.11e-1) $ - $ | 3.0240e-1 (2.34e-2) $ \approx $ | 4.2697e-1 (1.92e-2) $ - $ | 4.0789e-1 (8.02e-3) $ - $ | 2.9088e-1 (3.59e-3) | |
| MaF2 | 5 | 14 | 1.1175e-1 (2.75e-3) $ - $ | 1.1591e-1 (1.02e-3) $ - $ | 1.4819e-1 (3.49e-3) $ - $ | 8.0600e-2 (1.10e-3) $ + $ | 1.2648e-1 (2.17e-3) $ - $ | 1.1666e-1 (7.76e-4) $ - $ | 9.3169e-2 (1.42e-3) |
| 10 | 19 | 2.0786e-1 (1.27e-2) $ - $ | 2.5117e-1 (1.81e-2) $ - $ | 2.4176e-1 (3.76e-3) $ - $ | 1.9495e-1 (7.87e-3) $ - $ | 1.8387e-1 (1.94e-3) $ - $ | 1.7124e-1 (3.52e-3) $ - $ | 1.6260e-1 (2.20e-3) | |
| 15 | 24 | 2.5667e-1 (2.82e-2) $ - $ | 7.0357e-1 (1.09e-1) $ - $ | 2.1510e-1 (3.51e-3) $ \approx $ | 3.0015e-1 (2.46e-2) $ - $ | 2.1522e-1 (3.18e-3) $ \approx $ | 2.1796e-1 (7.13e-3) $ - $ | 1.9880e-1 (2.52e-3) | |
| MaF3 | 5 | 14 | 7.7658e-2 (5.48e-3) $ + $ | 2.0514e-1 (1.28e-1) $ - $ | 1.4994e-1 (2.37e-2) $ - $ | 2.2080e-1 (1.66e-1) $ - $ | 1.0687e-1 (2.28e-2) $ \approx $ | 1.2120e-1 (3.98e-2) $ - $ | 9.4125e-2 (5.81e-3) |
| 10 | 19 | 1.8372e+2 (2.28e+2) $ - $ | 8.9908e-1 (5.78e-1) $ - $ | 3.9731e-1 (1.94e-1) $ - $ | 1.5877e+1 (9.42e+0) $ - $ | 1.2335e-1 (8.51e-3) $ \approx $ | 2.8923e-1 (6.64e-2) $ - $ | 2.0168e-1 (2.46e-1) | |
| 15 | 24 | 4.5781e+1 (1.08e+2) $ - $ | 1.0100e-1 (4.80e-3) $ + $ | 2.3712e-1 (2.12e-1) $ - $ | 1.5856e-1 (7.32e-2) $ - $ | 1.8609e-1 (2.83e-2) $ - $ | 7.1590e+5 (9.20e+5) $ - $ | 1.0980e-1 (4.30e-3) | |
| MaF4 | 5 | 14 | 2.8205e+0 (7.22e-1) $ - $ | 3.5731e+0 (8.15e-1) $ - $ | 3.1007e+0 (4.89e-1) $ - $ | 3.1097e+0 (5.18e-1) $ - $ | 2.8769e+0 (9.80e-2) $ - $ | 2.8891e+0 (2.62e-1) $ - $ | 2.0313e+0 (7.54e-2) |
| 10 | 19 | 9.7435e+1 (5.35e+0) $ - $ | 1.8558e+2 (2.52e+1) $ - $ | 1.5111e+2 (2.27e+1) $ - $ | 1.2357e+2 (1.86e+1) $ - $ | 8.6259e+1 (7.06e+0) $ - $ | 1.1181e+2 (9.77e+0) $ - $ | 5.3091e+1 (2.05e+0) | |
| 15 | 24 | 4.3936e+3 (2.26e+2) $ - $ | 7.8711e+3 (2.62e+3) $ - $ | 1.5273e+4 (8.13e+3) $ - $ | 8.8258e+3 (2.68e+3) $ - $ | 4.9481e+3 (4.93e+2) $ - $ | 3.6663e+3 (3.62e+2) $ - $ | 2.2801e+3 (1.82e+2) | |
| MaF5 | 5 | 14 | 1.9704e+0 (2.65e-3) $ - $ | 1.9719e+0 (2.23e-3) $ - $ | 1.9269e+0 (2.30e-2) $ - $ | 3.7179e+0 (4.57e-1) $ - $ | 2.2132e+0 (4.51e-2) $ - $ | 1.9609e+0 (9.42e-3) $ - $ | 1.8264e+0 (2.57e-2) |
| 10 | 19 | 7.7358e+1 (6.82e-1) $ - $ | 9.4917e+1 (8.78e+0) $ - $ | 5.6693e+1 (4.40e+0) $ \approx $ | 2.5553e+2 (2.13e+1) $ - $ | 8.1962e+1 (1.38e+0) $ - $ | 6.8867e+1 (2.53e+0) $ - $ | 5.2805e+1 (2.15e+0) | |
| 15 | 24 | 2.9732e+3 (3.91e+2) $ \approx $ | 3.4405e+3 (3.17e+2) $ - $ | 7.1810e+3 (2.06e+2) $ - $ | 3.3465e+3 (6.39e+2) $ \approx $ | 2.7237e+3 (4.67e+2) $ \approx $ | 2.7973e+3 (3.82e+2) $ - $ | 1.9311e+3 (9.37e+1) | |
| MaF6 | 5 | 14 | 1.7091e-2 (4.80e-3) $ - $ | 6.5710e-2 (8.06e-3) $ - $ | 3.7627e-1 (1.75e-1) $ - $ | 1.5278e-1 (2.57e-2) $ - $ | 1.4969e-1 (3.67e-2) $ - $ | 9.3489e-2 (1.20e-2) $ - $ | 2.1829e-3 (1.22e-4) |
| 10 | 19 | 5.1392e-1 (1.37e-1) $ - $ | 9.6871e-2 (2.37e-2) $ + $ | 2.6096e-1 (8.98e-2) $ \approx $ | 7.5239e-1 (2.55e-1) $ - $ | 2.6798e+0 (5.12e+0) $ - $ | 1.6405e-1 (5.96e-2) $ + $ | 3.1961e-1 (6.90e-2) | |
| 15 | 24 | 3.1400e-1 (7.06e-2) $ \approx $ | 1.7163e-1 (6.01e-7) $ + $ | 8.6088e+0 (9.50e+0) $ - $ | 1.5886e-1 (3.06e-2) $ + $ | 1.2430e+1 (3.08e+1) $ - $ | 3.8032e-1 (1.63e-1) $ \approx $ | 3.1966e-1 (4.75e-2) | |
| MaF7 | 5 | 24 | 2.8163e-1 (8.99e-3) $ - $ | 7.8828e-1 (1.03e-1) $ - $ | 3.1363e-1 (7.14e-3) $ - $ | 3.0032e-1 (2.29e-2) $ - $ | 3.7629e-1 (5.08e-2) $ - $ | 3.0510e-1 (3.46e-3) $ - $ | 2.3753e-1 (3.46e-3) |
| 10 | 29 | 1.1300e+0 (1.44e-1) $ \approx $ | 3.0577e+0 (1.63e+0) $ - $ | 2.0908e+0 (5.39e-1) $ - $ | 2.0597e+0 (4.40e-1) $ - $ | 1.3901e+0 (1.29e-2) $ - $ | 1.6103e+0 (9.52e-2) $ - $ | 1.0949e+0 (1.03e-1) | |
| 15 | 34 | 7.6126e+0 (1.26e+0) $ - $ | 4.1200e+0 (1.36e+0) $ - $ | 6.6959e+0 (1.04e+0) $ - $ | 5.1948e+0 (4.46e-1) $ - $ | 6.7687e+0 (1.79e-1) $ - $ | 8.2024e+0 (3.78e-1) $ - $ | 2.4257e+0 (3.02e-1) | |
| MaF8 | 5 | 2 | 1.5616e-1 (1.13e-2) $ - $ | 3.3402e-1 (2.84e-2) $ - $ | 3.0683e-1 (3.86e-2) $ - $ | 8.3643e-2 (1.62e-3) $ \approx $ | 2.2278e-1 (4.07e-2) $ - $ | 1.4864e-1 (6.43e-3) $ - $ | 8.2350e-2 (6.26e-3) |
| 10 | 2 | 3.5278e-1 (1.01e-1) $ - $ | 9.9749e-1 (1.29e-1) $ - $ | 6.9682e-1 (8.72e-2) $ - $ | 1.2152e-1 (3.27e-3) $ \approx $ | 2.9131e-1 (7.31e-2) $ - $ | 2.9153e-1 (1.09e-1) $ - $ | 1.2209e-1 (4.52e-3) | |
| 15 | 2 | 4.1765e-1 (3.93e-2) $ - $ | 1.3049e+0 (2.48e-1) $ - $ | 9.9097e-1 (1.58e-1) $ - $ | 2.1885e-1 (9.41e-3) $ + $ | 5.1606e-1 (6.52e-2) $ - $ | 4.5377e-1 (2.97e-2) $ - $ | 2.7222e-1 (3.96e-2) | |
| MaF9 | 5 | 2 | 3.2052e-1 (6.83e-2) $ - $ | 2.8137e-1 (3.16e-2) $ - $ | 2.3289e-1 (4.14e-2) $ - $ | 1.5154e-1 (4.65e-3) $ - $ | 1.5336e-1 (1.93e-2) $ - $ | 1.8296e-1 (3.19e-2) $ - $ | 7.5546e-2 (2.24e-3) |
| 10 | 2 | 5.3758e-1 (1.52e-1) $ - $ | 7.7406e-1 (2.37e-1) $ - $ | 4.3598e-1 (9.37e-2) $ - $ | 2.2451e-1 (6.70e-2) $ - $ | 4.8400e-1 (1.10e-1) $ - $ | 3.4440e-1 (4.80e-2) $ - $ | 1.2820e-1 (8.32e-3) | |
| 15 | 2 | 3.6908e+0 (5.49e+0) $ - $ | 1.8150e+0 (3.80e-1) $ - $ | 7.8709e-1 (1.69e-1) $ - $ | 3.3958e-1 (8.91e-2) $ - $ | 7.3567e+0 (5.42e+0) $ - $ | 3.7792e+0 (5.43e+0) $ - $ | 1.8485e-1 (4.08e-3) | |
| MaF10 | 5 | 14 | 4.9673e-1 (2.12e-2) $ - $ | 3.7616e-1 (1.17e-2) $ - $ | 4.0004e-1 (1.44e-2) $ - $ | 7.3363e-1 (5.32e-2) $ - $ | 4.4329e-1 (9.09e-3) $ - $ | 3.7721e-1 (9.23e-3) $ - $ | 3.6411e-1 (4.85e-3) |
| 10 | 19 | 1.4464e+0 (1.32e-1) $ - $ | 1.0918e+0 (3.13e-2) $ - $ | 1.0352e+0 (3.16e-2) $ - $ | 1.5495e+0 (4.82e-2) $ - $ | 1.6057e+0 (2.35e-1) $ - $ | 9.5575e-1 (3.25e-2) $ \approx $ | 9.6470e-1 (1.40e-2) | |
| 15 | 24 | 1.7889e+0 (1.15e-1) $ - $ | 1.7714e+0 (3.23e-2) $ - $ | 1.9862e+0 (8.38e-2) $ - $ | 2.2074e+0 (5.11e-2) $ - $ | 2.8502e+0 (7.26e-1) $ - $ | 1.6574e+0 (6.36e-2) $ \approx $ | 1.6561e+0 (2.52e-2) | |
| $ +/-/\approx $ | 1/26/3 | 3/27/0 | 0/27/3 | 5/21/4 | 0/26/4 | 1/26/3 | |||
DownLoad:
CSV
| Problem | $ m $ | $ D $ | RVEA | MOEA/D-AWA | RPS-NSGA-II | MOEA/D-UR | MaOEA-PDS | HEA | SDEA |
| MaF1 | 5 | 14 | 6.5160e-3 (3.49e-4) $ - $ | 3.3381e-3 (2.76e-4) $ - $ | 7.2824e-3 (3.71e-4) $ - $ | 1.2931e-2 (6.69e-5) $ - $ | 2.4605e-3 (3.33e-4) $ - $ | 8.5443e-3 (1.78e-4) $ - $ | 1.3086e-2 (1.73e-4) |
| 10 | 19 | 4.6265e-7 (3.10e-8) $ \approx $ | 9.6417e-9 (3.87e-9) $ - $ | 5.4721e-7 (4.89e-8) $ \approx $ | 2.1973e-7 (3.51e-7) $ - $ | 2.4711e-7 (9.06e-8) $ - $ | 6.7236e-7 (3.24e-7) $ - $ | 8.9684e-7 (9.44e-8) | |
| 15 | 24 | 2.8020e-12 (2.45e-13) $ + $ | 2.3083e-14 (1.51e-14) $ + $ | 3.3293e-12 (1.68e-12) $ + $ | 2.8918e-11 (7.54e-11) $ \approx $ | 4.2472e-13 (2.14e-13) $ + $ | 1.2032e-12 (1.92e-13) $ + $ | 0.0000e+0 (0.00e+0) | |
| MaF2 | 5 | 14 | 1.8617e-1 (1.96e-3) $ - $ | 1.7705e-1 (1.71e-3) $ - $ | 1.8712e-1 (3.16e-3) $ - $ | 1.8890e-1 (2.04e-3) $ - $ | 1.7784e-1 (2.41e-3) $ - $ | 1.7737e-1 (1.39e-3) $ - $ | 2.0025e-1 (1.94e-3) |
| 10 | 19 | 2.1840e-1 (3.84e-3) $ - $ | 1.7054e-1 (2.67e-3) $ - $ | 2.0546e-1 (4.52e-3) $ - $ | 2.0151e-1 (5.21e-3) $ - $ | 1.9617e-1 (2.69e-3) $ - $ | 2.0335e-1 (1.82e-3) $ - $ | 2.2616e-1 (2.74e-3) | |
| 15 | 24 | 1.6970e-1 (9.83e-3) $ - $ | 8.8735e-2 (2.55e-2) $ - $ | 1.6401e-1 (4.65e-3) $ - $ | 1.6881e-1 (3.69e-3) $ - $ | 1.5509e-1 (3.46e-3) $ - $ | 1.5267e-1 (2.55e-3) $ - $ | 2.0144e-1 (3.56e-3) | |
| MaF3 | 5 | 14 | 9.9871e-1 (6.32e-4) $ \approx $ | 9.2138e-1 (1.60e-1) $ - $ | 9.7611e-1 (1.31e-2) $ - $ | 8.3051e-1 (2.34e-1) $ - $ | 9.9660e-1 (1.86e-3) $ - $ | 9.9315e-1 (7.94e-3) $ - $ | 9.9834e-1 (3.46e-4) |
| 10 | 19 | 1.4286e-1 (3.78e-1) $ - $ | 3.2615e-1 (4.53e-1) $ - $ | 7.0304e-1 (2.30e-1) $ - $ | 0.0000e+0 (0.00e+0) $ - $ | 9.9806e-1 (1.36e-3) $ \approx $ | 8.4629e-1 (1.01e-1) $ - $ | 9.0368e-1 (2.54e-1) | |
| 15 | 24 | 4.7947e-1 (4.64e-1) $ - $ | 9.9884e-1 (6.82e-4) $ \approx $ | 8.7677e-1 (2.48e-1) $ - $ | 9.7772e-1 (5.51e-2) $ \approx $ | 9.5232e-1 (3.53e-2) $ - $ | 4.7688e-1 (4.54e-1) $ - $ | 9.9932e-1 (1.02e-3) | |
| MaF4 | 5 | 14 | 7.0518e-2 (1.00e-2) $ - $ | 3.1046e-2 (1.42e-2) $ - $ | 1.0284e-1 (4.64e-3) $ - $ | 7.9981e-2 (1.28e-2) $ - $ | 8.4317e-2 (5.99e-3) $ - $ | 7.2699e-2 (2.38e-3) $ - $ | 1.3276e-1 (1.86e-3) |
| 10 | 19 | 2.1562e-4 (1.72e-5) $ - $ | 6.8904e-8 (1.59e-8) $ - $ | 4.0569e-5 (2.07e-5) $ - $ | 2.9345e-6 (2.36e-6) $ - $ | 2.0362e-4 (2.78e-5) $ - $ | 1.5428e-4 (2.68e-5) $ - $ | 3.6693e-4 (2.87e-5) | |
| 15 | 24 | 2.1729e-7 (3.19e-8) $ + $ | 2.0133e-12 (4.88e-12) $ - $ | 8.7797e-9 (1.37e-8) $ - $ | 1.1794e-11 (1.86e-11) $ - $ | 6.5427e-8 (1.42e-8) $ - $ | 2.7130e-8 (3.86e-8) $ - $ | 1.0839e-7 (3.45e-8) | |
| MaF5 | 5 | 14 | 8.1195e-1 (3.14e-4) $ + $ | 8.1213e-1 (3.04e-4) $ + $ | 7.9326e-1 (1.88e-3) $ - $ | 6.3853e-1 (1.69e-2) $ - $ | 7.9520e-1 (1.80e-3) $ - $ | 8.0824e-1 (1.59e-3) $ \approx $ | 8.0826e-1 (8.30e-4) |
| 10 | 19 | 9.6929e-1 (1.69e-4) $ - $ | 9.4729e-1 (2.30e-3) $ - $ | 9.5845e-1 (2.82e-3) $ - $ | 6.8320e-1 (3.73e-2) $ - $ | 9.5502e-1 (2.18e-3) $ - $ | 9.7005e-1 (1.56e-3) $ - $ | 9.7215e-1 (4.14e-4) | |
| 15 | 24 | 9.8923e-1 (3.61e-3) $ + $ | 9.1661e-1 (2.82e-2) $ - $ | 9.6073e-1 (4.97e-3) $ - $ | 5.6169e-1 (7.22e-2) $ - $ | 9.7546e-1 (6.23e-3) $ - $ | 9.8372e-1 (2.64e-3) $ - $ | 9.8844e-1 (1.64e-3) | |
| MaF6 | 5 | 14 | 1.2347e-1 (1.42e-3) $ - $ | 1.1722e-1 (2.29e-3) $ - $ | 9.3332e-2 (2.73e-2) $ - $ | 9.8929e-3 (5.99e-3) $ - $ | 1.0144e-1 (7.60e-3) $ - $ | 1.1860e-1 (4.22e-3) $ - $ | 1.2996e-1 (2.56e-4) |
| 10 | 19 | 6.5015e-4 (1.72e-3) $ - $ | 9.4780e-2 (6.13e-4) $ + $ | 6.3933e-2 (2.35e-2) $ \approx $ | 0.0000e+0 (0.00e+0) $ - $ | 1.3009e-13 (2.91e-13) $ - $ | 9.6725e-2 (1.66e-3) $ + $ | 3.8061e-2 (3.31e-2) | |
| 15 | 24 | 5.4888e-2 (2.95e-2) $ - $ | 9.1740e-2 (2.74e-4) $ + $ | 0.0000e+0 (0.00e+0) $ - $ | 5.8975e-4 (1.04e-3) $ - $ | 3.8785e-3 (6.00e-3) $ - $ | 8.9102e-2 (2.03e-3) $ + $ | 7.5767e-2 (7.28e-3) | |
| MaF7 | 5 | 24 | 2.5925e-1 (3.34e-3) $ - $ | 1.0563e-1 (1.96e-2) $ - $ | 2.5670e-1 (4.79e-3) $ - $ | 1.7534e-1 (1.92e-2) $ - $ | 2.4798e-1 (4.14e-3) $ - $ | 2.5353e-1 (2.11e-3) $ - $ | 2.7150e-1 (1.58e-3) |
| 10 | 29 | 1.7289e-1 (7.15e-3) $ - $ | 2.0858e-4 (2.64e-4) $ - $ | 1.8403e-1 (3.88e-3) $ \approx $ | 2.9045e-4 (1.31e-4) $ - $ | 1.7472e-1 (3.20e-3) $ - $ | 1.7483e-1 (7.17e-3) $ - $ | 1.8090e-1 (4.24e-3) | |
| 15 | 34 | 1.3662e-1 (2.01e-2) $ \approx $ | 1.5746e-5 (2.06e-5) $ - $ | 1.5022e-1 (7.15e-3) $ \approx $ | 0.0000e+0 (0.00e+0) $ - $ | 1.2905e-1 (1.31e-2) $ - $ | 1.4487e-1 (6.11e-3) $ \approx $ | 1.4421e-1 (2.17e-3) | |
| MaF8 | 5 | 2 | 1.1023e-1 (2.19e-3) $ - $ | 7.5719e-2 (3.12e-3) $ - $ | 8.6853e-2 (4.74e-3) $ - $ | 1.2532e-1 (3.04e-4) $ - $ | 1.0468e-1 (3.16e-3) $ - $ | 1.1021e-1 (1.30e-3) $ - $ | 1.2574e-1 (2.10e-4) |
| 10 | 2 | 9.1903e-3 (3.22e-4) $ - $ | 3.3919e-3 (7.76e-4) $ - $ | 5.0007e-3 (9.34e-4) $ - $ | 1.0950e-2 (5.36e-5) $ - $ | 8.0310e-3 (9.54e-4) $ - $ | 8.3357e-3 (2.99e-4) $ - $ | 1.1111e-2 (3.84e-5) | |
| 15 | 2 | 3.6538e-4 (3.32e-5) $ - $ | 1.3700e-4 (5.54e-5) $ - $ | 2.0735e-4 (3.08e-5) $ - $ | 5.7711e-4 (1.10e-5) $ - $ | 2.1292e-4 (4.32e-5) $ - $ | 1.6319e-4 (6.33e-5) $ - $ | 5.9484e-4 (1.15e-5) | |
| MaF9 | 5 | 2 | 2.2823e-1 (2.05e-2) $ - $ | 2.2671e-1 (1.30e-2) $ - $ | 2.4771e-1 (1.51e-2) $ - $ | 2.7906e-1 (3.33e-3) $ - $ | 2.9014e-1 (8.68e-3) $ - $ | 2.7870e-1 (1.23e-2) $ - $ | 3.1991e-1 (2.13e-3) |
| 10 | 2 | 9.4947e-3 (1.80e-3) $ - $ | 5.8386e-3 (1.68e-3) $ - $ | 1.0978e-2 (1.56e-3) $ - $ | 1.4818e-2 (1.35e-3) $ - $ | 7.6859e-3 (2.37e-3) $ - $ | 1.0519e-2 (1.43e-3) $ - $ | 1.7500e-2 (3.84e-4) | |
| 15 | 2 | 4.4083e-4 (3.29e-4) $ - $ | 1.1750e-4 (6.45e-5) $ - $ | 3.0851e-4 (7.06e-5) $ - $ | 8.6941e-4 (1.13e-4) $ - $ | 8.2382e-5 (2.01e-4) $ - $ | 3.1479e-4 (2.46e-4) $ - $ | 1.1605e-3 (2.45e-5) | |
| MaF10 | 5 | 14 | 8.7077e-1 (2.14e-2) $ - $ | 9.9743e-1 (8.94e-4) $ + $ | 9.8892e-1 (2.84e-3) $ - $ | 8.6314e-1 (2.11e-2) $ - $ | 9.9311e-1 (4.81e-3) $ - $ | 9.9815e-1 (4.06e-4) $ + $ | 9.9680e-1 (6.63e-4) |
| 10 | 19 | 6.7584e-1 (5.61e-2) $ - $ | 9.9732e-1 (9.19e-4) $ - $ | 9.8156e-1 (3.23e-2) $ - $ | 8.9473e-1 (5.33e-2) $ - $ | 9.8488e-1 (5.26e-3) $ - $ | 9.9958e-1 (3.37e-4) $ \approx $ | 9.9962e-1 (2.54e-4) | |
| 15 | 24 | 9.9936e-1 (4.69e-4) $ - $ | 9.9716e-1 (3.23e-4) $ - $ | 9.9931e-1 (6.81e-4) $ - $ | 9.9600e-1 (8.78e-4) $ - $ | 9.6931e-1 (5.50e-2) $ - $ | 9.9999e-1 (7.04e-6) $ + $ | 9.9996e-1 (7.72e-6) | |
| $ +/-/\approx $ | 4/23/3 | 5/24/1 | 1/25/4 | 0/28/2 | 1/28/1 | 5/22/3 | |||
DownLoad:
CSV
| SDEA vs | IGD | HV | |||||
| $ R^{+} $ | $ R^{-} $ | $ p $-value | $ R^{+} $ | $ R^{-} $ | $ p $-value | ||
| RVEA | 461.0 | 4.0 | 0.000002 | 434.0 | 31.0 | 0.000026 | |
| MOEA/D-AWA | 441.0 | 24.0 | 0.000017 | 412.5 | 52.5 | 0.000198 | |
| RPS-NSGA-II | 430.0 | 35.0 | 0.000047 | 425.0 | 40.0 | 0.000069 | |
| MORA/D-UR | 402.0 | 33.0 | 0.000063 | 461.0 | 4.0 | 0.000002 | |
| MaOEA-PDS | 456.5 | 8.5 | 0.000004 | 434.0 | 31.0 | 0.000031 | |
| HEA | 384.0 | 51.0 | 0.000305 | 365.5 | 69.5 | 0.001283 | |
DownLoad:
CSV
| Problem | $ m $ | $ D $ | RVEA | MOEA/D-AWA | RPS-NSGA-II | MOEA/D-UR | MaOEA-PDS | HEA | SDEA |
| IMOP1 | 2 | 10 | 1.3871e-1 (6.02e-3) $ - $ | 1.0191e-1 (5.20e-3) $ \approx $ | 1.3700e-1 (3.71e-2) $ - $ | 3.7121e-1 (4.93e-2) $ - $ | 2.5130e-1 (2.32e-2) $ - $ | 3.5259e-1 (1.02e-2) $ - $ | 9.0222e-2 (1.27e-2) |
| IMOP2 | 2 | 10 | 7.3070e-1 (1.44e-1) $ - $ | 1.2196e-2 (4.62e-3) $ - $ | 5.3323e-1 (3.55e-1) $ - $ | 5.2281e-1 (8.74e-2) $ - $ | 7.8495e-1 (5.80e-5) $ - $ | 1.5455e-1 (2.23e-2) $ - $ | 4.6425e-3 (7.53e-5) |
| IMOP3 | 2 | 10 | 8.1834e-3 (1.42e-3) $ - $ | 6.2844e-3 (9.36e-4) $ \approx $ | 8.5002e-3 (6.34e-4) $ - $ | 4.7468e-1 (1.05e-1) $ - $ | 1.2144e-2 (1.60e-3) $ - $ | 4.2638e-2 (1.51e-2) $ - $ | 5.3531e-3 (5.38e-4) |
| IMOP4 | 3 | 10 | 4.4279e-2 (6.47e-3) $ - $ | 1.6830e-2 (1.43e-3) $ - $ | 6.5909e-2 (1.91e-2) $ - $ | 2.8222e-2 (5.84e-3) $ - $ | 9.7195e-3 (1.11e-3) $ - $ | 3.6545e-2 (5.73e-3) $ - $ | 7.4608e-3 (3.43e-4) |
| IMOP5 | 3 | 10 | 6.6704e-2 (2.13e-3) $ - $ | 4.8526e-2 (1.75e-3) $ - $ | 6.8496e-2 (7.75e-3) $ - $ | 5.1696e-2 (4.83e-3) $ - $ | 3.6481e-2 (1.62e-3) $ - $ | 6.2797e-2 (3.36e-3) $ - $ | 3.3979e-2 (1.08e-3) |
| IMOP6 | 3 | 10 | 1.9018e-1 (2.16e-1) $ - $ | 1.1559e-1 (1.91e-1) $ - $ | 5.6896e-2 (6.64e-3) $ - $ | 4.7488e-2 (2.04e-3) $ - $ | 4.7660e-1 (1.94e-1) $ - $ | 1.4029e-1 (1.82e-1) $ - $ | 3.2074e-2 (7.17e-4) |
| IMOP7 | 3 | 10 | 8.2949e-1 (2.87e-1) $ - $ | 6.4672e-1 (4.12e-1) $ - $ | 5.8980e-2 (2.87e-3) $ - $ | 1.3336e-1 (2.34e-1) $ - $ | 9.3555e-1 (1.67e-3) $ - $ | 9.0978e-1 (2.72e-2) $ - $ | 3.5658e-2 (6.98e-4) |
| IMOP8 | 3 | 10 | 1.3487e-1 (2.99e-3) $ - $ | 1.0245e-1 (1.11e-3) $ - $ | 9.8327e-2 (3.51e-3) $ - $ | 8.7488e-2 (1.62e-3) $ - $ | 1.3496e-1 (1.47e-2) $ - $ | 1.9142e-1 (1.53e-1) $ - $ | 7.5254e-2 (2.42e-3) |
| $ +/-/\approx $ | 0/8/0 | 0/6/2 | 0/8/0 | 0/8/0 | 0/8/0 | 0/8/0 | |||
DownLoad:
CSV
| Problem | $ m $ | $ D $ | RVEA | MOEA/D-AWA | RPS-NSGA-II | MOEA/D-UR | MaOEA-PDS | HEA | SDEA |
| IMOP1 | 2 | 10 | 9.8505e-1 (1.03e-4) $ \approx $ | 9.8642e-1 (8.82e-5) $ + $ | 9.8406e-1 (9.09e-4) $ - $ | 6.6970e-1 (1.08e-1) $ - $ | 9.8193e-1 (1.43e-3) $ - $ | 9.6919e-1 (2.54e-3) $ - $ | 9.8550e-1 (9.05e-4) |
| IMOP2 | 2 | 10 | 2.3134e-1 (6.03e-5) $ - $ | 1.0160e-1 (2.83e-2) $ - $ | 1.3071e-1 (6.80e-2) $ - $ | 9.3322e-2 (2.30e-3) $ - $ | 9.0909e-2 (9.62e-9) $ - $ | 2.1301e-1 (5.95e-3) $ - $ | 2.3168e-1 (3.22e-5) |
| IMOP3 | 2 | 10 | 6.5536e-1 (3.17e-4) $ - $ | 6.5257e-1 (8.66e-4) $ - $ | 6.5257e-1 (7.38e-4) $ - $ | 2.7852e-1 (6.65e-2) $ - $ | 6.5573e-1 (3.54e-3) $ \approx $ | 6.4070e-1 (1.50e-2) $ - $ | 6.5745e-1 (2.15e-4) |
| IMOP4 | 3 | 10 | 4.2570e-1 (2.64e-3) $ - $ | 4.1148e-1 (2.27e-3) $ - $ | 4.0835e-1 (3.48e-3) $ - $ | 4.2179e-1 (2.96e-3) $ - $ | 4.3319e-1 (5.31e-4) $ - $ | 4.1798e-1 (2.77e-3) $ - $ | 4.3436e-1 (1.79e-4) |
| IMOP5 | 3 | 10 | 4.6123e-1 (5.43e-3) $ - $ | 4.3088e-1 (1.08e-2) $ - $ | 4.7988e-1 (1.88e-2) $ \approx $ | 5.0180e-1 (2.24e-2) $ + $ | 4.9271e-1 (1.40e-2) $ + $ | 4.7985e-1 (1.05e-2) $ \approx $ | 4.7432e-1 (6.32e-3) |
| IMOP6 | 3 | 10 | 4.8716e-1 (6.81e-2) $ - $ | 3.8249e-1 (1.87e-1) $ - $ | 5.0737e-1 (3.38e-3) $ - $ | 5.1478e-1 (2.26e-3) $ - $ | 1.3785e-1 (1.68e-1) $ - $ | 4.6991e-1 (6.48e-2) $ - $ | 5.2810e-1 (4.61e-4) |
| IMOP7 | 3 | 10 | 2.1513e-1 (2.05e-1) $ - $ | 1.3956e-1 (1.29e-1) $ - $ | 4.9775e-1 (3.87e-3) $ - $ | 4.5807e-1 (1.41e-1) $ - $ | 9.0920e-2 (1.23e-5) $ - $ | 9.3341e-2 (2.27e-3) $ - $ | 5.2769e-1 (5.24e-4) |
| IMOP8 | 3 | 10 | 4.8617e-1 (1.65e-3) $ - $ | 4.6607e-1 (7.19e-3) $ - $ | 5.3480e-1 (3.13e-2) $ \approx $ | 5.4614e-1 (2.99e-2) $ \approx $ | 4.8785e-1 (1.18e-2) $ - $ | 4.5786e-1 (3.35e-2) $ - $ | 5.5282e-1 (3.41e-2) |
| $ +/-/\approx $ | 0/7/1 | 1/7/0 | 0/6/2 | 1/6/1 | 1/6/1 | 0/7/1 | |||
DownLoad:
CSV
| Problem | $ m $ | $ D $ | SDEA1 | SDEA2 | SDEA3 | SDEA4 | SDEA5 | SDEA |
| WFG1 | 5 | 14 | 7.2318e-1 (6.27e-3) $ - $ | 9.4742e-1 (6.19e-3) $ - $ | 9.3968e-1 (6.35e-4) $ - $ | 6.7091e-1 (4.29e-3) $ - $ | 8.9150e-1 (4.27e-4) $ - $ | 9.9801e-1 (3.79e-4) |
| 10 | 19 | 8.9180e-1 (2.25e-4) $ - $ | 9.9752e-1 (3.69e-3) $ \approx $ | 9.5308e-1 (1.86e-3) $ - $ | 8.5984e-1 (5.27e-3) $ - $ | 6.9721e-1 (4.81e-4) $ - $ | 9.9963e-1 (3.80e-4) | |
| 15 | 24 | 8.7286e-1 (5.95e-3) $ - $ | 9.8711e-1 (3.02e-2) $ - $ | 9.2462e-1 (6.65e-4) $ \approx $ | 8.3671e-1 (4.22e-3) $ - $ | 9.1062e-1 (3.71e-4) $ - $ | 9.9998e-1 (2.41e-5) | |
| WFG2 | 5 | 14 | 9.9527e-1 (5.95e-4) $ \approx $ | 9.5968e-1 (2.08e-3) $ - $ | 9.9052e-1 (2.71e-3) $ \approx $ | 8.5132e-1 (2.94e-3) $ - $ | 7.0914e-1 (5.22e-4) $ - $ | 9.9366e-1 (7.60e-4) |
| 10 | 19 | 9.2610e-1 (2.83e-3) $ - $ | 9.9485e-1 (1.88e-3) $ \approx $ | 9.6025e-1 (3.73e-3) $ - $ | 7.5916e-1 (5.21e-3) $ - $ | 8.3706e-1 (3.11e-4) $ - $ | 9.9221e-1 (9.41e-4) | |
| 15 | 24 | 9.1029e-1 (3.59e-3) $ - $ | 9.9317e-1 (3.77e-3) $ \approx $ | 9.9051e-1 (4.57e-3) $ \approx $ | 7.8204e-1 (3.95e-3) $ - $ | 8.7618e-1 (2.56e-4) $ - $ | 9.9681e-1 (1.72e-3) | |
| WFG3 | 5 | 14 | 5.2384e-2 (2.57e-2) $ \approx $ | 7.3816e-2 (4.76e-2) $ - $ | 6.1034e-2 (1.52e-2) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 1.4825e-1 (3.29e-2) |
| 10 | 19 | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) | |
| 15 | 24 | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) | |
| WFG4 | 5 | 14 | 7.8652e-1 (4.19e-3) $ - $ | 8.0753e-1 (2.71e-3) $ \approx $ | 7.6037e-1 (1.02e-2) $ - $ | 7.6052e-1 (3.96e-3) $ - $ | 6.2513e-1 (2.41e-3) $ - $ | 8.0615e-1 (1.22e-3) |
| 10 | 19 | 8.7023e-1 (3.95e-3) $ - $ | 9.3051e-1 (4.15e-3) $ - $ | 8.5127e-1 (3.72e-2) $ - $ | 7.7108e-1 (1.95e-3) $ - $ | 8.4058e-1 (2.11e-3) $ - $ | 9.4963e-1 (2.90e-3) | |
| 15 | 24 | 7.8157e-1 (2.74e-3) $ - $ | 9.8016e-1 (2.43e-3) $ \approx $ | 8.6352e-1 (4.91e-2) $ - $ | 8.8205e-1 (2.33e-3) $ - $ | 8.6073e-1 (1.56e-4) $ - $ | 9.8303e-1 (1.78e-3) | |
| WFG5 | 5 | 14 | 8.5082e-1 (1.43e-3) $ + $ | 7.8365e-1 (1.24e-3) $ - $ | 7.6019e-1 (5.31e-3) $ - $ | 5.3601e-1 (4.28e-3) $ - $ | 4.7201e-1 (2.17e-4) $ - $ | 7.6022e-1 (4.13e-4) |
| 10 | 19 | 8.9681e-1 (1.78e-3) $ \approx $ | 9.0254e-1 (1.38e-3) $ + $ | 8.5597e-1 (5.40e-3) $ - $ | 6.2801e-1 (5.28e-3) $ - $ | 7.0921e-1 (4.71e-4) $ - $ | 8.9832e-1 (6.00e-4) | |
| 15 | 24 | 8.9087e-1 (6.15e-4) $ - $ | 8.7194e-1 (2.83e-3) $ - $ | 8.5701e-1 (4.22e-3) $ - $ | 8.2071e-1 (3.45e-3) $ - $ | 9.0427e-1 (3.18e-4) $ \approx $ | 9.1455e-1 (4.77e-4) | |
| WFG6 | 5 | 14 | 7.2679e-1 (1.37e-2) $ - $ | 7.4503e-1 (1.43e-2) $ \approx $ | 7.6805e-1 (1.47e-2) $ + $ | 4.2131e-1 (4.37e-2) $ - $ | 7.3638e-1 (2.32e-2) $ \approx $ | 7.4043e-1 (1.41e-2) |
| 10 | 19 | 8.6787e-1 (1.69e-2) $ \approx $ | 8.3508e-1 (1.28e-2) $ - $ | 6.8542e-1 (3.64e-2) $ - $ | 6.9415e-1 (3.28e-3) $ - $ | 6.7155e-1 (2.66e-2) $ - $ | 8.6998e-1 (1.36e-2) | |
| 15 | 24 | 8.9551e-1 (1.21e-2) $ \approx $ | 8.8418e-1 (2.80e-2) $ \approx $ | 6.5042e-1 (1.26e-1) $ - $ | 7.3942e-1 (5.28e-1) $ - $ | 6.5218e-1 (4.01e-2) $ - $ | 8.8649e-1 (1.31e-2) | |
| WFG7 | 5 | 14 | 7.2351e-1 (2.51e-3) $ - $ | 8.0052e-1 (2.23e-3) $ - $ | 8.0617e-1 (5.39e-3) $ \approx $ | 6.5035e-1 (4.81e-3) $ - $ | 7.1468e-1 (3.26e-4) $ - $ | 8.0758e-1 (2.98e-4) |
| 10 | 19 | 9.6087e-1 (2.53e-3) $ - $ | 9.6095e-1 (2.04e-3) $ - $ | 8.6218e-1 (2.76e-2) $ - $ | 7.8901e-1 (3.27e-3) $ - $ | 8.9122e-1 (2.16e-3) $ - $ | 9.5879e-1 (1.82e-3) | |
| 15 | 24 | 9.8611e-1 (9.20e-4) $ + $ | 9.8672e-1 (7.25e-4) $ + $ | 9.1340e-1 (2.51e-2) $ - $ | 7.5011e-1 (3.28e-3) $ - $ | 8.9041e-1 (2.75e-4) $ - $ | 9.8483e-1 (1.88e-3) | |
| WFG8 | 5 | 14 | 6.6585e-1 (4.70e-3) $ - $ | 6.6738e-1 (3.69e-3) $ - $ | 6.9475e-1 (6.81e-4) $ \approx $ | 3.7602e-1 (2.81e-3) $ - $ | 4.9130e-1 (4.26e-4) $ - $ | 6.9502e-1 (8.48e-4) |
| 10 | 19 | 8.5285e-1 (1.97e-2) $ - $ | 8.5119e-1 (2.81e-2) $ - $ | 7.7624e-1 (3.74e-2) $ - $ | 5.7165e-1 (3.25e-3) $ - $ | 6.3022e-1 (5.41e-3) $ - $ | 8.6059e-1 (5.74e-3) | |
| 15 | 24 | 9.2307e-1 (3.57e-3) $ + $ | 9.1868e-1 (4.57e-3) $ \approx $ | 9.1706e-1 (5.20e-3) $ \approx $ | 7.2511e-1 (3.70e-3) $ - $ | 9.2908e-1 (4.71e-3) $ - $ | 9.1724e-1 (5.33e-3) | |
| WFG9 | 5 | 14 | 6.5026e-1 (3.14e-3) $ - $ | 7.3908e-1 (2.57e-3) $ - $ | 7.2057e-1 (5.37e-3) $ - $ | 6.4742e-1 (5.96e-3) $ - $ | 6.3968e-1 (4.35e-3) $ - $ | 7.6045e-1 (3.47e-3) |
| 10 | 19 | 8.9267e-1 (5.22e-3) $ + $ | 8.9287e-1 (6.82e-3) $ + $ | 8.3925e-1 (4.68e-2) $ - $ | 6.4472e-1 (4.28e-3) $ - $ | 6.3082e-1 (5.37e-3) $ - $ | 8.7926e-1 (6.82e-3) | |
| 15 | 24 | 8.5781e-1 (6.30e-2) $ - $ | 8.5037e-1 (6.25e-2) $ - $ | 9.1051e-1 (8.25e-3) $ + $ | 4.7027e-1 (6.81e-3) $ - $ | 5.066e-1 (4.27e-2) $ - $ | 8.6638e-1 (1.92e-2) | |
| $ +/-/\approx $ | 3/17/7 | 3/14/10 | 2/16/9 | 0/24/3 | 0/22/5 | |||
DownLoad:
CSV
| Problem | $ m $ | $ D $ | RVEA | MOEA/D-AWA | RPS-NSGA-II | MOEA/D-UR | MaOEA-PDS | HEA | SDEA |
| CSIDP | 3 | 7 | 5.7035e-1 (1.72e-3) $ - $ | 3.7053e-1 (2.52e-2) $ - $ | 5.0271e-1 (2.17e-2) $ - $ | 5.1019e-1 (1.34e-3) $ - $ | 4.5127e-1 (4.32e-2) $ - $ | 6.3021e-1 (2.85e-3) $ \approx $ | 6.2908e-1 (1.35e-3) |
| WRMP | 5 | 3 | 4.1208e-1 (2.53e-2) $ - $ | 5.23721e-1 (1.59e-2) $ - $ | 3.9806e-1 (1.75e-2) $ - $ | 4.3253e-1 (2.98e-3) $ - $ | 5.7025e-1 (2.35e-2) $ - $ | 5.3920e-1 (2.12e-2) $ - $ | 7.5019e-1 (1.95e-2) |
| $ +/-/\approx $ | 0/2/0 | 0/2/0 | 0/2/0 | 0/2/0 | 0/2/0 | 0/1/1 | |||
DownLoad:
CSV
| $ m $ | Division($ H_{1} $, $ H_{2} $) | $ N $ |
| 5 | (6, 0) | 210 |
| 10 | (3, 2) | 275 |
| 15 | (2, 1) | 135 |
| Problem | $ m $ | $ D $ | RVEA | MOEA/D-AWA | RPS-NSGA-II | MOEA/D-UR | MaOEA-PDS | HEA | SDEA |
| WFG1 | 5 | 14 | 5.4635e-1 (7.46e-2) $ - $ | 3.9587e-1 (1.26e-2) $ - $ | 7.2685e-1 (4.79e-2) $ - $ | 7.0831e-1 (4.08e-2) $ - $ | 3.8544e-1 (1.66e-2) $ \approx $ | 4.4033e-1 (1.03e-2) $ - $ | 3.7929e-1 (1.06e-2) |
| 10 | 19 | 1.4450e+0 (9.21e-2) $ - $ | 1.0525e+0 (2.06e-2) $ - $ | 1.5323e+0 (4.55e-2) $ - $ | 1.8904e+0 (3.14e-2) $ - $ | 1.5440e+0 (1.27e-1) $ - $ | 1.0905e+0 (5.08e-2) $ - $ | 9.4664e-1 (1.55e-2) | |
| 15 | 24 | 1.9111e+0 (1.67e-1) $ \approx $ | 1.9978e+0 (6.68e-2) $ - $ | 2.1808e+0 (3.71e-2) $ - $ | 2.4509e+0 (4.71e-2) $ - $ | 2.4542e+0 (5.92e-1) $ - $ | 1.7976e+0 (3.28e-2) $ - $ | 1.6338e+0 (4.70e-2) | |
| WFG2 | 5 | 14 | 3.8976e-1 (2.07e-3) $ \approx $ | 4.5517e-1 (1.33e-2) $ - $ | 6.0492e-1 (4.93e-2) $ - $ | 6.6631e-1 (7.09e-2) $ - $ | 4.0538e-1 (1.17e-2) $ - $ | 4.5076e-1 (7.40e-3) $ - $ | 3.8297e-1 (1.04e-2) |
| 10 | 19 | 1.0615e+0 (8.53e-2) $ \approx $ | 1.1886e+0 (3.23e-2) $ - $ | 1.7249e+0 (6.11e-2) $ - $ | 1.7981e+0 (5.37e-2) $ - $ | 1.0226e+0 (2.03e-2) $ + $ | 1.0919e+0 (2.49e-2) $ \approx $ | 1.1072e+0 (3.42e-2) | |
| 15 | 24 | 2.0018e+0 (9.86e-2) $ - $ | 2.4181e+0 (6.59e-2) $ - $ | 2.4740e+0 (4.28e-2) $ - $ | 1.7691e+0 (4.42e-2) $ \approx $ | 2.0263e+0 (2.09e-1) $ - $ | 1.8323e+0 (7.73e-2) $ - $ | 1.6338e+0 (4.70e-2) | |
| WFG3 | 5 | 14 | 4.4053e-1 (4.97e-2) $ \approx $ | 3.4144e-1 (3.25e-2) $ + $ | 1.8024e+0 (2.61e-1) $ - $ | 1.2584e+0 (1.48e-1) $ - $ | 5.0335e-1 (1.06e-1) $ \approx $ | 3.0801e-1 (5.42e-2) $ + $ | 4.5416e-1 (2.63e-2) |
| 10 | 19 | 2.0550e+0 (1.94e-1) $ + $ | 1.6373e+0 (2.49e-1) $ + $ | 6.6493e+0 (7.25e-1) $ - $ | 5.1583e+0 (2.80e-1) $ - $ | 1.8029e+0 (1.88e-1) $ + $ | 1.3715e+0 (4.29e-1) $ + $ | 3.4318e+0 (9.29e-1) | |
| 15 | 24 | 3.5356e+0 (1.39e+0) $ + $ | 3.5804e+0 (1.58e+0) $ + $ | 1.3448e+1 (5.47e-1) $ - $ | 9.3436e+0 (6.32e-1) $ \approx $ | 4.8727e+0 (3.73e-1) $ + $ | 3.1511e+0 (1.07e+0) $ + $ | 8.5894e+0 (2.28e+0) | |
| WFG4 | 5 | 14 | 9.6578e-1 (1.33e-3) $ \approx $ | 1.0927e+0 (1.07e-2) $ - $ | 1.3665e+0 (6.93e-2) $ - $ | 1.4823e+0 (1.09e-1) $ - $ | 9.6670e-1 (1.50e-3) $ - $ | 1.1283e+0 (2.19e-2) $ - $ | 9.6503e-1 (1.29e-3) |
| 10 | 19 | 4.5176e+0 (1.11e-2) $ - $ | 4.5238e+0 (8.79e-2) $ - $ | 5.9297e+0 (1.33e-1) $ - $ | 5.9074e+0 (1.31e-1) $ - $ | 4.5598e+0 (2.09e-2) $ - $ | 4.8097e+0 (2.22e-2) $ - $ | 4.3489e+0 (5.08e-2) | |
| 15 | 24 | 9.4038e+0 (3.78e-2) $ - $ | 9.6090e+0 (2.38e-1) $ - $ | 1.1576e+1 (3.12e-1) $ - $ | 1.1622e+1 (3.11e-1) $ - $ | 9.4306e+0 (2.12e-2) $ - $ | 1.0005e+1 (1.06e-1) $ - $ | 9.3085e+0 (1.21e-1) | |
| WFG5 | 5 | 14 | 9.5715e-1 (6.34e-4) $ \approx $ | 1.0975e+0 (1.38e-2) $ - $ | 1.2890e+0 (6.34e-2) $ - $ | 1.3457e+0 (7.76e-2) $ - $ | 9.5725e-1 (1.32e-3) $ \approx $ | 1.1162e+0 (2.48e-2) $ - $ | 9.5719e-1 (1.01e-3) |
| 10 | 19 | 4.5065e+0 (1.33e-2) $ - $ | 4.9460e+0 (6.56e-2) $ - $ | 5.7480e+0 (1.62e-1) $ - $ | 5.7359e+0 (2.36e-1) $ - $ | 4.5447e+0 (9.44e-3) $ - $ | 4.7827e+0 (3.58e-2) $ - $ | 4.3479e+0 (2.75e-2) | |
| 15 | 24 | 9.2848e+0 (6.48e-3) $ - $ | 1.0669e+1 (1.74e-1) $ - $ | 1.1357e+1 (4.98e-1) $ - $ | 1.1187e+1 (2.09e-1) $ - $ | 9.3812e+0 (1.09e-2) $ - $ | 9.9388e+0 (1.70e-1) $ - $ | 9.2056e+0 (6.68e-2) | |
| WFG6 | 5 | 14 | 9.6252e-1 (1.14e-3) $ - $ | 1.0948e+0 (1.85e-2) $ - $ | 1.5668e+0 (1.11e-1) $ - $ | 1.7897e+0 (1.18e-1) $ - $ | 9.5952e-1 (9.96e-4) $ + $ | 1.1153e+0 (1.30e-2) $ - $ | 9.6094e-1 (8.47e-4) |
| 10 | 19 | 4.5765e+0 (1.85e-2) $ - $ | 4.5781e+0 (1.62e-1) $ - $ | 6.8764e+0 (2.00e-1) $ - $ | 6.6838e+0 (1.58e-1) $ - $ | 4.5748e+0 (1.36e-2) $ - $ | 4.8668e+0 (3.52e-2) $ - $ | 4.2328e+0 (4.95e-2) | |
| 15 | 24 | 9.4004e+0 (1.63e-1) $ + $ | 1.0520e+1 (5.68e-1) $ - $ | 1.2902e+1 (4.00e-1) $ - $ | 1.2160e+1 (2.80e-1) $ - $ | 9.3998e+0 (1.63e-2) $ + $ | 1.0004e+1 (1.66e-1) $ \approx $ | 9.8458e+0 (3.07e-1) | |
| WFG7 | 5 | 14 | 9.6656e-1 (6.58e-4) $ \approx $ | 1.0954e+0 (1.10e-2) $ - $ | 1.5246e+0 (1.34e-1) $ - $ | 1.8936e+0 (1.74e-1) $ - $ | 9.6620e-1 (5.91e-4) $ + $ | 1.1273e+0 (1.38e-2) $ - $ | 9.6704e-1 (7.35e-4) |
| 10 | 19 | 4.5568e+0 (5.79e-2) $ - $ | 4.5835e+0 (5.31e-2) $ - $ | 5.9259e+0 (1.69e-1) $ - $ | 6.1533e+0 (2.17e-1) $ - $ | 4.5483e+0 (2.31e-2) $ - $ | 4.8166e+0 (1.48e-2) $ - $ | 4.3796e+0 (4.55e-2) | |
| 15 | 24 | 9.3673e+0 (7.65e-2) $ \approx $ | 9.6379e+0 (1.09e-1) $ - $ | 1.1123e+1 (4.45e-1) $ - $ | 1.1392e+1 (5.91e-1) $ - $ | 9.3975e+0 (2.34e-2) $ - $ | 1.0018e+1 (1.48e-1) $ - $ | 9.3218e+0 (8.84e-2) | |
| WFG8 | 5 | 14 | 9.9498e-1 (7.53e-3) $ \approx $ | 1.1109e+0 (9.54e-3) $ - $ | 1.5536e+0 (1.08e-1) $ - $ | 1.5334e+0 (1.01e-1) $ - $ | 9.9122e-1 (1.92e-3) $ \approx $ | 1.1370e+0 (1.90e-2) $ - $ | 9.8996e-1 (1.47e-3) |
| 10 | 19 | 4.6331e+0 (2.82e-1) $ - $ | 4.4239e+0 (6.29e-2) $ - $ | 6.3655e+0 (3.95e-1) $ - $ | 6.4510e+0 (3.48e-1) $ - $ | 4.6307e+0 (3.14e-2) $ - $ | 4.8988e+0 (2.70e-2) $ - $ | 4.2874e+0 (8.91e-2) | |
| 15 | 24 | 1.0290e+1 (4.26e-1) $ - $ | 1.1028e+1 (6.27e-1) $ - $ | 1.1822e+1 (4.75e-1) $ - $ | 9.3252e+0 (3.36e-2) $ \approx $ | 9.7060e+0 (7.39e-2) $ - $ | 9.3963e+0 (3.55e-1) $ - $ | 9.1878e+0 (1.93e-1) | |
| WFG9 | 5 | 14 | 9.3407e-1 (2.99e-3) $ + $ | 1.0364e+0 (2.11e-2) $ - $ | 1.2761e+0 (3.72e-2) $ - $ | 1.4277e+0 (9.88e-2) $ - $ | 9.3237e-1 (3.36e-3) $ + $ | 1.0890e+0 (1.53e-2) $ - $ | 9.4329e-1 (1.94e-3) |
| 10 | 19 | 4.3721e+0 (4.86e-2) $ - $ | 4.4745e+0 (1.14e-1) $ - $ | 5.4624e+0 (1.45e-1) $ - $ | 5.5268e+0 (2.36e-1) $ - $ | 4.4653e+0 (3.26e-2) $ - $ | 4.7361e+0 (2.93e-2) $ - $ | 4.2489e+0 (5.03e-2) | |
| 15 | 24 | 8.7261e+0 (1.09e-1) $ + $ | 9.6567e+0 (2.62e-1) $ - $ | 1.0325e+1 (2.92e-1) $ - $ | 1.0114e+1 (4.07e-1) $ - $ | 8.8524e+0 (1.04e-1) $ + $ | 9.5723e+0 (9.41e-2) $ - $ | 9.1838e+0 (1.46e-1) | |
| $ +/-/\approx $ | 5/13/9 | 3/24/0 | 0/27/0 | 0/24/3 | 6/18/3 | 3/22/2 | |||
| Problem | $ m $ | $ D $ | RVEA | MOEA/D-AWA | RPS-NSGA-II | MOEA/D-UR | MaOEA-PDS | HEA | SDEA |
| WFG1 | 5 | 14 | 8.3053e-1 (2.90e-2) $ - $ | 9.7421e-1 (2.46e-2) $ - $ | 8.5218e-1 (4.63e-2) $ - $ | 9.7917e-1 (2.42e-2) $ - $ | 9.8167e-1 (1.93e-2) $ - $ | 9.8866e-1 (1.93e-2) $ \approx $ | 9.9801e-1 (3.79e-4) |
| 10 | 19 | 7.0216e-1 (7.03e-2) $ - $ | 9.9014e-1 (1.37e-2) $ - $ | 9.0923e-1 (5.62e-2) $ - $ | 9.9337e-1 (1.75e-3) $ - $ | 9.9065e-1 (1.89e-3) $ - $ | 9.9715e-1 (1.56e-3) $ - $ | 9.9963e-1 (3.80e-4) | |
| 15 | 24 | 9.9970e-1 (2.81e-4) $ - $ | 9.9908e-1 (5.68e-4) $ - $ | 9.9719e-1 (7.74e-4) $ - $ | 9.9822e-1 (9.05e-4) $ - $ | 9.1853e-1 (9.60e-2) $ - $ | 9.9713e-1 (6.41e-4) $ - $ | 9.9998e-1 (2.41e-5) | |
| WFG2 | 5 | 14 | 9.9206e-1 (1.59e-3) $ - $ | 9.7930e-1 (7.83e-3) $ - $ | 9.5050e-1 (1.24e-2) $ - $ | 9.7737e-1 (6.58e-3) $ - $ | 9.8845e-1 (2.47e-3) $ - $ | 9.9039e-1 (1.66e-3) $ - $ | 9.9366e-1 (7.60e-4) |
| 10 | 19 | 9.9048e-1 (4.84e-3) $ \approx $ | 9.9382e-1 (1.38e-3) $ + $ | 9.7457e-1 (6.38e-3) $ - $ | 9.8970e-1 (1.93e-3) $ - $ | 9.8425e-1 (3.78e-3) $ - $ | 9.8200e-1 (3.18e-3) $ - $ | 9.9221e-1 (9.41e-4) | |
| 15 | 24 | 9.9283e-1 (2.47e-3) $ - $ | 9.9575e-1 (9.90e-4) $ \approx $ | 9.7657e-1 (4.57e-3) $ - $ | 9.9191e-1 (2.00e-3) $ - $ | 9.8856e-1 (4.47e-3) $ - $ | 9.7026e-1 (1.01e-2) $ - $ | 9.9681e-1 (1.72e-3) | |
| WFG3 | 5 | 14 | 1.6003e-1 (1.76e-2) $ \approx $ | 2.1443e-1 (2.02e-2) $ + $ | 0.0000e+0 (0.00e+0) $ - $ | 6.2622e-2 (1.19e-2) $ - $ | 2.0853e-1 (1.16e-2) $ + $ | 1.5886e-1 (1.06e-2) $ \approx $ | 1.4825e-1 (3.29e-2) |
| 10 | 19 | 0.0000e+0 (0.00e+0) $ \approx $ | 3.7907e-3 (5.95e-3) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 3.5069e-2 (2.21e-2) $ + $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) | |
| 15 | 24 | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) | |
| WFG4 | 5 | 14 | 7.9689e-1 (1.75e-3) $ - $ | 7.8109e-1 (1.54e-3) $ - $ | 6.0579e-1 (1.99e-2) $ - $ | 6.8128e-1 (1.21e-2) $ - $ | 7.8801e-1 (3.28e-3) $ - $ | 7.9944e-1 (1.74e-3) $ - $ | 8.0615e-1 (1.22e-3) |
| 10 | 19 | 9.2669e-1 (4.49e-3) $ - $ | 9.4142e-1 (2.19e-3) $ - $ | 6.1073e-1 (2.34e-2) $ - $ | 7.9287e-1 (6.16e-3) $ - $ | 9.2540e-1 (3.47e-3) $ - $ | 9.2109e-1 (5.45e-3) $ - $ | 9.4963e-1 (2.90e-3) | |
| 15 | 24 | 9.8122e-1 (1.93e-3) $ \approx $ | 9.4296e-1 (3.86e-3) $ - $ | 4.7679e-1 (3.71e-2) $ - $ | 8.3205e-1 (1.21e-2) $ - $ | 9.7710e-1 (1.66e-3) $ - $ | 9.7070e-1 (4.84e-3) $ - $ | 9.8303e-1 (1.78e-3) | |
| WFG5 | 5 | 14 | 7.5959e-1 (5.56e-4) $ - $ | 7.3861e-1 (1.22e-3) $ - $ | 6.0160e-1 (1.92e-2) $ - $ | 6.5696e-1 (8.53e-3) $ - $ | 7.4441e-1 (3.15e-3) $ - $ | 7.5920e-1 (8.25e-4) $ - $ | 7.6022e-1 (4.13e-4) |
| 10 | 19 | 8.9239e-1 (1.01e-3) $ - $ | 8.8790e-1 (1.36e-3) $ - $ | 6.2494e-1 (2.15e-2) $ - $ | 7.5820e-1 (1.48e-2) $ - $ | 8.8046e-1 (3.92e-3) $ - $ | 8.8853e-1 (1.92e-3) $ - $ | 8.9832e-1 (6.00e-4) | |
| 15 | 24 | 9.1403e-1 (3.10e-4) $ - $ | 8.6622e-1 (5.15e-3) $ - $ | 5.3440e-1 (5.33e-2) $ - $ | 7.8067e-1 (1.10e-2) $ - $ | 9.0666e-1 (2.57e-3) $ - $ | 9.1592e-1 (2.40e-4) $ + $ | 9.1455e-1 (4.77e-4) | |
| WFG6 | 5 | 14 | 7.2850e-1 (8.39e-3) $ - $ | 7.2679e-1 (1.51e-2) $ - $ | 5.2228e-1 (1.93e-2) $ - $ | 5.7880e-1 (1.52e-2) $ - $ | 7.2883e-1 (1.18e-2) $ - $ | 7.3955e-1 (1.05e-2) $ \approx $ | 7.4043e-1 (1.41e-2) |
| 10 | 19 | 8.6206e-1 (1.81e-2) $ - $ | 8.5783e-1 (1.57e-2) $ - $ | 4.6979e-1 (2.58e-2) $ - $ | 6.4730e-1 (3.00e-2) $ - $ | 8.5984e-1 (1.60e-2) $ - $ | 8.4746e-1 (9.01e-3) $ - $ | 8.6998e-1 (1.36e-2) | |
| 15 | 24 | 8.8396e-1 (3.18e-2) $ - $ | 8.2787e-1 (3.14e-2) $ - $ | 3.0173e-1 (2.60e-2) $ - $ | 6.6774e-1 (3.40e-2) $ - $ | 8.8606e-1 (3.44e-2) $ \approx $ | 6.9644e-1 (9.03e-2) $ - $ | 8.8649e-1 (1.31e-2) | |
| WFG7 | 5 | 14 | 8.0157e-1 (1.42e-3) $ - $ | 7.9170e-1 (1.66e-3) $ - $ | 6.0729e-1 (3.24e-2) $ - $ | 6.3656e-1 (1.59e-2) $ - $ | 7.9105e-1 (2.35e-3) $ - $ | 8.0340e-1 (7.00e-4) $ - $ | 8.0758e-1 (2.98e-4) |
| 10 | 19 | 9.4331e-1 (3.60e-3) $ - $ | 9.5594e-1 (1.10e-3) $ - $ | 5.6192e-1 (3.11e-2) $ - $ | 7.8347e-1 (1.37e-2) $ - $ | 9.4214e-1 (1.66e-3) $ - $ | 9.3428e-1 (2.94e-3) $ - $ | 9.5879e-1 (1.82e-3) | |
| 15 | 24 | 9.7887e-1 (4.38e-3) $ - $ | 9.5899e-1 (3.22e-3) $ - $ | 4.4289e-1 (4.06e-2) $ - $ | 8.2889e-1 (2.07e-2) $ - $ | 9.7911e-1 (1.70e-3) $ - $ | 8.9832e-1 (1.20e-1) $ - $ | 9.8483e-1 (1.88e-3) | |
| WFG8 | 5 | 14 | 6.8616e-1 (3.90e-3) $ - $ | 6.5493e-1 (4.43e-3) $ - $ | 4.2548e-1 (2.67e-2) $ - $ | 5.4135e-1 (2.03e-2) $ - $ | 6.8318e-1 (3.65e-3) $ - $ | 6.9016e-1 (1.71e-3) $ - $ | 6.9502e-1 (8.48e-4) |
| 10 | 19 | 8.4110e-1 (8.57e-3) $ - $ | 8.5124e-1 (3.00e-2) $ \approx $ | 2.9935e-1 (6.35e-2) $ - $ | 5.1359e-1 (1.07e-1) $ - $ | 8.3601e-1 (1.18e-2) $ - $ | 7.1634e-1 (7.48e-2) $ - $ | 8.6059e-1 (5.74e-3) | |
| 15 | 24 | 9.2249e-1 (1.86e-2) $ \approx $ | 8.3888e-1 (2.18e-2) $ - $ | 3.1650e-1 (1.76e-1) $ - $ | 4.7231e-1 (1.04e-1) $ - $ | 9.0555e-1 (3.02e-3) $ - $ | 6.3933e-1 (7.66e-2) $ - $ | 9.1724e-1 (5.33e-3) | |
| WFG9 | 5 | 14 | 7.4953e-1 (5.75e-3) $ - $ | 7.5078e-1 (3.47e-3) $ - $ | 6.1318e-1 (9.20e-3) $ - $ | 6.4374e-1 (9.46e-3) $ - $ | 7.4210e-1 (3.51e-3) $ - $ | 7.6033e-1 (2.41e-3) $ \approx $ | 7.6045e-1 (3.47e-3) |
| 10 | 19 | 8.4481e-1 (5.63e-2) $ \approx $ | 8.8742e-1 (5.34e-3) $ + $ | 6.7655e-1 (2.62e-2) $ - $ | 7.2650e-1 (5.38e-2) $ - $ | 8.5557e-1 (1.25e-2) $ - $ | 8.5015e-1 (2.13e-2) $ - $ | 8.7926e-1 (6.82e-3) | |
| 15 | 24 | 8.7216e-1 (7.24e-2) $ + $ | 8.6629e-1 (1.39e-2) $ \approx $ | 5.9362e-1 (3.55e-2) $ - $ | 6.9505e-1 (6.75e-2) $ - $ | 8.9247e-1 (7.23e-3) $ + $ | 8.0237e-1 (6.23e-2) $ - $ | 8.6638e-1 (1.92e-2) | |
| $ +/-/\approx $ | 1/19/7 | 3/19/5 | 0/25/2 | 0/25/2 | 3/22/2 | 1/20/6 | |||
| SDEA vs | IGD | HV | |||||
| $ R^{+} $ | $ R^{-} $ | $ p $-value | $ R^{+} $ | $ R^{-} $ | $ p $-value | ||
| RVEA | 313 | 65.0 | 0.00278 | 327.5 | 50.5 | 0.000381 | |
| MOEA/D-AWA | 308.0 | 70.0 | 0.00259 | 330.5 | 47.5 | 0.000512 | |
| RPS-NSGA-II | 311.0 | 67.0 | 0.002105 | 376 | 2 | 0.000058 | |
| MORA/D-UR | 378.0 | 0.0 | 0.000005 | 376 | 2 | 0.000058 | |
| MaOEA-PDS | 378.0 | 0.0 | 0.000005 | 301.5 | 76.5 | 0.004471 | |
| HEA | 320.0 | 58.0 | 0.001021 | 351.5 | 26.5 | 0.000074 | |
| Problem | $ m $ | $ D $ | RVEA | MOEA/D-AWA | RPS-NSGA-II | MOEA/D-UR | MaOEA-PDS | HEA | SDEA |
| MaF1 | 5 | 14 | 1.8093e-1 (9.11e-3) $ - $ | 2.7675e-1 (1.65e-2) $ - $ | 1.9100e-1 (8.70e-3) $ - $ | 1.0217e-1 (7.92e-4) $ + $ | 3.0799e-1 (1.99e-2) $ - $ | 1.5632e-1 (2.59e-3) $ - $ | 1.0332e-1 (5.15e-4) |
| 10 | 19 | 2.8503e-1 (1.00e-2) $ - $ | 5.7593e-1 (5.61e-2) $ - $ | 2.8201e-1 (9.95e-3) $ - $ | 2.0642e-1 (8.54e-4) $ + $ | 3.4548e-1 (1.93e-2) $ - $ | 3.3494e-1 (2.12e-2) $ - $ | 2.1234e-1 (1.28e-3) | |
| 15 | 24 | 3.3595e-1 (4.33e-3) $ - $ | 7.5949e-1 (1.01e-1) $ - $ | 3.8999e-1 (1.11e-1) $ - $ | 3.0240e-1 (2.34e-2) $ \approx $ | 4.2697e-1 (1.92e-2) $ - $ | 4.0789e-1 (8.02e-3) $ - $ | 2.9088e-1 (3.59e-3) | |
| MaF2 | 5 | 14 | 1.1175e-1 (2.75e-3) $ - $ | 1.1591e-1 (1.02e-3) $ - $ | 1.4819e-1 (3.49e-3) $ - $ | 8.0600e-2 (1.10e-3) $ + $ | 1.2648e-1 (2.17e-3) $ - $ | 1.1666e-1 (7.76e-4) $ - $ | 9.3169e-2 (1.42e-3) |
| 10 | 19 | 2.0786e-1 (1.27e-2) $ - $ | 2.5117e-1 (1.81e-2) $ - $ | 2.4176e-1 (3.76e-3) $ - $ | 1.9495e-1 (7.87e-3) $ - $ | 1.8387e-1 (1.94e-3) $ - $ | 1.7124e-1 (3.52e-3) $ - $ | 1.6260e-1 (2.20e-3) | |
| 15 | 24 | 2.5667e-1 (2.82e-2) $ - $ | 7.0357e-1 (1.09e-1) $ - $ | 2.1510e-1 (3.51e-3) $ \approx $ | 3.0015e-1 (2.46e-2) $ - $ | 2.1522e-1 (3.18e-3) $ \approx $ | 2.1796e-1 (7.13e-3) $ - $ | 1.9880e-1 (2.52e-3) | |
| MaF3 | 5 | 14 | 7.7658e-2 (5.48e-3) $ + $ | 2.0514e-1 (1.28e-1) $ - $ | 1.4994e-1 (2.37e-2) $ - $ | 2.2080e-1 (1.66e-1) $ - $ | 1.0687e-1 (2.28e-2) $ \approx $ | 1.2120e-1 (3.98e-2) $ - $ | 9.4125e-2 (5.81e-3) |
| 10 | 19 | 1.8372e+2 (2.28e+2) $ - $ | 8.9908e-1 (5.78e-1) $ - $ | 3.9731e-1 (1.94e-1) $ - $ | 1.5877e+1 (9.42e+0) $ - $ | 1.2335e-1 (8.51e-3) $ \approx $ | 2.8923e-1 (6.64e-2) $ - $ | 2.0168e-1 (2.46e-1) | |
| 15 | 24 | 4.5781e+1 (1.08e+2) $ - $ | 1.0100e-1 (4.80e-3) $ + $ | 2.3712e-1 (2.12e-1) $ - $ | 1.5856e-1 (7.32e-2) $ - $ | 1.8609e-1 (2.83e-2) $ - $ | 7.1590e+5 (9.20e+5) $ - $ | 1.0980e-1 (4.30e-3) | |
| MaF4 | 5 | 14 | 2.8205e+0 (7.22e-1) $ - $ | 3.5731e+0 (8.15e-1) $ - $ | 3.1007e+0 (4.89e-1) $ - $ | 3.1097e+0 (5.18e-1) $ - $ | 2.8769e+0 (9.80e-2) $ - $ | 2.8891e+0 (2.62e-1) $ - $ | 2.0313e+0 (7.54e-2) |
| 10 | 19 | 9.7435e+1 (5.35e+0) $ - $ | 1.8558e+2 (2.52e+1) $ - $ | 1.5111e+2 (2.27e+1) $ - $ | 1.2357e+2 (1.86e+1) $ - $ | 8.6259e+1 (7.06e+0) $ - $ | 1.1181e+2 (9.77e+0) $ - $ | 5.3091e+1 (2.05e+0) | |
| 15 | 24 | 4.3936e+3 (2.26e+2) $ - $ | 7.8711e+3 (2.62e+3) $ - $ | 1.5273e+4 (8.13e+3) $ - $ | 8.8258e+3 (2.68e+3) $ - $ | 4.9481e+3 (4.93e+2) $ - $ | 3.6663e+3 (3.62e+2) $ - $ | 2.2801e+3 (1.82e+2) | |
| MaF5 | 5 | 14 | 1.9704e+0 (2.65e-3) $ - $ | 1.9719e+0 (2.23e-3) $ - $ | 1.9269e+0 (2.30e-2) $ - $ | 3.7179e+0 (4.57e-1) $ - $ | 2.2132e+0 (4.51e-2) $ - $ | 1.9609e+0 (9.42e-3) $ - $ | 1.8264e+0 (2.57e-2) |
| 10 | 19 | 7.7358e+1 (6.82e-1) $ - $ | 9.4917e+1 (8.78e+0) $ - $ | 5.6693e+1 (4.40e+0) $ \approx $ | 2.5553e+2 (2.13e+1) $ - $ | 8.1962e+1 (1.38e+0) $ - $ | 6.8867e+1 (2.53e+0) $ - $ | 5.2805e+1 (2.15e+0) | |
| 15 | 24 | 2.9732e+3 (3.91e+2) $ \approx $ | 3.4405e+3 (3.17e+2) $ - $ | 7.1810e+3 (2.06e+2) $ - $ | 3.3465e+3 (6.39e+2) $ \approx $ | 2.7237e+3 (4.67e+2) $ \approx $ | 2.7973e+3 (3.82e+2) $ - $ | 1.9311e+3 (9.37e+1) | |
| MaF6 | 5 | 14 | 1.7091e-2 (4.80e-3) $ - $ | 6.5710e-2 (8.06e-3) $ - $ | 3.7627e-1 (1.75e-1) $ - $ | 1.5278e-1 (2.57e-2) $ - $ | 1.4969e-1 (3.67e-2) $ - $ | 9.3489e-2 (1.20e-2) $ - $ | 2.1829e-3 (1.22e-4) |
| 10 | 19 | 5.1392e-1 (1.37e-1) $ - $ | 9.6871e-2 (2.37e-2) $ + $ | 2.6096e-1 (8.98e-2) $ \approx $ | 7.5239e-1 (2.55e-1) $ - $ | 2.6798e+0 (5.12e+0) $ - $ | 1.6405e-1 (5.96e-2) $ + $ | 3.1961e-1 (6.90e-2) | |
| 15 | 24 | 3.1400e-1 (7.06e-2) $ \approx $ | 1.7163e-1 (6.01e-7) $ + $ | 8.6088e+0 (9.50e+0) $ - $ | 1.5886e-1 (3.06e-2) $ + $ | 1.2430e+1 (3.08e+1) $ - $ | 3.8032e-1 (1.63e-1) $ \approx $ | 3.1966e-1 (4.75e-2) | |
| MaF7 | 5 | 24 | 2.8163e-1 (8.99e-3) $ - $ | 7.8828e-1 (1.03e-1) $ - $ | 3.1363e-1 (7.14e-3) $ - $ | 3.0032e-1 (2.29e-2) $ - $ | 3.7629e-1 (5.08e-2) $ - $ | 3.0510e-1 (3.46e-3) $ - $ | 2.3753e-1 (3.46e-3) |
| 10 | 29 | 1.1300e+0 (1.44e-1) $ \approx $ | 3.0577e+0 (1.63e+0) $ - $ | 2.0908e+0 (5.39e-1) $ - $ | 2.0597e+0 (4.40e-1) $ - $ | 1.3901e+0 (1.29e-2) $ - $ | 1.6103e+0 (9.52e-2) $ - $ | 1.0949e+0 (1.03e-1) | |
| 15 | 34 | 7.6126e+0 (1.26e+0) $ - $ | 4.1200e+0 (1.36e+0) $ - $ | 6.6959e+0 (1.04e+0) $ - $ | 5.1948e+0 (4.46e-1) $ - $ | 6.7687e+0 (1.79e-1) $ - $ | 8.2024e+0 (3.78e-1) $ - $ | 2.4257e+0 (3.02e-1) | |
| MaF8 | 5 | 2 | 1.5616e-1 (1.13e-2) $ - $ | 3.3402e-1 (2.84e-2) $ - $ | 3.0683e-1 (3.86e-2) $ - $ | 8.3643e-2 (1.62e-3) $ \approx $ | 2.2278e-1 (4.07e-2) $ - $ | 1.4864e-1 (6.43e-3) $ - $ | 8.2350e-2 (6.26e-3) |
| 10 | 2 | 3.5278e-1 (1.01e-1) $ - $ | 9.9749e-1 (1.29e-1) $ - $ | 6.9682e-1 (8.72e-2) $ - $ | 1.2152e-1 (3.27e-3) $ \approx $ | 2.9131e-1 (7.31e-2) $ - $ | 2.9153e-1 (1.09e-1) $ - $ | 1.2209e-1 (4.52e-3) | |
| 15 | 2 | 4.1765e-1 (3.93e-2) $ - $ | 1.3049e+0 (2.48e-1) $ - $ | 9.9097e-1 (1.58e-1) $ - $ | 2.1885e-1 (9.41e-3) $ + $ | 5.1606e-1 (6.52e-2) $ - $ | 4.5377e-1 (2.97e-2) $ - $ | 2.7222e-1 (3.96e-2) | |
| MaF9 | 5 | 2 | 3.2052e-1 (6.83e-2) $ - $ | 2.8137e-1 (3.16e-2) $ - $ | 2.3289e-1 (4.14e-2) $ - $ | 1.5154e-1 (4.65e-3) $ - $ | 1.5336e-1 (1.93e-2) $ - $ | 1.8296e-1 (3.19e-2) $ - $ | 7.5546e-2 (2.24e-3) |
| 10 | 2 | 5.3758e-1 (1.52e-1) $ - $ | 7.7406e-1 (2.37e-1) $ - $ | 4.3598e-1 (9.37e-2) $ - $ | 2.2451e-1 (6.70e-2) $ - $ | 4.8400e-1 (1.10e-1) $ - $ | 3.4440e-1 (4.80e-2) $ - $ | 1.2820e-1 (8.32e-3) | |
| 15 | 2 | 3.6908e+0 (5.49e+0) $ - $ | 1.8150e+0 (3.80e-1) $ - $ | 7.8709e-1 (1.69e-1) $ - $ | 3.3958e-1 (8.91e-2) $ - $ | 7.3567e+0 (5.42e+0) $ - $ | 3.7792e+0 (5.43e+0) $ - $ | 1.8485e-1 (4.08e-3) | |
| MaF10 | 5 | 14 | 4.9673e-1 (2.12e-2) $ - $ | 3.7616e-1 (1.17e-2) $ - $ | 4.0004e-1 (1.44e-2) $ - $ | 7.3363e-1 (5.32e-2) $ - $ | 4.4329e-1 (9.09e-3) $ - $ | 3.7721e-1 (9.23e-3) $ - $ | 3.6411e-1 (4.85e-3) |
| 10 | 19 | 1.4464e+0 (1.32e-1) $ - $ | 1.0918e+0 (3.13e-2) $ - $ | 1.0352e+0 (3.16e-2) $ - $ | 1.5495e+0 (4.82e-2) $ - $ | 1.6057e+0 (2.35e-1) $ - $ | 9.5575e-1 (3.25e-2) $ \approx $ | 9.6470e-1 (1.40e-2) | |
| 15 | 24 | 1.7889e+0 (1.15e-1) $ - $ | 1.7714e+0 (3.23e-2) $ - $ | 1.9862e+0 (8.38e-2) $ - $ | 2.2074e+0 (5.11e-2) $ - $ | 2.8502e+0 (7.26e-1) $ - $ | 1.6574e+0 (6.36e-2) $ \approx $ | 1.6561e+0 (2.52e-2) | |
| $ +/-/\approx $ | 1/26/3 | 3/27/0 | 0/27/3 | 5/21/4 | 0/26/4 | 1/26/3 | |||
| Problem | $ m $ | $ D $ | RVEA | MOEA/D-AWA | RPS-NSGA-II | MOEA/D-UR | MaOEA-PDS | HEA | SDEA |
| MaF1 | 5 | 14 | 6.5160e-3 (3.49e-4) $ - $ | 3.3381e-3 (2.76e-4) $ - $ | 7.2824e-3 (3.71e-4) $ - $ | 1.2931e-2 (6.69e-5) $ - $ | 2.4605e-3 (3.33e-4) $ - $ | 8.5443e-3 (1.78e-4) $ - $ | 1.3086e-2 (1.73e-4) |
| 10 | 19 | 4.6265e-7 (3.10e-8) $ \approx $ | 9.6417e-9 (3.87e-9) $ - $ | 5.4721e-7 (4.89e-8) $ \approx $ | 2.1973e-7 (3.51e-7) $ - $ | 2.4711e-7 (9.06e-8) $ - $ | 6.7236e-7 (3.24e-7) $ - $ | 8.9684e-7 (9.44e-8) | |
| 15 | 24 | 2.8020e-12 (2.45e-13) $ + $ | 2.3083e-14 (1.51e-14) $ + $ | 3.3293e-12 (1.68e-12) $ + $ | 2.8918e-11 (7.54e-11) $ \approx $ | 4.2472e-13 (2.14e-13) $ + $ | 1.2032e-12 (1.92e-13) $ + $ | 0.0000e+0 (0.00e+0) | |
| MaF2 | 5 | 14 | 1.8617e-1 (1.96e-3) $ - $ | 1.7705e-1 (1.71e-3) $ - $ | 1.8712e-1 (3.16e-3) $ - $ | 1.8890e-1 (2.04e-3) $ - $ | 1.7784e-1 (2.41e-3) $ - $ | 1.7737e-1 (1.39e-3) $ - $ | 2.0025e-1 (1.94e-3) |
| 10 | 19 | 2.1840e-1 (3.84e-3) $ - $ | 1.7054e-1 (2.67e-3) $ - $ | 2.0546e-1 (4.52e-3) $ - $ | 2.0151e-1 (5.21e-3) $ - $ | 1.9617e-1 (2.69e-3) $ - $ | 2.0335e-1 (1.82e-3) $ - $ | 2.2616e-1 (2.74e-3) | |
| 15 | 24 | 1.6970e-1 (9.83e-3) $ - $ | 8.8735e-2 (2.55e-2) $ - $ | 1.6401e-1 (4.65e-3) $ - $ | 1.6881e-1 (3.69e-3) $ - $ | 1.5509e-1 (3.46e-3) $ - $ | 1.5267e-1 (2.55e-3) $ - $ | 2.0144e-1 (3.56e-3) | |
| MaF3 | 5 | 14 | 9.9871e-1 (6.32e-4) $ \approx $ | 9.2138e-1 (1.60e-1) $ - $ | 9.7611e-1 (1.31e-2) $ - $ | 8.3051e-1 (2.34e-1) $ - $ | 9.9660e-1 (1.86e-3) $ - $ | 9.9315e-1 (7.94e-3) $ - $ | 9.9834e-1 (3.46e-4) |
| 10 | 19 | 1.4286e-1 (3.78e-1) $ - $ | 3.2615e-1 (4.53e-1) $ - $ | 7.0304e-1 (2.30e-1) $ - $ | 0.0000e+0 (0.00e+0) $ - $ | 9.9806e-1 (1.36e-3) $ \approx $ | 8.4629e-1 (1.01e-1) $ - $ | 9.0368e-1 (2.54e-1) | |
| 15 | 24 | 4.7947e-1 (4.64e-1) $ - $ | 9.9884e-1 (6.82e-4) $ \approx $ | 8.7677e-1 (2.48e-1) $ - $ | 9.7772e-1 (5.51e-2) $ \approx $ | 9.5232e-1 (3.53e-2) $ - $ | 4.7688e-1 (4.54e-1) $ - $ | 9.9932e-1 (1.02e-3) | |
| MaF4 | 5 | 14 | 7.0518e-2 (1.00e-2) $ - $ | 3.1046e-2 (1.42e-2) $ - $ | 1.0284e-1 (4.64e-3) $ - $ | 7.9981e-2 (1.28e-2) $ - $ | 8.4317e-2 (5.99e-3) $ - $ | 7.2699e-2 (2.38e-3) $ - $ | 1.3276e-1 (1.86e-3) |
| 10 | 19 | 2.1562e-4 (1.72e-5) $ - $ | 6.8904e-8 (1.59e-8) $ - $ | 4.0569e-5 (2.07e-5) $ - $ | 2.9345e-6 (2.36e-6) $ - $ | 2.0362e-4 (2.78e-5) $ - $ | 1.5428e-4 (2.68e-5) $ - $ | 3.6693e-4 (2.87e-5) | |
| 15 | 24 | 2.1729e-7 (3.19e-8) $ + $ | 2.0133e-12 (4.88e-12) $ - $ | 8.7797e-9 (1.37e-8) $ - $ | 1.1794e-11 (1.86e-11) $ - $ | 6.5427e-8 (1.42e-8) $ - $ | 2.7130e-8 (3.86e-8) $ - $ | 1.0839e-7 (3.45e-8) | |
| MaF5 | 5 | 14 | 8.1195e-1 (3.14e-4) $ + $ | 8.1213e-1 (3.04e-4) $ + $ | 7.9326e-1 (1.88e-3) $ - $ | 6.3853e-1 (1.69e-2) $ - $ | 7.9520e-1 (1.80e-3) $ - $ | 8.0824e-1 (1.59e-3) $ \approx $ | 8.0826e-1 (8.30e-4) |
| 10 | 19 | 9.6929e-1 (1.69e-4) $ - $ | 9.4729e-1 (2.30e-3) $ - $ | 9.5845e-1 (2.82e-3) $ - $ | 6.8320e-1 (3.73e-2) $ - $ | 9.5502e-1 (2.18e-3) $ - $ | 9.7005e-1 (1.56e-3) $ - $ | 9.7215e-1 (4.14e-4) | |
| 15 | 24 | 9.8923e-1 (3.61e-3) $ + $ | 9.1661e-1 (2.82e-2) $ - $ | 9.6073e-1 (4.97e-3) $ - $ | 5.6169e-1 (7.22e-2) $ - $ | 9.7546e-1 (6.23e-3) $ - $ | 9.8372e-1 (2.64e-3) $ - $ | 9.8844e-1 (1.64e-3) | |
| MaF6 | 5 | 14 | 1.2347e-1 (1.42e-3) $ - $ | 1.1722e-1 (2.29e-3) $ - $ | 9.3332e-2 (2.73e-2) $ - $ | 9.8929e-3 (5.99e-3) $ - $ | 1.0144e-1 (7.60e-3) $ - $ | 1.1860e-1 (4.22e-3) $ - $ | 1.2996e-1 (2.56e-4) |
| 10 | 19 | 6.5015e-4 (1.72e-3) $ - $ | 9.4780e-2 (6.13e-4) $ + $ | 6.3933e-2 (2.35e-2) $ \approx $ | 0.0000e+0 (0.00e+0) $ - $ | 1.3009e-13 (2.91e-13) $ - $ | 9.6725e-2 (1.66e-3) $ + $ | 3.8061e-2 (3.31e-2) | |
| 15 | 24 | 5.4888e-2 (2.95e-2) $ - $ | 9.1740e-2 (2.74e-4) $ + $ | 0.0000e+0 (0.00e+0) $ - $ | 5.8975e-4 (1.04e-3) $ - $ | 3.8785e-3 (6.00e-3) $ - $ | 8.9102e-2 (2.03e-3) $ + $ | 7.5767e-2 (7.28e-3) | |
| MaF7 | 5 | 24 | 2.5925e-1 (3.34e-3) $ - $ | 1.0563e-1 (1.96e-2) $ - $ | 2.5670e-1 (4.79e-3) $ - $ | 1.7534e-1 (1.92e-2) $ - $ | 2.4798e-1 (4.14e-3) $ - $ | 2.5353e-1 (2.11e-3) $ - $ | 2.7150e-1 (1.58e-3) |
| 10 | 29 | 1.7289e-1 (7.15e-3) $ - $ | 2.0858e-4 (2.64e-4) $ - $ | 1.8403e-1 (3.88e-3) $ \approx $ | 2.9045e-4 (1.31e-4) $ - $ | 1.7472e-1 (3.20e-3) $ - $ | 1.7483e-1 (7.17e-3) $ - $ | 1.8090e-1 (4.24e-3) | |
| 15 | 34 | 1.3662e-1 (2.01e-2) $ \approx $ | 1.5746e-5 (2.06e-5) $ - $ | 1.5022e-1 (7.15e-3) $ \approx $ | 0.0000e+0 (0.00e+0) $ - $ | 1.2905e-1 (1.31e-2) $ - $ | 1.4487e-1 (6.11e-3) $ \approx $ | 1.4421e-1 (2.17e-3) | |
| MaF8 | 5 | 2 | 1.1023e-1 (2.19e-3) $ - $ | 7.5719e-2 (3.12e-3) $ - $ | 8.6853e-2 (4.74e-3) $ - $ | 1.2532e-1 (3.04e-4) $ - $ | 1.0468e-1 (3.16e-3) $ - $ | 1.1021e-1 (1.30e-3) $ - $ | 1.2574e-1 (2.10e-4) |
| 10 | 2 | 9.1903e-3 (3.22e-4) $ - $ | 3.3919e-3 (7.76e-4) $ - $ | 5.0007e-3 (9.34e-4) $ - $ | 1.0950e-2 (5.36e-5) $ - $ | 8.0310e-3 (9.54e-4) $ - $ | 8.3357e-3 (2.99e-4) $ - $ | 1.1111e-2 (3.84e-5) | |
| 15 | 2 | 3.6538e-4 (3.32e-5) $ - $ | 1.3700e-4 (5.54e-5) $ - $ | 2.0735e-4 (3.08e-5) $ - $ | 5.7711e-4 (1.10e-5) $ - $ | 2.1292e-4 (4.32e-5) $ - $ | 1.6319e-4 (6.33e-5) $ - $ | 5.9484e-4 (1.15e-5) | |
| MaF9 | 5 | 2 | 2.2823e-1 (2.05e-2) $ - $ | 2.2671e-1 (1.30e-2) $ - $ | 2.4771e-1 (1.51e-2) $ - $ | 2.7906e-1 (3.33e-3) $ - $ | 2.9014e-1 (8.68e-3) $ - $ | 2.7870e-1 (1.23e-2) $ - $ | 3.1991e-1 (2.13e-3) |
| 10 | 2 | 9.4947e-3 (1.80e-3) $ - $ | 5.8386e-3 (1.68e-3) $ - $ | 1.0978e-2 (1.56e-3) $ - $ | 1.4818e-2 (1.35e-3) $ - $ | 7.6859e-3 (2.37e-3) $ - $ | 1.0519e-2 (1.43e-3) $ - $ | 1.7500e-2 (3.84e-4) | |
| 15 | 2 | 4.4083e-4 (3.29e-4) $ - $ | 1.1750e-4 (6.45e-5) $ - $ | 3.0851e-4 (7.06e-5) $ - $ | 8.6941e-4 (1.13e-4) $ - $ | 8.2382e-5 (2.01e-4) $ - $ | 3.1479e-4 (2.46e-4) $ - $ | 1.1605e-3 (2.45e-5) | |
| MaF10 | 5 | 14 | 8.7077e-1 (2.14e-2) $ - $ | 9.9743e-1 (8.94e-4) $ + $ | 9.8892e-1 (2.84e-3) $ - $ | 8.6314e-1 (2.11e-2) $ - $ | 9.9311e-1 (4.81e-3) $ - $ | 9.9815e-1 (4.06e-4) $ + $ | 9.9680e-1 (6.63e-4) |
| 10 | 19 | 6.7584e-1 (5.61e-2) $ - $ | 9.9732e-1 (9.19e-4) $ - $ | 9.8156e-1 (3.23e-2) $ - $ | 8.9473e-1 (5.33e-2) $ - $ | 9.8488e-1 (5.26e-3) $ - $ | 9.9958e-1 (3.37e-4) $ \approx $ | 9.9962e-1 (2.54e-4) | |
| 15 | 24 | 9.9936e-1 (4.69e-4) $ - $ | 9.9716e-1 (3.23e-4) $ - $ | 9.9931e-1 (6.81e-4) $ - $ | 9.9600e-1 (8.78e-4) $ - $ | 9.6931e-1 (5.50e-2) $ - $ | 9.9999e-1 (7.04e-6) $ + $ | 9.9996e-1 (7.72e-6) | |
| $ +/-/\approx $ | 4/23/3 | 5/24/1 | 1/25/4 | 0/28/2 | 1/28/1 | 5/22/3 | |||
| SDEA vs | IGD | HV | |||||
| $ R^{+} $ | $ R^{-} $ | $ p $-value | $ R^{+} $ | $ R^{-} $ | $ p $-value | ||
| RVEA | 461.0 | 4.0 | 0.000002 | 434.0 | 31.0 | 0.000026 | |
| MOEA/D-AWA | 441.0 | 24.0 | 0.000017 | 412.5 | 52.5 | 0.000198 | |
| RPS-NSGA-II | 430.0 | 35.0 | 0.000047 | 425.0 | 40.0 | 0.000069 | |
| MORA/D-UR | 402.0 | 33.0 | 0.000063 | 461.0 | 4.0 | 0.000002 | |
| MaOEA-PDS | 456.5 | 8.5 | 0.000004 | 434.0 | 31.0 | 0.000031 | |
| HEA | 384.0 | 51.0 | 0.000305 | 365.5 | 69.5 | 0.001283 | |
| Problem | $ m $ | $ D $ | RVEA | MOEA/D-AWA | RPS-NSGA-II | MOEA/D-UR | MaOEA-PDS | HEA | SDEA |
| IMOP1 | 2 | 10 | 1.3871e-1 (6.02e-3) $ - $ | 1.0191e-1 (5.20e-3) $ \approx $ | 1.3700e-1 (3.71e-2) $ - $ | 3.7121e-1 (4.93e-2) $ - $ | 2.5130e-1 (2.32e-2) $ - $ | 3.5259e-1 (1.02e-2) $ - $ | 9.0222e-2 (1.27e-2) |
| IMOP2 | 2 | 10 | 7.3070e-1 (1.44e-1) $ - $ | 1.2196e-2 (4.62e-3) $ - $ | 5.3323e-1 (3.55e-1) $ - $ | 5.2281e-1 (8.74e-2) $ - $ | 7.8495e-1 (5.80e-5) $ - $ | 1.5455e-1 (2.23e-2) $ - $ | 4.6425e-3 (7.53e-5) |
| IMOP3 | 2 | 10 | 8.1834e-3 (1.42e-3) $ - $ | 6.2844e-3 (9.36e-4) $ \approx $ | 8.5002e-3 (6.34e-4) $ - $ | 4.7468e-1 (1.05e-1) $ - $ | 1.2144e-2 (1.60e-3) $ - $ | 4.2638e-2 (1.51e-2) $ - $ | 5.3531e-3 (5.38e-4) |
| IMOP4 | 3 | 10 | 4.4279e-2 (6.47e-3) $ - $ | 1.6830e-2 (1.43e-3) $ - $ | 6.5909e-2 (1.91e-2) $ - $ | 2.8222e-2 (5.84e-3) $ - $ | 9.7195e-3 (1.11e-3) $ - $ | 3.6545e-2 (5.73e-3) $ - $ | 7.4608e-3 (3.43e-4) |
| IMOP5 | 3 | 10 | 6.6704e-2 (2.13e-3) $ - $ | 4.8526e-2 (1.75e-3) $ - $ | 6.8496e-2 (7.75e-3) $ - $ | 5.1696e-2 (4.83e-3) $ - $ | 3.6481e-2 (1.62e-3) $ - $ | 6.2797e-2 (3.36e-3) $ - $ | 3.3979e-2 (1.08e-3) |
| IMOP6 | 3 | 10 | 1.9018e-1 (2.16e-1) $ - $ | 1.1559e-1 (1.91e-1) $ - $ | 5.6896e-2 (6.64e-3) $ - $ | 4.7488e-2 (2.04e-3) $ - $ | 4.7660e-1 (1.94e-1) $ - $ | 1.4029e-1 (1.82e-1) $ - $ | 3.2074e-2 (7.17e-4) |
| IMOP7 | 3 | 10 | 8.2949e-1 (2.87e-1) $ - $ | 6.4672e-1 (4.12e-1) $ - $ | 5.8980e-2 (2.87e-3) $ - $ | 1.3336e-1 (2.34e-1) $ - $ | 9.3555e-1 (1.67e-3) $ - $ | 9.0978e-1 (2.72e-2) $ - $ | 3.5658e-2 (6.98e-4) |
| IMOP8 | 3 | 10 | 1.3487e-1 (2.99e-3) $ - $ | 1.0245e-1 (1.11e-3) $ - $ | 9.8327e-2 (3.51e-3) $ - $ | 8.7488e-2 (1.62e-3) $ - $ | 1.3496e-1 (1.47e-2) $ - $ | 1.9142e-1 (1.53e-1) $ - $ | 7.5254e-2 (2.42e-3) |
| $ +/-/\approx $ | 0/8/0 | 0/6/2 | 0/8/0 | 0/8/0 | 0/8/0 | 0/8/0 | |||
| Problem | $ m $ | $ D $ | RVEA | MOEA/D-AWA | RPS-NSGA-II | MOEA/D-UR | MaOEA-PDS | HEA | SDEA |
| IMOP1 | 2 | 10 | 9.8505e-1 (1.03e-4) $ \approx $ | 9.8642e-1 (8.82e-5) $ + $ | 9.8406e-1 (9.09e-4) $ - $ | 6.6970e-1 (1.08e-1) $ - $ | 9.8193e-1 (1.43e-3) $ - $ | 9.6919e-1 (2.54e-3) $ - $ | 9.8550e-1 (9.05e-4) |
| IMOP2 | 2 | 10 | 2.3134e-1 (6.03e-5) $ - $ | 1.0160e-1 (2.83e-2) $ - $ | 1.3071e-1 (6.80e-2) $ - $ | 9.3322e-2 (2.30e-3) $ - $ | 9.0909e-2 (9.62e-9) $ - $ | 2.1301e-1 (5.95e-3) $ - $ | 2.3168e-1 (3.22e-5) |
| IMOP3 | 2 | 10 | 6.5536e-1 (3.17e-4) $ - $ | 6.5257e-1 (8.66e-4) $ - $ | 6.5257e-1 (7.38e-4) $ - $ | 2.7852e-1 (6.65e-2) $ - $ | 6.5573e-1 (3.54e-3) $ \approx $ | 6.4070e-1 (1.50e-2) $ - $ | 6.5745e-1 (2.15e-4) |
| IMOP4 | 3 | 10 | 4.2570e-1 (2.64e-3) $ - $ | 4.1148e-1 (2.27e-3) $ - $ | 4.0835e-1 (3.48e-3) $ - $ | 4.2179e-1 (2.96e-3) $ - $ | 4.3319e-1 (5.31e-4) $ - $ | 4.1798e-1 (2.77e-3) $ - $ | 4.3436e-1 (1.79e-4) |
| IMOP5 | 3 | 10 | 4.6123e-1 (5.43e-3) $ - $ | 4.3088e-1 (1.08e-2) $ - $ | 4.7988e-1 (1.88e-2) $ \approx $ | 5.0180e-1 (2.24e-2) $ + $ | 4.9271e-1 (1.40e-2) $ + $ | 4.7985e-1 (1.05e-2) $ \approx $ | 4.7432e-1 (6.32e-3) |
| IMOP6 | 3 | 10 | 4.8716e-1 (6.81e-2) $ - $ | 3.8249e-1 (1.87e-1) $ - $ | 5.0737e-1 (3.38e-3) $ - $ | 5.1478e-1 (2.26e-3) $ - $ | 1.3785e-1 (1.68e-1) $ - $ | 4.6991e-1 (6.48e-2) $ - $ | 5.2810e-1 (4.61e-4) |
| IMOP7 | 3 | 10 | 2.1513e-1 (2.05e-1) $ - $ | 1.3956e-1 (1.29e-1) $ - $ | 4.9775e-1 (3.87e-3) $ - $ | 4.5807e-1 (1.41e-1) $ - $ | 9.0920e-2 (1.23e-5) $ - $ | 9.3341e-2 (2.27e-3) $ - $ | 5.2769e-1 (5.24e-4) |
| IMOP8 | 3 | 10 | 4.8617e-1 (1.65e-3) $ - $ | 4.6607e-1 (7.19e-3) $ - $ | 5.3480e-1 (3.13e-2) $ \approx $ | 5.4614e-1 (2.99e-2) $ \approx $ | 4.8785e-1 (1.18e-2) $ - $ | 4.5786e-1 (3.35e-2) $ - $ | 5.5282e-1 (3.41e-2) |
| $ +/-/\approx $ | 0/7/1 | 1/7/0 | 0/6/2 | 1/6/1 | 1/6/1 | 0/7/1 | |||
| Problem | $ m $ | $ D $ | SDEA1 | SDEA2 | SDEA3 | SDEA4 | SDEA5 | SDEA |
| WFG1 | 5 | 14 | 7.2318e-1 (6.27e-3) $ - $ | 9.4742e-1 (6.19e-3) $ - $ | 9.3968e-1 (6.35e-4) $ - $ | 6.7091e-1 (4.29e-3) $ - $ | 8.9150e-1 (4.27e-4) $ - $ | 9.9801e-1 (3.79e-4) |
| 10 | 19 | 8.9180e-1 (2.25e-4) $ - $ | 9.9752e-1 (3.69e-3) $ \approx $ | 9.5308e-1 (1.86e-3) $ - $ | 8.5984e-1 (5.27e-3) $ - $ | 6.9721e-1 (4.81e-4) $ - $ | 9.9963e-1 (3.80e-4) | |
| 15 | 24 | 8.7286e-1 (5.95e-3) $ - $ | 9.8711e-1 (3.02e-2) $ - $ | 9.2462e-1 (6.65e-4) $ \approx $ | 8.3671e-1 (4.22e-3) $ - $ | 9.1062e-1 (3.71e-4) $ - $ | 9.9998e-1 (2.41e-5) | |
| WFG2 | 5 | 14 | 9.9527e-1 (5.95e-4) $ \approx $ | 9.5968e-1 (2.08e-3) $ - $ | 9.9052e-1 (2.71e-3) $ \approx $ | 8.5132e-1 (2.94e-3) $ - $ | 7.0914e-1 (5.22e-4) $ - $ | 9.9366e-1 (7.60e-4) |
| 10 | 19 | 9.2610e-1 (2.83e-3) $ - $ | 9.9485e-1 (1.88e-3) $ \approx $ | 9.6025e-1 (3.73e-3) $ - $ | 7.5916e-1 (5.21e-3) $ - $ | 8.3706e-1 (3.11e-4) $ - $ | 9.9221e-1 (9.41e-4) | |
| 15 | 24 | 9.1029e-1 (3.59e-3) $ - $ | 9.9317e-1 (3.77e-3) $ \approx $ | 9.9051e-1 (4.57e-3) $ \approx $ | 7.8204e-1 (3.95e-3) $ - $ | 8.7618e-1 (2.56e-4) $ - $ | 9.9681e-1 (1.72e-3) | |
| WFG3 | 5 | 14 | 5.2384e-2 (2.57e-2) $ \approx $ | 7.3816e-2 (4.76e-2) $ - $ | 6.1034e-2 (1.52e-2) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 1.4825e-1 (3.29e-2) |
| 10 | 19 | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) | |
| 15 | 24 | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) $ \approx $ | 0.0000e+0 (0.00e+0) | |
| WFG4 | 5 | 14 | 7.8652e-1 (4.19e-3) $ - $ | 8.0753e-1 (2.71e-3) $ \approx $ | 7.6037e-1 (1.02e-2) $ - $ | 7.6052e-1 (3.96e-3) $ - $ | 6.2513e-1 (2.41e-3) $ - $ | 8.0615e-1 (1.22e-3) |
| 10 | 19 | 8.7023e-1 (3.95e-3) $ - $ | 9.3051e-1 (4.15e-3) $ - $ | 8.5127e-1 (3.72e-2) $ - $ | 7.7108e-1 (1.95e-3) $ - $ | 8.4058e-1 (2.11e-3) $ - $ | 9.4963e-1 (2.90e-3) | |
| 15 | 24 | 7.8157e-1 (2.74e-3) $ - $ | 9.8016e-1 (2.43e-3) $ \approx $ | 8.6352e-1 (4.91e-2) $ - $ | 8.8205e-1 (2.33e-3) $ - $ | 8.6073e-1 (1.56e-4) $ - $ | 9.8303e-1 (1.78e-3) | |
| WFG5 | 5 | 14 | 8.5082e-1 (1.43e-3) $ + $ | 7.8365e-1 (1.24e-3) $ - $ | 7.6019e-1 (5.31e-3) $ - $ | 5.3601e-1 (4.28e-3) $ - $ | 4.7201e-1 (2.17e-4) $ - $ | 7.6022e-1 (4.13e-4) |
| 10 | 19 | 8.9681e-1 (1.78e-3) $ \approx $ | 9.0254e-1 (1.38e-3) $ + $ | 8.5597e-1 (5.40e-3) $ - $ | 6.2801e-1 (5.28e-3) $ - $ | 7.0921e-1 (4.71e-4) $ - $ | 8.9832e-1 (6.00e-4) | |
| 15 | 24 | 8.9087e-1 (6.15e-4) $ - $ | 8.7194e-1 (2.83e-3) $ - $ | 8.5701e-1 (4.22e-3) $ - $ | 8.2071e-1 (3.45e-3) $ - $ | 9.0427e-1 (3.18e-4) $ \approx $ | 9.1455e-1 (4.77e-4) | |
| WFG6 | 5 | 14 | 7.2679e-1 (1.37e-2) $ - $ | 7.4503e-1 (1.43e-2) $ \approx $ | 7.6805e-1 (1.47e-2) $ + $ | 4.2131e-1 (4.37e-2) $ - $ | 7.3638e-1 (2.32e-2) $ \approx $ | 7.4043e-1 (1.41e-2) |
| 10 | 19 | 8.6787e-1 (1.69e-2) $ \approx $ | 8.3508e-1 (1.28e-2) $ - $ | 6.8542e-1 (3.64e-2) $ - $ | 6.9415e-1 (3.28e-3) $ - $ | 6.7155e-1 (2.66e-2) $ - $ | 8.6998e-1 (1.36e-2) | |
| 15 | 24 | 8.9551e-1 (1.21e-2) $ \approx $ | 8.8418e-1 (2.80e-2) $ \approx $ | 6.5042e-1 (1.26e-1) $ - $ | 7.3942e-1 (5.28e-1) $ - $ | 6.5218e-1 (4.01e-2) $ - $ | 8.8649e-1 (1.31e-2) | |
| WFG7 | 5 | 14 | 7.2351e-1 (2.51e-3) $ - $ | 8.0052e-1 (2.23e-3) $ - $ | 8.0617e-1 (5.39e-3) $ \approx $ | 6.5035e-1 (4.81e-3) $ - $ | 7.1468e-1 (3.26e-4) $ - $ | 8.0758e-1 (2.98e-4) |
| 10 | 19 | 9.6087e-1 (2.53e-3) $ - $ | 9.6095e-1 (2.04e-3) $ - $ | 8.6218e-1 (2.76e-2) $ - $ | 7.8901e-1 (3.27e-3) $ - $ | 8.9122e-1 (2.16e-3) $ - $ | 9.5879e-1 (1.82e-3) | |
| 15 | 24 | 9.8611e-1 (9.20e-4) $ + $ | 9.8672e-1 (7.25e-4) $ + $ | 9.1340e-1 (2.51e-2) $ - $ | 7.5011e-1 (3.28e-3) $ - $ | 8.9041e-1 (2.75e-4) $ - $ | 9.8483e-1 (1.88e-3) | |
| WFG8 | 5 | 14 | 6.6585e-1 (4.70e-3) $ - $ | 6.6738e-1 (3.69e-3) $ - $ | 6.9475e-1 (6.81e-4) $ \approx $ | 3.7602e-1 (2.81e-3) $ - $ | 4.9130e-1 (4.26e-4) $ - $ | 6.9502e-1 (8.48e-4) |
| 10 | 19 | 8.5285e-1 (1.97e-2) $ - $ | 8.5119e-1 (2.81e-2) $ - $ | 7.7624e-1 (3.74e-2) $ - $ | 5.7165e-1 (3.25e-3) $ - $ | 6.3022e-1 (5.41e-3) $ - $ | 8.6059e-1 (5.74e-3) | |
| 15 | 24 | 9.2307e-1 (3.57e-3) $ + $ | 9.1868e-1 (4.57e-3) $ \approx $ | 9.1706e-1 (5.20e-3) $ \approx $ | 7.2511e-1 (3.70e-3) $ - $ | 9.2908e-1 (4.71e-3) $ - $ | 9.1724e-1 (5.33e-3) | |
| WFG9 | 5 | 14 | 6.5026e-1 (3.14e-3) $ - $ | 7.3908e-1 (2.57e-3) $ - $ | 7.2057e-1 (5.37e-3) $ - $ | 6.4742e-1 (5.96e-3) $ - $ | 6.3968e-1 (4.35e-3) $ - $ | 7.6045e-1 (3.47e-3) |
| 10 | 19 | 8.9267e-1 (5.22e-3) $ + $ | 8.9287e-1 (6.82e-3) $ + $ | 8.3925e-1 (4.68e-2) $ - $ | 6.4472e-1 (4.28e-3) $ - $ | 6.3082e-1 (5.37e-3) $ - $ | 8.7926e-1 (6.82e-3) | |
| 15 | 24 | 8.5781e-1 (6.30e-2) $ - $ | 8.5037e-1 (6.25e-2) $ - $ | 9.1051e-1 (8.25e-3) $ + $ | 4.7027e-1 (6.81e-3) $ - $ | 5.066e-1 (4.27e-2) $ - $ | 8.6638e-1 (1.92e-2) | |
| $ +/-/\approx $ | 3/17/7 | 3/14/10 | 2/16/9 | 0/24/3 | 0/22/5 | |||
| Problem | $ m $ | $ D $ | RVEA | MOEA/D-AWA | RPS-NSGA-II | MOEA/D-UR | MaOEA-PDS | HEA | SDEA |
| CSIDP | 3 | 7 | 5.7035e-1 (1.72e-3) $ - $ | 3.7053e-1 (2.52e-2) $ - $ | 5.0271e-1 (2.17e-2) $ - $ | 5.1019e-1 (1.34e-3) $ - $ | 4.5127e-1 (4.32e-2) $ - $ | 6.3021e-1 (2.85e-3) $ \approx $ | 6.2908e-1 (1.35e-3) |
| WRMP | 5 | 3 | 4.1208e-1 (2.53e-2) $ - $ | 5.23721e-1 (1.59e-2) $ - $ | 3.9806e-1 (1.75e-2) $ - $ | 4.3253e-1 (2.98e-3) $ - $ | 5.7025e-1 (2.35e-2) $ - $ | 5.3920e-1 (2.12e-2) $ - $ | 7.5019e-1 (1.95e-2) |
| $ +/-/\approx $ | 0/2/0 | 0/2/0 | 0/2/0 | 0/2/0 | 0/2/0 | 0/1/1 | |||
