Tuning extreme learning machine by an improved electromagnetism-like mechanism algorithm for classification problem

1.   Introduction

A metaheuristic optimization algorithm is a novel population-based global search algorithm, and is more suitable for solving complex problems ^[1,2]. Rao et al. proposed a teaching-learning-based optimization (TLBO) algorithm to solve large-scale optimization problems. The simulation results of the standard test function showed that the TLBO algorithm effectively solves complex optimization problems ^[3]. To solve a complex constrained optimization problem, Sayed E et al. proposed a decomposition evolutionary algorithm ^[4]. Mohapatra et al. proposed a competitive swarm optimizer algorithm ^[5]. To overcome the shortcoming of particle swarm optimization (PSO) falling easily into a local optimum, an improved quantum PSO algorithm with the cultural gene algorithm and memory mechanism was proposed to solve continuous nonlinear problems ^[6,7]. Ali et al. presented a multi-population differential evolution global optimization algorithm ^[1]. Ant colony optimization, proposed by Ismkhan, has been applied to solving complex problems ^[8]. Using the symbiotic organism search algorithm for fractional fuzzy controllers ^[9] and so on.

In an industrial control system, the proportional integral derivative (PID) controller has been widely applied. It accounts for more than 90% of the actual control system ^[10]. The PID parameter tuning problem proposed by Ziegler and Nichols has caused extensive concern. However, the traditional PID parameter tuning method has the following problems: The control performance index is not ideal and, typically, the method has a large overshoot and long adjustment time. The control effect of an intelligent optimization algorithm in PID parameter tuning is better than that of the traditional tuning method, and it can avoid some shortcomings of traditional methods ^[11]. Jiang et al. proposed a PID tuning premature with a genetic algorithm (GA) to enhance the search and convergence speed, but there were problems of premature convergence and parameter dependence ^[12]. Yu et al. proposed seeker search algorithm optimization PID controller parameters; improved the control precision of the system; accelerated the response speed and robustness of the system; and optimized the optimal parameters for the control system PID, but the optimization formula complex, need to set more parameters ^[13]. P. B. de Moura Oliveira, et al. designed Posicast PID control systems using a gravitational search algorithm (GSA) ^[14]. Guo-qiang Zeng et al. designed multivariable PID controllers using real-coded population-based extremal optimization ^[15]. A. Belkadi et al. proposed a PSO-based approach on the robust PID adaptive controller for exoskeletons ^[16]. M. Gheisarnejad proposed an effective hybrid harmony search (HS) and cuckoo optimization algorithm-based fuzzy PID controller for load frequency control ^[17]. Amal Moharam, et al. designed an optimal PID controller using hybrid differential evolution and PSO with an aging leader and challengers ^[18].

The spotted hyena optimizer (SHO) ^[19] is a novel intelligence algorithm proposed by Dhiman and Kumar in 2017. It was inspired by the social and collaborative behavior of spotted hyenas, which exist in nature. Spotted hyenas typically perform four processes: Search, encirclement, hunt, and attack prey. The SHO has the characteristics of simple, easy to implement programming and adjust the parameters set less features. Since the SHO was proposed, there have been various improved versions of the SHO algorithm. For example, N. Panda, et al. used an improved SHO (ISHO) with space transformational search to train a pi-sigma higher-order neural network ^[20]. H. Moayedi et al. proposed using the SHO and ant lion optimization to predict the shear strength of soil ^[21]. Q. Luo, et al. proposed using the SHO with lateral inhibition for image matching ^[22]. G. Dhiman et al. proposed a multi-objective optimization algorithm for engineering problems ^[23] and used the SHO to solve the nonlinear economic load power dispatch problem ^[24]. Xu Y, et al. proposed an enhanced moth-flame optimizer with a mutation strategy for global optimization ^[25].

In function optimization and engineering optimization, it has been proved that the performance of the SHO is superior to that of the grey wolf optimizer (GWO), binary GWO ^[26], PSO, moth-flame optimization (MFO), multi-verse optimizer, sine cosine algorithm (SCA), GSA, GA, HS, Harris hawks optimization ^[27], bacterial foraging optimization ^[28], and flower pollination algorithm ^[29] in terms of precision and the convergence speed ^[22]. In this paper, an ISHO algorithm is applied to solve PID parameter problems in an automatic voltage regulator (AVR).

The remainder of the paper is organized as follows: The basic SHO algorithm is presented in Section 2. In Section 3, the ISHO is introduced. In Section 4, the ISHO is proposed to optimize PID parameters and compared with well-known algorithms. Finally, conclusions are provided in Section 5.

2.   Spotted hyena optimizer

The social relationships and habits of animals are the source of inspiration for our work. This social behavior is also present in the spotted hyena, whose scientific name is Crocuta. According to llany et al. ^[30], spotted hyenas typically live in groups, with as many as 100 group members, and they have mutual trust and interdependence. The communication between spotted hyenas is typically posed, and given a special signal, they track prey using sight, hearing, and smell. There are four main steps for a spotted hyena to attack prey: Search for prey, encircle prey, hunt prey, and attack prey. To generate a mathematical model for encircling, the equations are as follows ^[31]:

where $t$ is the current iteration, $\vec B$ and $\vec E$ are the coefficient vectors, ${\vec X_p}$ is the position vector of the prey, $\vec X$ is the position vector, $\vec D$ is the distance between the prey and spotted hyena, and ${\rm{||}} \bullet {\rm{||}}$ represents the absolute value.

The coefficient vectors $\vec B$ and $\vec E$ are calculated as follows:

where $t = 1, 2, 3, \cdots, T$ to balance exploration and exploitation, $\vec h$ linearly decreases from 5 to 0, and $r{\vec d_1}$ and $r{\vec d_2}$ are random vectors in [0, 1].

To define the spotted hyena behavior mathematically, the best search agent represents the location of the prey. The other search agents move toward the best search agent and save the best solutions obtained thus far to update their positions. The mathematical model can be formulated as follows:

where ${\vec X_h}$ is the position of the best spotted hyena, ${\vec X_k}$ is the position of other spotted hyena, N is the number of spotted hyenas, $\vec M$ is a random vector in [0.5, 1], $nos$ is the number of solutions, $coun{t_{nos}}$ is the count of all candidate solutions, and ${\vec C_h}$ is a cluster of N optimal solutions.

For spotted hyenas in the attack prey stage, to determine the optimal solution, it is necessary to continuously reduce the value of $\vec h$, where $\vec h$ is the step size that the spotted hyena takes to attack prey, and it is clear that, when looking for prey, the spotted hyena continues to increase the number of steps gain steps. The formulation for attacking prey is

where $\vec X(t + 1)$ is the position of the current solution and $t$ is the number of iterations. The SHO allows its agents to update their positions in the direction of the prey.

The parameters $\vec B$ and $\vec E$ oblige the SHO algorithm to explore and exploit the search space. As $\vec B$ decreases, half the iterations are dedicated to exploration (when $\left| {\vec E} \right| > 1$) and the remainder are dedicated to exploitation (when $\left| {\vec E} \right| < 1$) ^[32]. From the above, the $\vec B$ vector contains random values in [0, 2]. This component provides random weights for prey to stochastically emphasize ($\vec B > 1$) or deemphasize ($\vec B < 1$) the effect of prey in defining the distance in Eq (2.3). This helps the random behavior of the SHO to increase during the course of optimization, and favors exploration and local optima avoidance. The pseudocode of the SHO algorithm is as above.

3.   Improved spotted hyena optimizer

3.1.   Population initialization based on opposite learning

Haupt et al. found that the initial population affects the algorithm’s accuracy and convergence speed ^[32]. The better than initial population can lay the foundation for the global search of the SHO algorithm ^[21]. However, without any prior knowledge of the global optimal solution of the problem, the SHO algorithm typically adopts a random method when generating the initial search agent, which thus affects the search efficiency. The opposition learning strategy is a new technology that has emerged in the field of intelligent computing. So far, the opposite learning strategy has been successfully applied to swarm intelligent algorithms, such as PSO, HS, and DE algorithms ^[33,34]. In this paper, the opposite learning strategy is embedded into the SHO for initialization.

Definition: Opposition-based ^[35]. Suppose$X$exists in $\left[{l, u} \right]$. The opposite point is $\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{X} = l + u - X$. Let $X = ({X_1}, {X_2}, \cdots, X{}_d)$ be a point in d-dimensional space, where ${X_1}, {X_2}, \cdots, {X_d} \in R$ and ${X_i} \in \left[{{l_i}, {u_i}} \right]$ $\forall i \in \left\{ {1, 2, \cdots, d} \right\}$. The opposition-based $\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{X} = \left({{{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{X} }_1}, {{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{X} }_2}, \cdots, {{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{X} }_d}} \right)$ is completely defined by its components:

According to the above definition, the specific steps for using the opposite learning strategy to generate the initial population are as above.

3.2.   Nonlinear variation convergence factor

Similar to other group intelligent optimization algorithms based on population iteration, it is crucial for the SHO to coordinate its exploration and exploitation capabilities. During exploration, groups need to detect a wider search area and avoid the SHO algorithm becoming stuck in a local optimum. The exploitation capacity mainly uses the group’s existing information to search some local solution’s areas of the solution. The convergence rate of the SHO algorithm has a decisive influence. Clearly, robustness and fast convergence are achieved only when the SHO algorithm improves the coordination of the exploration and exploitation capabilities.

According to ^[21], the SHO algorithm’s exploration and exploitation abilities depend on the change of the convergence factor $\vec h$. The larger the convergence factor $\vec h$, the better the global search ability and the more likely the SHO algorithm avoids falling into a local optimum. The smaller the convergence factor $\vec h$, the stronger the local search ability, which speeds up the convergence of the SHO algorithm. However, in the basic SHO algorithm, the convergence factor $\vec h$ decreases linearly from 5 to 0 as the number of iterations increases. The linear decreasing strategy of the convergence factor $\vec h$ has a good global search ability in the early stages of the algorithm, but the convergence speed is slow. In the latter part of the algorithm to speed up the convergence rate, but easy to fall into a local optimum, particularly in multimode functions problems. Therefore, in the evolutionary search process, the convergence factor $\vec h$ with the number of iterations linearly decreasing strategy cannot fully reflect the actual optimization of the search process in the SHO algorithm ^[36]. In fact, the SHO is expected to have a strong global search ability in the pre-search period while maintaining a fast convergence rate. Additionally, Enns, et al. and Zeng, et al. found that performance improved if the control parameter was chosen as a nonlinearly decreasing quantity rather than using a linearly decreasing strategy ^[30,32]. Thus, the control parameter $\vec h$ is modified as follows:

where $t$ is the current iteration, $Max\_iteration$ is the maximum number of iterations, $u$ is the nonlinear modulation index, and ${\vec h_{initial}}$ and ${h_{final}}$ are the initial value and final value of control parameter $\vec h$, respectively. According to ^[32], when ${\vec h_{initial}}$ is set to 5, ${h_{final}}$ becomes 0.

Figure 1 shows typical control parameter $\vec h$ variations with iterations for different values of $u$. We conducted several SHO experiments with a nonlinear modulation index $u$ in the interval (0, 2.0). On average, the results are better than those of existing algorithms: The larger the value of u (u > 2.0), the greater the failed convergence rate.

3.3.   Diversity mutation operation

Similar to other population-based intelligent optimization algorithms, in the late iteration of the SHO, all the spotted hyenas move closer to the optimal individual region, which results in a reduction of population diversity. In this case, if the current optimal individual is the local optimal, then the SHO algorithm falls into a local optimum. This is also an inherent characteristic of other group intelligent optimization algorithms. To reduce the probability of premature convergence for the SHO algorithm, in this paper, a diversity mutation operation is performed on the current optimal spotted hyena individuals. The steps are as follows:

Assume that an individual ${X_i} = ({x_{i1}},{x_{i2}}, \cdots ,{x_{id}})$ of the spotted hyena species selects one element ${x_k}(k = 1,2, \cdots ,d)$ randomly from the individual with a probability of and randomly generates a real number in the range $[{l_i},{u_i}]$ instead of the element from the individual ${X_i}$, thus producing a new individual. $X_i^' = (x_{i1}^',x_{i2}^', \cdots ,x_{id}^'$ The variation mutation operation is

where ${l_i}$ and u are the upper and lower bounds of the variable x, respectively, and $\lambda \in [0, 1]$ is a random. The ISHO optimizer steps are presented in Algorithm 3.

4.   Improved spotted hyena optimizer for PID parameters in an AVR system

4.1.   Description and modeling of an AVR system

Providing constancy and stability at rated voltage levels in electricity, the network is also one of the main problems in power system control. If the rated voltage level deviates from this value, then the performance degrades and the life expectancy reduces. Another important reason for this control is true line loss, which depends on the real and reactive power flow. The reactive power flow is largely dependent on the terminal voltage of the power system. However, it is necessary to reduce the loss caused by the solid line by controlling the rated voltage level. To solve these control problems, an AVR system is applied to power generation units ^[36]. The role of the AVR is to maintain the terminal voltage of the synchronous alternator at the rated voltage value.

4.1.1.   PID controller

Using the PID controller to improve the dynamic response while reducing or eliminating the steady-state error, the derivative controller adds a finite zero to the open-loop plant, which enables the improvement of the transient response. The PID controller transfer function is

A simple AVR system has four parts: Amplifier, exciter, generator, and sensor. The mathematical transfer function of the above four components is considered as linear and time constant. To analyze the dynamic performance of an AVR, the transfer functions of these components are in ^[37,38].

4.1.2.   Amplifier model

The amplifier model is represented by a gain ${K_A}$ and time constant ${\tau _A}$. The transfer function is given by

where the range of ${K_A}$ is [10,400] and the amplifier time constant ranges from 0.02–0.1 s.

4.1.3.   Exciter model

The transfer function of an exciter is modeled by a gain ${K_E}$ and time constant ${\tau _E}$, and given by

where ${K_E}$ is typically in the range [10,400] and the time constant ${\tau _E}$ is in the range 0.5–1.0 s.

4.1.4.   Generator model

The generator model is represented by a gain ${K_G}$ and time constant ${\tau _G}$. The transfer function is given by

where ${K_R}$ is in the range [0.7, 1.0] and ${\tau _R}$ is in the range 1.0–2.0 s. The generator gain ${K_R}$ and time constant ${\tau _R}$ are load dependent.

4.1.5.   Sensor model

The sensor is modeled by a gain ${K_R}$ and time constant ${\tau _R}$. The transfer function is given by

where ${K_R}$ is in the range [10,400] and ${\tau _R}$ is in the range 0.001–0.06 s.

The complete transfer function model of the AVR system is given in Figure 2. In the work of Gozde and Taplanmacioglu ^[37], the parameters of the AVR system were ${K_A} = 10.0$, ${\tau _A} = 0.1$, ${K_E} = 1.0$, ${\tau _E} = 0.4$, ${K_G} = 1.0$, ${\tau _G} = 1.0$, ${K_R} = 1.0$, and ${\tau _R} = 0.01$.

The transformer function of the AVR system with the above parameters is

To improve the dynamic response of the AVR system and maintain the terminal voltage at 1.0 pu, a PID controller is included, as shown in Figure 2.

With the PID controller, the transfer function of the AVR system of Figure 2 becomes

4.2.   PID controller based on the ISHO

An AVR system with a PID controller tuned by the ISHO algorithm is shown in Figure 2. The gains of the PID controller are regulated by the ISHO algorithm. If the proportional gain is too high, then the system becomes unstable and the proportional gain becomes too low, which results in a larger error and lower sensitivity. For an AVR system, the ranges commonly used in the literature are [0.0, 1.5] and [0.2, 2.0] in ^[38,39]. To increase the search space for better optimization gains, the resulting lower and upper bounds are chosen to be 0.01 and 2, respectively.

To improve the performance control, the optimal PID parameters use the ISHO. The maximum overshoot, rise time, steady state error as a typical time domain analysis method index, and integral time multiplied by the absolute error (ITAE) are considered as control performance indicators in the design. The ITAE is the objective function with time value:

An ISHO-PID controller is presented for searching the optimal or near optimal controller parameters ${K_p}$, ${K_i}$, and ${K_d}$ using the ISHO algorithm. Each individual K contains three members: ${K_p}$, ${K_i}$, and ${K_d}$. The searching procedures of the proposed ISHO-PID controller are presented in Algorithm 4. The three controller parameters set in the algorithm are shown in Table 1.

4.3.   Experimental results and analysis

The ISHO algorithm was applied to optimize the PID controller for an AVR system and determine a set of optimal gains that minimize the value of the objective function. To prove the superiority of the ISHO algorithm, we compared the ISHO with other algorithms that contain the SHO ^[19], GWO ^[40], PSOGSA ^[41], FPA ^[42], and SCA ^[43]. The results showed that the ISHO performed better than the other algorithms. All the parameters set in the algorithms are given in Table 2.

Setting the six algorithms’ parameters, we obtained the best parameter values for each algorithm to optimize the PID parameters using the population size of 50, 20 iterations, and 20 runs independently. "Best" is the optimal fitness value, "Worst" is the worst fitness value, "Mean" is the mean fitness value, and "Std." is the standard deviation. Table 3 shows that the best fitness value of ISHO was significantly better than those of the other algorithms, and the standard error of the ISHO was the smallest. It is thus proved that the ISHO is better than the standard SHO and other algorithms (SCA, FPA, PSOGSA, and GWO) in terms of obtaining the optimal PID parameters.

In Table 2, we label the optimal PID parameters with the optimal fitness values, and the minimum standard error is indicated by the black bold line. The table shows that, although the PSOGSA algorithm is also a hybrid of the PSO and GSA algorithms, the effect of optimally searching for PID parameters was still not as good as that of the ISHO algorithm.

The transient and steady-state behavior of the system can be analyzed from the transient analysis of the ISHO optimized PID controller in the AVR system, as shown in Figure 4. For comparison, responses for the SCA, FPA, PSOGSA, GWO, and SHO algorithms are shown in Table 8. The figure shows that the maximum overshoot for the ISHO algorithm is 5% less than that of the SCA algorithm, 3.5% less than that of the FPA algorithm, 4.82% less than that of the GWO algorithm, and 6.8% less than that of the SHO algorithm. The peak time for the ISHO algorithm is more than that of the SHO, GWO, PSOGSA, FPA, and SCA algorithms. The maximum overshoot and settling time for the ISHO algorithm are better than those of the SHO, GWO, PSOGSA, FPA, and SCA algorithms, and are major factors for comparing the stability analysis of systems.

The convergence characteristics are shown in Figure 4. The figure shows that the ISHO algorithm’s fitness value decreases the fastest compared with those of the other algorithms. This shows that the ISHO algorithm has a strong global search capability and higher precision. The ISHO algorithm is considered as an optimization of the PID controller parameters in the AVR system, which has promising potential applications.

5.   Conclusions

The SHO was inspired by the social behavior of a spotted hyena swarm. Its mathematical model is relatively simple, but the control parameter directly affects the balance between the global search ability and local search ability in the SHO algorithm. Based on the analysis of the above characteristics of the SHO, in this paper, a nonlinear adjustment strategy was adopted for the control parameters and the mutation strategy was used to deal with the update of the intelligent individual position. The performance of the improved algorithm was verified by a simulation. The ISHO quickly approached the theoretical value and significantly improved the convergence speed and optimization efficiency. The ISHO algorithm was used to determine the parameters of the PID controller for an AVR system. It is clear from the results that the proposed ISHO algorithm avoided the shortcoming of the premature convergence of the SHO, GWO, PSOGSA, FPA, and SCA algorithms and obtained global solutions with better computation efficiency.

Acknowledgments

This work was supported by the Project of China University of Political Science and Law Research Innovation under Grant No. 10818441 and the Young Scholar Fund of China University of Political Science and Law under Grant No. 10819144. We thank Maxine Garcia, PhD, from Liwen Bianji, Edanz Group China (www.liwenbianji.cn/ac) for editing the English text of a draft of this manuscript.

Conflict of Interests

The authors declare no conflict of interest.