Defect detection in code characters with complex backgrounds based on BBE

1.   Introduction

Permutation Entropy (PE) ^[6] is probably becoming one of the most successful complexity estimators in the recent years. The number of works based on this measure is sky-rocketing ^[38], arguably due its simplicity, robustness, and ability to capture the underlying dynamics of the time series under analysis.

PE has already been applied in a diversity of fields, and other new ones will surely emerge due its versatility. In medicine, PE has been used in practically all its specialities. For example, in cardiology, it has been applied to heart rate variability data series for sleep breathing pause detection ^[29], to classify emotional changes ^[33], to assess a possible cardiac autonomic neuropathy ^[9], or to find out if atrial fibrillation is stochastic or deterministic ^[3]. Neurology is also a medical field where PE has been extensively exploited. For example, the paper ^[28] describes a method based on PE to track anaesthetic induced changes using electroencephalograms (EEG). In the same context, other works have used PE to segment sleep stages ^[27]. PE and the EEG can even be used to detect epileptic seizures ^[35]. Other medical specialities have not been so widely exploited yet, but there are significant studies already. In internal medicine, PE has been used to classify temperature records as healthy or febrile–prone patients ^[13], or to analyse blood glucose time series to anticipate the possible development of a diabetes. In physical medicine and rehabilitation, there are many studies based on PE analysis of gait data, as in ^[37], where authors successfully classified normal and pathological gait using PE. There are other scientific and technical fields that have benefited from the capabilities of PE. In mechanics, PE is being used as a fault diagnosis tool ^[32,18]. PE is also becoming a popular tool in econometrics applications ^[42,39,21]. Finally, PE has had its place in time series analysis related to climate data ^[19].

Since its conception, there are two limitations that are frequently considered when working with PE: the ordinal ambiguity of equal values in subsequences ^[15,41], and the lack of information related to the sample differences in amplitude ^[17]. In the first case, consider, for example, two subsequences drawn from a time series, $\mathbf{x}_{i} = \left\{1, 3, 2\right\}$ and $\mathbf{x}_{j} = \left\{1, 3, 3\right\}$. If each sample in a subsequence is assigned a value corresponding to its position, starting at 0, $\mathbf{x}_{i}$ and $\mathbf{x}_{j}$ could be re-written as $\mathbf{x}_{i} = \left\{1_{0}, 3_{1}, 2_{2}\right\}$ and $\mathbf{x}_{j} = \left\{1_{0}, 3_{1}, 3_{2}\right\}$, where the sample sub–indices account for the position within the subsequence. If those subsequences are ordered in ascending order, as it is one of the first steps in PE calculation, $\mathbf{x}_{i}$ results in $\mathbf{y}_{i} = \left\{1_{0}, 2_{2}, 3_{1}\right\}$ (order $\left\{0, 2, 1\right\}$), but $\mathbf{x}_{j}$ can result both in $\mathbf{y}_{j} = \left\{1_{0}, 3_{1}, 3_{2}\right\}$ (order $\left\{0, 1, 2\right\}$) or $\mathbf{y}_{j} = \left\{1_{0}, 3_{2}, 3_{1}\right\}$ (order $\left\{0, 2, 1\right\}$). As a consequence, the ordinal patterns can be incorrectly assigned and PE computation be skewed. Some methods have been proposed in the literature to account for these ambiguities ^[7,4], and their influence has been heuristically characterised in ^[15].

As for the second case of differences in amplitude, let's consider again two subsequences and their orders: $\mathbf{x}_{i} = \left\{1_{0}, 3_{1}, 2_{2}\right\}$ and $\mathbf{x}_{j} = \left\{1_{0}, 30_{1}, 29.9_{2}\right\}$. If sorted, the resulting order in both cases is $\left\{0, 2, 1\right\}$, namely, they provide the same information to PE, despite having very different amplitude values that could be also related to the dynamics of the time series. Other methods have also been proposed recently to address this possible weakness of PE, such as Weighted–Permutation Entropy (WPE) ^[17], Fine–Grained Permutation Entropy (FGPE) ^[34], and Amplitude–Aware Permutation Entropy (AAPE) ^[4], among others. In this case, there is no comparative study to assess the real effectiveness of these approaches, and the practical influence of amplitude differences has not been characterized yet, the goal of the present paper.

This characterization study will specifically test the performance of the following methods: standard PE, WPE, FGPE, and AAPE. The experimental dataset was chosen carefully to ensure a representative variation of time series properties and features. This data set was drawn from publicly available databases for replication and comparison purposes. The experiments will consist on assessing the classification performance of all the methods under analysis, and then compare the classification performance achieved. The results confirmed that a combination of ordinal and amplitude information, the whole, could achieve a higher classification accuracy than each type of information considered independently, its parts, PE (ordinal information) and Sample Entropy (SampEn, amplitude information) ^[23].

2.   Materials and method

2.1.   Permutation entropy

PE was proposed in the well known seminal paper by Bandt and Pompe ^[6]. This method was devised to estimate the complexity of a time series based on the relative frequency of the ordinal patterns found. PE inputs are a time series $\mathbf{x}$ of length $N$, $\mathbf{x} = \left\{x_{0}, x_{1}, \ldots, x_{N-1}\right\}$, and an embedded dimension $m > 2$. All the possible $N-(m-1)$ subsequences, starting at index $j$, with $0\leq j < N-m+1$, of length $m$ are then extracted from vector ${\mathbf{x}}$, termed $\mathbf{x}_j^{m} = \left\{x_{j}, x_{j+1}, \ldots, x_{j+m-1}\right\}$. Initially, each $\mathbf{x}_j^{m}$ has a default sample order given by the indices of the samples taken from initial index $j$, that is, $\boldsymbol{\pi}^{m} = \left\{0, 1, \ldots, m-1\right\}$. When $\mathbf{x}_j^{m}$ is sorted in ascending order, a new order vector emerges $\boldsymbol{\pi}_{j}^{m} = \left\{\pi_{0}, \pi_{1}, \ldots, \pi_{m-1}\right\}$, such that $x_{j+\pi_{0}}\leq x_{j+\pi_{1}}\leq x_{j+\pi_{2}} \ldots \leq x_{j+\pi_{m-1}}$. There are potentially $m!$ different ordinal patterns of length $m$, termed $\boldsymbol{\Pi}_{i}^{m}$, with $0 \leq i < m!$. All these patterns can be computed in advance, or added dynamically to a list when they are first found. Each time a $\boldsymbol{\pi}_{j}^{m}$ matches a $\boldsymbol{\Pi}_{i}^{m}$, a motif counter $c_{i}$ is incremented, being all these counters members of a vector $\mathbf{c}$ of length $m!$. Finally, the relative frequencies of each possible pattern can be computed and stored in another vector $\mathbf{p}$, whose non–zero members provide the final PE result as:

The steps to compute PE are described in detail in Algorithm 1.

For example, let $\mathbf{x}$ be the sequence $\left\{-0.45\right.$, 1.9, 0.87, -0.91, 2.3, 1.1, 0.75, 1.3, -1.6, 0.47, -0.15, 0.65, 0.55, -1.1, $\left. 0.3 \right\}$ (a random sequence with mean 0 and standard deviation 1.0). In order to compute PE$\left(\mathbf{x}, m, N\right)$ with $N = 15$, and taking $m = 3$, the following $m! = 6$ ordinal patterns of length 3 have to be considered first: $\boldsymbol{\Pi}_{0}^{3} = \left\{0, 1, 2\right\}$, $\boldsymbol{\Pi}_{1}^{3} = \left\{0, 2, 1\right\}$, $\boldsymbol{\Pi}_{2}^{3} = \left\{1, 0, 2\right\}$, $\boldsymbol{\Pi}_{3}^{3} = \left\{2, 0, 1\right\}$, $\boldsymbol{\Pi}_{4}^{3} = \left\{1, 2, 0\right\}$, $\boldsymbol{\Pi}_{5}^{3} = \left\{2, 1, 0\right\}$.

Then, the $N-m+1 = 13$ subsequences that can be drawn from $\mathbf{x}$ are $\mathbf{x}_0^{3} = \left\{-0.45, 1.9, 0.87\right\}$, $\mathbf{x}_1^{3} = \left\{1.9, 0.87, -0.91\right\}$, $\mathbf{x}_2^{3} = \left\{0.87, -0.91, 2.3\right\}$, $\mathbf{x}_3^{3} = \left\{-0.91, 2.3, 1.1\right\}$, $\mathbf{x}_4^{3} = \left\{2.3, 1.1, 0.75\right\}$, $\mathbf{x}_5^{3} = \left\{1.1, 0.75, 1.3\right\}$, $\mathbf{x}_6^{3} = \left\{0.75, 1.3, -1.6\right\}$, $\mathbf{x}_7^{3} = \left\{ 1.3, -1.6, 0.47\right\}$, $\mathbf{x}_8^{3} = \left\{-1.6, 0.47, -0.15\right\}$, $\mathbf{x}_9^{3} = \left\{0.47, -0.15, 0.65\right\}$, $\mathbf{x}_{10}^{3} = \left\{-0.15, 0.65, 0.55\right\}$, $\mathbf{x}_{11}^{3} = \left\{ 0.65, 0.55, -1.1\right\}$, $\mathbf{x}_{12}^{3} = \left\{0.55, -1.1, 0.3\right\}$. All these subsequences have an associated ordinal pattern $\boldsymbol{\pi}^{3} = \left\{0, 1, 2\right\}$ initially.

When the previous subsequences are ordered, the results are: $\mathbf{y}_0^{3} = \left\{-0.45, 0.87, 1.9\right\}$, with $\boldsymbol{\pi}_{0}^{3} = \left\{0, 2, 1\right\}$; $\mathbf{y}_1^{3} = \left\{-0.91, 0.87, 1.9\right\}$, with $\boldsymbol{\pi}_{1}^{3} = \left\{2, 1, 0\right\}$; $\mathbf{y}_2^{3} = \left\{-0.91, 0.87, 2.3\right\}$, with $\boldsymbol{\pi}_{2}^{3} = \left\{1, 0, 2\right\}$; $\mathbf{y}_3^{3} = \left\{-0.91, 1.1, 2.3\right\}$, with $\boldsymbol{\pi}_{3}^{3} = \left\{0, 2, 1\right\}$; $\mathbf{y}_4^{3} = \left\{0.75, 1.1, 2.3\right\}$, with $\boldsymbol{\pi}_{4}^{3} = \left\{2, 1, 0\right\}$; $\mathbf{y}_5^{3} = \left\{0.75, 1.1, 1.3\right\}$, with $\boldsymbol{\pi}_{5}^{3} = \left\{1, 0, 2\right\}$; $\mathbf{y}_6^{3} = \left\{-1.6, 0.75, 1.3\right\}$, with $\boldsymbol{\pi}_{6}^{3} = \left\{2, 0, 1\right\}$; $\mathbf{y}_7^{3} = \left\{-1.6, 0.47, 1.3\right\}$, with $\boldsymbol{\pi}_{7}^{3} = \left\{1, 2, 0\right\}$; $\mathbf{y}_8^{3} = \left\{-1.6, -0.15, 0.47\right\}$, with $\boldsymbol{\pi}_{8}^{3} = \left\{0, 2, 1\right\}$; $\mathbf{y}_9^{3} = \left\{-0.15, 0.47, 0.65\right\}$, with $\boldsymbol{\pi}_{9}^{3} = \left\{1, 0, 2\right\}$; $\mathbf{y}_{10}^{3} = \left\{-0.15, 0.55, 0.65\right\}$, with $\boldsymbol{\pi}_{10}^{3} = \left\{0, 2, 1\right\}$; $\mathbf{y}_{11}^{3} = \left\{ -1.1, 0.55, 0.65\right\}$, with $\boldsymbol{\pi}_{11}^{3} = \left\{2, 1, 0\right\}$ and $\mathbf{y}_{12}^{3} = \left\{-1.1, 0.3, 0.55\right\}$, with $\boldsymbol{\pi}_{12}^{3} = \left\{1, 2, 0\right\}$.

All the $\boldsymbol{\pi}_{j}^{3}$ ordinal patterns have to be compared with the pattern templates in $\boldsymbol{\Pi}^{3}$. Thus, $\boldsymbol{\Pi}_{0}^{3} = \left\{0, 1, 2\right\}$ is not found in $\boldsymbol{\pi}_{j}^{3}$. $\boldsymbol{\Pi}_{1}^{3} = \left\{0, 2, 1\right\}$ is found 4 times. $\boldsymbol{\Pi}_{2}^{3} = \left\{1, 0, 2\right\}$ is found 3 times. $\boldsymbol{\Pi}_{3}^{3} = \left\{2, 0, 1\right\}$ is found once. $\boldsymbol{\Pi}_{4}^{3} = \left\{1, 2, 0\right\}$ is found twice, and $\boldsymbol{\Pi}_{5}^{3} = \left\{2, 1, 0\right\}$ appears three times. Therefore, $\mathbf{c} = \left\{0, 4, 3, 1, 2, 3\right\}$, and $\mathbf{p} = \left\{0, 0.31, 0.23, 0.08, 0.15, 0.23\right\}$, from which PE$\left(\mathbf{x}, 3, 15\right) = -0.31\log_2 0.31-0.23\log_2 0.23-0.08\log_2 0.08-0.15\log_2 0.15-0.23\log_2 0.23 = 2.20$.

2.2.   Weighted–permutation entropy

WPE was introduced in ^[17]. It was devised as an improvement over PE to account for the variability of amplitudes that result in the same motif. For example, given three subsequences $\mathbf{x}_i^{3} = \left\{1, 2, 3\right\}$, $\mathbf{x}_j^{3} = \left\{1, 2, 3000\right\}$, and $\mathbf{x}_k^{3} = \left\{1, 2000, 2001\right\}$, they result in the same ordinal pattern $\boldsymbol{\pi}_{ijk}^{3} = \left\{0, 1, 2\right\}$, despite their clear amplitude and even shape differences. The basic idea is to apply a correcting factor or weight to the relative frequencies that takes into account sample variability, not only order. These weights $w_{j}$ are added prior to computing such relative frequencies, and are given by $w_{j} = \frac{1}{m} \sum\limits_{k = 0}^{m-1}\left(x_{j+k}-\overline{X}_{j}^{\; m}\right)^{2}$, where $\overline{X}_{j}^{\; m}$ is the arithmetic mean of $\mathbf{x}_j^{m}$, and $W$ will be the new denominator instead of $N-m-1$, with $W = \sum\limits_{j = 0}^{N-m}w_{j}$. The steps to compute WPE are described in detail in Algorithm 2.

Using the same example as for PE, the weight for each subsequence (the variance of each $\mathbf{x}_j^{m}$) would be as follows: $w_0 = 0.925$, $w_1 = 0.227$, $w_2 = 1.724$, $w_3 = 1.754$, $w_4 = 0.441$, $w_5 = 0.052$, $w_6 = 1.582$, $w_7 = 1.487$, $w_8 = 0.752$, $w_9 = 0.117$, $w_{10} = 0.127$, $w_{11} = 0.644$, $w_{12} = 0.527$, with $W = 10.359$. The ordinal patterns found are the same as for PE, and therefore each weight has to be added to the corresponding relative frequency, $\mathbf{p} = \left\{0, \frac{3.558}{10.359}, \frac{1.893}{10.359}, \frac{1.582}{10.359}, \frac{2.014}{10.359}, \frac{1.312}{10.359}\right\}$, from which WPE$\left(\mathbf{x}, m, N\right) = -0.3435\log_2 0.3435-0.1827\log_2 0.1827-0.1527\log_2 0.1527-0.1944\log_2 0.1944-0.1267\log_2 0.1267 = 2.23$.

2.3.   Fine–grained permutation entropy

FGPE was first described in ^[34]. It is another approach to the problem of not considering amplitude information in the standard PE algorithm. This method proposes to include an additional parameter $q$ to quantify the differences $\mathbf{d}_{j}$ between consecutive values in each pattern $\mathbf{x}_j^{m}$ as:

where $\alpha$ is another user–defined parameter, $\mathbf{d}_{j} = \left\{\left|x_{j+1}-x_{j}\right|, \ldots, \left|x_{j+m-1}-x_{j+m-2}\right|\right\}$, and $\left\lfloor \right\rfloor$ is the floor function, it keeps the integer part of the operand. For simplicity, we used $\alpha = 1$, as in ^[34]. The resulting value of $q$ is added as an additional symbol at the end of each $\boldsymbol{\pi}_{j}^{m}$. As a consequence, $\boldsymbol{\Pi}^{m}$ is not known in advance, it has to be updated dynamically when a new pattern is found. The algorithm for FGPE is shown in Algorithm 3 ($\#$ is the sizeof operator).

2.4.   Amplitude–aware permutation entropy

Probably, the most ambitious method to address PE weaknesses is the Amplitude Aware Permutation Entropy (AAPE) method ^[4]. This method feeds the amplitude into the PE basic method by quantifying the contribution of the subsequence mean and differences, but it also quantifies the possible effect of ties ^[15]. In this study, this later contribution will be removed from the AAPE method since it is not related to the amplitude influence, using a simplified version as in ^[12].

Only an additional parameter is required in this method. This parameter is termed $A$ and lies in the interval $[0, 1]$. The histogram of relative frequencies, instead of being updated by adding 1 each time a match is found, it is updated using a more elaborated expression:

where the term $\sum\limits_{k = 0}^{m-1}\left(\frac{A}{m}\left|x_{j+k}\right|\right)$ accounts for the mean influence, and the term $\sum\limits_{k = 1}^{m-1}\left(\frac{1-A}{m-1}\left|x_{j+k}-x_{j+k-1}\right|\right)$ for the amplitude differences. The resulting modification of the PE method is shown in Algorithm 4.

2.5.   Sample Entropy

SampEn is a sample amplitude–based entropy statistic that has been included in the present study for comparative purposes, but it is not the main focus of the assessment. It was first defined in ^[23], and since then it has become one of the most successful non–linear tool used for time series classification, with an endless list of applications ^{[14,1,10,30,11,31,24]}.

In this case, $\mathbf{x}^{m}_{j}$ is compared with all the other possible subsequences in the time series $\mathbf{x}^{m}_{i}$, with $i \neq j$, in terms of amplitude differences. This difference is given by $d_{ji} = \max(\vert x_{j+k}-x_{i+k}\vert), 0 \leq k \leq m-1$. An additional pre–defined input parameter, $r$, is the difference threshold beyond which the subsequences are taken as dissimilar.

If the subsequences under comparison are considered similar, a specific counter $B_{j}^{m}(r)$ is increased. This process is repeated for all $\mathbf{x}^{m}_{i}$, and the final average number of similar subsequences is computed as:

Another counter $A^{m}(r)$ is obtained in the same way making $m = m+1$ and repeating the previous process. From these two counters, SampEn can finally be calculated:

2.6.   Experimental dataset

The experimental dataset is composed of 6 representative datasets of real and synthetic time series specially devised for classification purposes. The specific datasets are described next:

● LORENZ. A Lorenz attractor was used to generate synthetic signals for classification. A total of 100 time series were created with random initialisation. One of the sets had the parameter values $\sigma = 10, \rho = 28$, and $\beta = 8/3$, with a time step of 0.0001. The other dataset had $\sigma = 9$, being the other parameters equal. These values were chosen just to ensure any classification method would be capable of finding differences between the two resulting classes. This type of time series have been used in a number of non–linear signal processing studies ^[40,16]. Examples of records in this dataset are shown in Figure 1a.

● MOTION. This database is included in the UEA and UCR public time series classification repository (www.timeseriesclassification.com) ^[5]. The time series were obtained from the movement of worms ^[36]. The objective was to compare phenotypes by assessing differences in movement patterns. The system employed to record the movement of the worms is described in detail in ^[8]. The specific time series included in this study correspond to the 77 records of length 900 in the test subset, first dimension only, with 2 classes, wild–type or one of four mutant types. Examples of these records are shown in Figure 1b.

● HOUSE. This database is also included in the same repository as the MOTION dataset. It contains two classes of 20 records each one. Class 1 is household agammaegate use of electricity, and Class 2 is agammaegate electricity load of tumble dryer and washing machine ^[26]. The length of the records is 1022 samples. Figure 1c depicts two examples drawn from this database.

● FANT. This dataset was drawn from the Fantasia database, available at ^[20]. It contains 40 RR–interval records from 20 young and 20 elderly healthy subjects monitored during 120 minutes. The length of the records is not uniform, but greater than 5000 samples in all cases. This database is described in ^[22]. Examples of records in this database can be found in Figure 1d.

● PAF. This is one of the many databases publicly available at Physionet ^[20]. The Paroxysmal Atrial Fibrillation (PAF) records of this database are described in detail in ^[25]. Specifically, only the short duration records of 5 minutes were included in the experiments, with a total of 50 records, 25 with PAF episodes, and 25 without. Examples of records in this database can be found in Figure 1e.

● EEG. This database contains 4097 samples–long electroencephalograph records recorded by the Department of Epileptology, University of Bonn ^[2]. This database has been used in many classification studies and has been included for comparison purposes. It is publicly available at http://epileptologie-bonn.de. Two classes out of the five originally available are included in the experiments, specifically records that correspond to the 100 seizure–free EEGs of this database from epilepsy patients, and 100 EEGs that do include seizures. Examples of records from the two classes are shown in Figure 1f.

3.   Results

The first experiment was devised to assess the classification performance in terms of accuracy of all the individual methods tested. This accuracy corresponds to the proportion of correctly classified time series (all the experimental datasets contain two classes) over the total number of records in the dataset. These results are shown in Tables 1, 2, 3, 4, 5, for PE–based measures, and $m = 3, 4, 6, 8, 9$ respectively, using the datasets described above. Parameters $\alpha$ for FPGE, and $A$ for AAPE were also varied.

For a better visualisation of the capabilities of each method, Table 6 includes the highest classification accuracy achieved with each method, regardless of the parameter values. In general, the metrics that include amplitude information in their computation outperform the standard PE method.

Table 7 shows the results obtained using SampEn, with $r = 0.15, 0.20, 0.25$, and $m = 1, 2, 3$. For comparison purposes, Table 8 summarizes the best results achieved using all the methods assessed.

4.   Discussion

There is not a clear winner in terms of classification performance, but there is arguably a clear loser, the standard PE method. This method yields in general the lowest performance of all the methods tested. It was only able to provide equal classification results for the EEG database (Table 6). Although PE classification was significant in most cases (except for the LORENZ database), there was always another amplitude–included metric that yielded a higher or at least comparable performance. In any case, there is a great dependence on the input parameters, mainly $m$.

WPE seems to perform best for high $m$ values. For $m = 8, 9$ (Tables 4 and 5), WPE was capable of finding differences in at least 4 datasets (the same in both cases, LORENZ, HOUSE, FANT, and PAF), whereas for lower $m$ values, only 2 or 3 datasets were correctly segmented. However, it is important to note that for $m = 3$, WPE was also able to segment the EEG database.

FGPE probably achieved the overall highest performance, with a classification accuracy above 0.7 in almost all cases (Table 6), the only one. It can be reasonably assumed that this performance could even be improved with a more exhaustive parameter search, mainly for $\alpha$. However, this performance is more scattered with regard to parameter space, and that entails a more cumbersome algorithm customisation for the problem at hand. In addition, the FPGE algorithm is more complex and slower since the number of motifs is not known in advance.

The AAPE algorithm is similar to that of WPE, and in contrast to FGPE, it is based on weights, not on additional symbols for the ordinal sequences. Of the three amplitude–included methods, this one yields the lowest performance, but it is still slightly better than that of the standard PE method. As for FGPE, other $A$ parameter values would probably improve such performance. In other words, there is room for improvement, whereas PE is already optimized by means of $m$. SampEn is an entropy metric based solely on amplitude. It was included in the experiments as a reference for the possible influence of amplitude, with no interference from ordinal patterns. The performance described in Table 7 confirmed the suitability of this analysis, since SampEn was also able to classify significantly all the datasets except MOTION. The value of the parameters were in the usually recommended region of $r = 0.20$ and $m = 2$, although as in any multiparameter metric, other values could even yield a higher accuracy.

The best SampEn results were achieved for the HOUSE database classification (0.95 accuracy). This can be due to the fact that these signals resemble a discrete Markov process where differences lie mainly on signal level. In this case, results were very significant regardless of the $m$ or $r$ value employed. This was also the only case in which SampEn outperformed the PE–based metrics. Of all the parameters tested, only for $m = 3$ and $r = 0.15$ was SampEn able to distinguish between classes in a maximum of 4 datasets. In such case, LORENZ, HOUSE, FANT, and EEG classes were detected significantly, with a classification accuracy higher than 0.700 (Table 7).

Other possibility we explored was to use a model that combined more than a single measure. This approach has yielded very good results in other similar works ^[13]. However, in this case the results achieved using a logistic model with PE and SampEn together, did not improve the results of WPE, FGPE, or AAPE. The models reached the same performance as that achieved by either PE or SampEn, but no synergy was observed. That is why this kind of solution has not been included in the present study.

Obviously, there will surely be datasets where the behaviour and performance of all these metrics will be different from that observed and reported in this paper. However, from the results, it can be stated that the general trend is that amplitude does have an influence on classification performance. If a generic metric is needed, with no time for customisation, WPE is probably the best choice, since it does not need to configure any parameter other than $m$, as the standard PE, and the algorithm is quite simple. If maximum performance is required, both FPGE and AAPE have the possibility of further customisation by means of an additional parameter, being FPGE in principle more accurate but also more complex. SampEn confirmed the significance of the information carried by the amplitude, but since the performance was not as high as with WPE, FGPE, or AAPE, it becomes apparent that the optimal solution is to combine ordinal and amplitude information, together as in this study, or maybe separately, as in ^[13].

5.   Conclusion

PE is a very efficient non-linear metric, in the sense of measuring tool, to assess the dynamics of a time series. The algorithm is very simple to implement, robust against noise, and only requires a weak stationarity ^[6]. However, since the computations are based on ordinal patterns only, the lack of amplitude consideration is often seen as a possible weakness.

In order to address such weakness, some approaches have been proposed in the last years ^[17,34,4]. These approaches have been reported to improve the results of the standard PE algorithm, but no comparative and general analysis had been developed so far. The study in this a paper used three of these methods: WPE, FGPE, and AAPE, with a varied and diverse experimental dataset and under the same conditions for all the methods, in order to assess the real influence of amplitude variations on PE performance.

When amplitude information was included in the calculations, the classification performance achieved was usually higher than with the standard PE method. Moreover, the three methods that combined ordinal and amplitude information, WPE, FGPE, and AAPE, also outperformed a method based only on amplitude, such as SampEn. Arguably, it can be concluded that a combination of both approaches is the best solution, and amplitude does have a significant influence on PE.

A generic recommendation would be to use WPE instead of PE and expect a better performance, without additional parameters, and with a little additional algorithm complexity. If classification accuracy is key, a proper customisation of FGPE will probably yield the best results, at the cost of a higher computational burden. AAPE is less complex, unless the complete version is used (including ties effect). If only amplitude is considered, AAPE can also yield good results with less algorithm changes than FGPE. The performance when ties are also included, has been studied in another paper ^[15], and the complexity of the algorithm raises substantially.

Conflict of interest

The author declares that he has no conflict of interest.