Cytokine signaling in the modulation of post-acute and chronic systemic inflammation: a review of the influence of exercise and certain drugs

1.   Introduction

In the 21^st century, a new viral infection known as SARS-CoV-2 (severe acute respiratory syndrome coronavirus 2) originated in Wuhan city of South China. The novel coronavirus has created an adverse impact on the health and socio-economic activities of the public, as it has spread rapidly to different regions of the world within a short span of time. To control the spread of the COVID-19 infection, the World Health Organization (WHO) declared COVID-19 as a pandemic on March 11, 2020. The announcement of COVID-19 as a pandemic has led to the announcement of strict lockdown policies such as the shutdown of national and international airports, traveling restrictions and the closure of non-essential services by most of the countries around the world to control the dissemination of infection. The fear created by the COVID-19 outbreak has caused significant consequences on societal health and well-being, as well as the global economy ^[1].

In this situation, the internet plays a key role in connecting individuals with the rest of the world. Most of the individuals are dependent on the internet to check the content that circulates on the social media platforms regarding the COVID-19 pandemic. Social media provides different platforms, such as Facebook, Twitter, Instagram and Reddit, to provide communication and share information and other data among the human community ^[2,3]. As the Twitter platform provides real-time coverage of current events that are vigorously and locally fluctuating, like the COVID-19 pandemic, it provides comprehensive and valuable information on the present state of public health and disease spreading in a region. Several studies have shown that tweets have been a vital source for the early identification, warning and intervention of infectious diseases like cholera ^[4], seasonal conjunctivitis ^[5], Ebola ^[6], the bubonic plague ^[7] and E. coli ^[8]. The WHO has reported that early identification of more than 60% of epidemics can be found via social media data ^[6]. Moreover, the mining of social media data can assist in the early identification of public health emergencies. The identification of a personal health mention (PHM) is one of the crucial steps in public health surveillance. The basic aim of a PHM is to identify the health condition of a person depending on online text information. To explore the identification of PHMs for COVID-19 pandemic using Twitter, a masked attention model was developed by Luo et al. ^[9]. Initially, a COVID-19 PHM dataset consisting of 11, 231 tweets from February to May 2020 was built. Then, these tweets were annotated with four types of COVID-19-related conditions. The experimental results revealed that the proposed approach attained superior performance when compared with other approaches. To address the class imbalance problem, a dual convolutional neural network (CNN) framework has been proposed by Luo et al. ^[10]. The proposed approach has been evaluated by using a COVID-19 PHM dataset containing more than 11, 000 annotated tweets, and the results indicate that the proposed approach can mitigate the impact of class imbalance problems and attain promising outcomes. As the mining of social media data minimizes the cost of information acquisition and analysis, it enhances the responsiveness of public health sectors and provides a new perspective on public health ^[11]. Therefore, the mining of tweets related to the COVID-19 pandemic can also assist the public health sector in monitoring the status of COVID-19 infection in a timely manner and making appropriate decisions to control the spread of infection. In this present pandemic situation, it is impossible for a person to work without social media, as it covers all of the updates, such as updates on COVID-19, updates on the stock market and so on ^[12]. Regardless of the large content present on the social media, these contents may cause contradictory impacts on the lives of people, such as negative or positive psychological influences ^[13]. In most cases, information related to COVID-19 that has circulated on social media has guided the people toward wrong decisions. Therefore, converting these posts into resources is highly valuable for making decisions regarding business and policy deployment, which can be done efficiently through sentimental analysis and social mining ^[14].

Computer technologies play a vital role in performing sentimental analysis of social media data in order to facilitate opportunities to fight against outbreaks of infectious disease ^[15]. Several researchers have indicated that outbreaks can be effectively controlled if experts make use of social media data ^[16]. Nowadays, machine learning approaches are being used in a variety of fields, such as opinion mining, image analysis, wireless sensor networks, fake news identification and many more, because such approaches have the ability to learn from itself without being explicitly programmed. Due to their versatility and veracity, machine learning approaches have been gaining more popularity in recent years in sentiment analysis. To improve user satisfaction in online shopping, a novel framework based on a machine learning approach known as a Local Search Improvised Bat Algorithm based Elman Neural Network (LSIBA-ENN) model was recommended by Zhao et al. ^[17] for the efficient sentiment analysis of online reviews of products. The major components of the proposed framework are data collection, pre-processing, feature extraction and polarity. In the first step, the customer online reviews of a product are gathered by using web scraping tools. In the next step, pre-processing of the scraped data extracted from the web is performed. Then, for additional processing, the processed data undergo TW (Term Weighting) and FS (Feature Selection) by applying a hybrid mutation-based earthworm algorithm (HMEWA) and Log Term Frequency-based Modified Inverse Class Frequency (LTF-MICF). In the last step, the customer reviews are classified as positive, neutral and negative by rendering HMEWA data to the suggested LSIBA-ENN. The performance of the suggested framework was analyzed by using '2' yardstick datasets, and the results revealed that the LSIBA-ENN exhibited the best performance as compared to other approaches. Moreover, it was also observed that the Elman neural network attained a recall of 87.79 when the LTF-MICF scheme was utilized, and a recall of 86.04, 85.48, 84.03 and 83.55, respectively, when the TF-DFS (Term Frequency-Distinguishing Feature selector), TF-IDF (Term Frequency-Inverse Document Frequency), TF (Term Frequency) and W2V (Word 2 Vector) schemes were utilized.

To predict the rates of currency exchange based on the analysis of Twitter messages, an approach depending on machine learning has been recommended by Naeem et al. ^[18]. Initially, a dataset was developed based on the data from the Forex Trading website. The dataset contains exchange rates between USD (United States Dollar) and PKR (Pakistani Rupee). In addition, the dataset also contains tweets related to finance words that were collected from the Pakistan business community. Then, data pre-processing was applied to the collected raw dataset. After performing data pre-processing, response variable labeling was assigned to the processed dataset. Further, the authors utilized principal component analysis and linear discriminant analysis for the better representation of the dataset in a three-dimensional vector space. To optimize the data, clusters that were formed through a sampling approach has been utilized. Finally, recommended classifiers such as a random forest, naïve Bayes, a simple logistic classifier, a support vector machine and bagging have been evaluated; the results indicate that simple logistic regression attained better performance, i.e., 82.14% accuracy, as compared with other approaches.

In recent years, deep learning approaches have been able to explore complex semantic descriptions of text automatically from data without any feature engineering. Moreover, deep learning approaches have also enhanced the modernity in many tasks of sentiment analysis, such as the sentiment classification of sentences, lexicon learning of sentiment and extraction of sentiments. Therefore, these approaches are presently surging as vigorous computational models in sentiment analysis. To perform precise sentiment classification of the Chinese language, a novel approach based on deep learning, known as HEMOS, has been developed by Li et al ^[19]. The authors also analyzed the significance of observing the impact of humor, slang and pictograms on the task of perceptual processing of the social media data. Initially, 576 periodic slang expressions from the internet were collected as slang lexicon. Then, a Chinese emoji lexicon was constructed from the textual features that have been obtained by converting 109 Weibo emojis. Later, the base for four-level sentiment classification of Weibo posts was created by carrying out two polarity annotations with "optimistic" and "pessimistic" humorous types. Then, positive and negative lexicons were applied to the proposed attention-based bi-directional long short-term memory recurrent neural network (AttBiLSTM) approach for the precise sentiment classification of the Chinese language. Further, the developed approach was evaluated by using undersized labeled data; the results indicate that the developed approach can substantially enhance the performance of standard methods in assessing sentiment polarity on Weibo emojis.

Pathak et al. ^[20] developed a deep learning-based topic-level sentiment analysis model for the efficient discovery and analysis of topics discussed on social media platforms, and for making efficient decisions in a timely manner. Basically, the developed model aims to extract the topic from sentence level data by means of online latent semantic indexing with a regularization constraint. Then, the model implements a topic-level attention process to carry out sentiment analysis in a long short-term memory (LSTM) network. As the model supports topic-level sentiment analysis and provides both scalable and dynamic topic modeling of streaming short text data, it is considered as a peculiar model. Regarding in-domain topic-level sentiment analysis, the model achieved a recall of 0.879 on the SemEval-2017 Task 4 Subtask B dataset. On the out-of-domain data, the model achieved average recalls of 0.794, 0.824 and 0.846 on datasets developed by respectively using the hashtags #facebook, #bitcoin and #ethereum from Twitter. Moreover, the scalability of the model was analyzed in terms of the average time in milliseconds, topics determined per second and seconds required for the construction of feature vectors, throughput and average time, respectively.

To capture the current conditions of the market by means of Twitter sentiments by using Google's bidirectional transformer BERT, two novel approaches known as Sentimental All-Weather and Sentimental MPT have been suggested by Leow et al. ^[21]. Further, the authors have utilized a genetic algorithm for the maximization of cumulative returns and the minimization of volatility. Then, the suggested approaches have been trained on tweets data for the USA stock market collected from August 2018 to December 31, 2019. The empirical results revealed that suggested models attained superior performance with regard to common measures of portfolio performance, the cumulative returns, the Sharpe ratio and the value-at-risk as compared to other benchmarks such as the MPT model, buy-and-hold spy index and CRB approach for the All-Weather Portfolio.

A systematic review on the mobility data usage from recent publications on COVID-19 and human mobility from a data-oriented perspective has been carried out by Hu et al. ^[22] to assist policymakers and reviewers in the decision-making of policies related to the COVID-19 pandemic and other infectious disease outbreaks. The authors identified the public transit system, social media-derived mobility data and mobility data as three primary sources of mobility data. Further, the authors utilized four distinct approaches, such as the use of social activity patterns, social media-derived mobility data, index-based mobility data and public transit-based flow, to determine human mobility. Then, the authors assessed the quality, data privacy, space-time coverage privacy, accessibility and high-performance data storage to compare the characteristics of mobility data. Moreover, the authors also presented challenges and future directions for utilizing mobility data.

To address the partial domain adaptation challenge, a model known as the deep residual correction network (DRCN) has been suggested by Li et al. ^[23]. To improve the adaptation from source to target and weaken the influence from inappropriate source classes, the authors plugged one residual block into the source network together with the task-specific feature layer. The plugged residual block that consists of several fully connected layers not only intensify the basic network, but it also improves its capability of feature representation. Moreover, a weighted class-wise domain alignment loss was designed by the authors to couple two domains. Then, the extensive experiments conducted on partial, traditional and fine-grained cross-domain visual recognition revealed that the DRCN outperforms other deep domain adaption approaches.

From the above discussion, it can be observed that different machine learning and deep learning approaches have been used to perform sentiment analysis efficiently on social media data. Hence, sentiment analyses for the present COVID-19 pandemic are also important, as COVID-19 stays as a controversial global topic in social media ^[24]. To observe the sentiments of Bangladeshi people's comments on COVID-19 posts posted on Facebook, a deep learning approach utilizing a CNN was proposed by Pran et al. ^[25]. The suggested architecture makes use of three classes, namely, analytical, depressed and angry, to investigate the emotions. Further, the suggested architecture was evaluated by using a dataset created in the Bangla language, and the results indicate that the CNN attained an accuracy of 97.24% in analyzing COVID-19 posts posted on Facebook by Bangladeshi people.

To perform the sentiment analysis of live-stream tweets related to COVID-19 and to predict COVID-19 cases, a novel approach based on deep learning has been recommended by Das et al ^[26]. Initially, the authors developed a large tweet corpus depending on COVID-19 tweets. Then, they performed trend analysis and polarity classification by splitting the data into training and testing data. Further, the precise outcome from the trend analysis assists in training the data in order to produce an incremental learning curve for the neural network. The model was evaluated, and the results indicate that, not only did the model attain a better accuracy of 90.67%, but it also maintained performance stability when validated against various prevalent open-source text corpora.

To address the issues related to COVID-19 by performing sentiment analysis of social media data, different deep learning approaches have been used. It can be observed from the discussion that the approaches using CNN architecture has the limitation that it can impact the accuracy of the model ^[27]. Due to their ability to catch complex patterns and obtain better performance, several studies have utilized the rectified linear activation function (ReLU) in their neural network models ^[28]. However, the usage of the ReLU function can cause dying neurons that may restrict the learning process of neural networks. To overcome this limitation, architecture such as that of LSTM has been recommended. As LSTM assists in solving sequential and time-series problems with superior results, it is considered as an important function ^[29]. Therefore, our proposed approach makes use of the LSTM activation function because of its ability to learn text sequences and find associations between words or phrases in sentiment analysis. Moreover, it can be used to enhance the semantic data of tweets, as well as the efficiency of the learning model, which yields superior performance on some datasets because the network is not acquiring the right volatility trends in the data.

Therefore, the main contribution of the present study is as follows:

i. An LSTM model has been proposed for the classification of COVID-19 related tweets.

ii. To enhance the overall performance of the model, a firefly optimization algorithm is proposed to tune the hyperparameters of the LSTM model.

iii. Further, the performance of the proposed model is compared with other state-of-the-art ensemble and machine learning-based methods, such as the decision tree (DT), the multi-layer perceptron (MLP), k-nearest neighbors (KNN), the random forest (RF), AdaBoost, gradient boost (GBoost), bagging and extreme gradient boost, to prove the efficacy of the developed model.

The remaining part of this paper is sectioned as follows. Section 2 describes some of the previous studies carried out on the analysis of Twitter-related data, along with their limitations. The description of the proposed methodology is illustrated in Section 3. Section 4 outlines the experimental setup and description of the dataset used in the proposed approach. Section 5 provides a comparative analysis of the results of the proposed approach and other state-of-the-art models. Lastly, Section 6 concludes the work with some important future directions.

2.   Related work

The outbreak of the COVID-19 pandemic has imposed a tremendous impact on the health and socio-economic activities of the public all over the world. With the rapid rise in the number of COVID-19 cases, it has become a major cause for depression, panic and anxiety among the people due to the deceitful information posted on social media. Among the social media platforms, Twitter stands on top regarding the time to spread COVID-19 news ^[30]. This section presents a brief discussion of some of the studies carried out on the analysis of Twitter COVID-19 data by using machine learning and deep learning approaches.

Using the R programming language, Kaur et al. ^[31] performed an interpretation of Twitter data that depends on keywords such as coronavirus, new case, COVID-19, recovered and deaths. In addition, a novel algorithm known as the hybrid heterogeneous support vector machine (H-SVM) has been suggested by the authors to carry out the sentiment classification. Then, the efficacy of the suggested H-SVM approach was evaluated by using various metrics, such as precision, accuracy, F1-score and recall. From the empirical results, it was concluded that the H-SVM obtained better performance in terms of the values of the precision, recall and F1-score, i.e., 0.86, 0.69 and 0.77, respectively, as compared to linear support vector machines (SVMs) and neural networks. To perform sentiment analysis of the coronavirus Twitter data of eight different countries, a novel approach has been suggested by Basiri et al. ^[32]. This approach is a synthesis of four deep learning approaches and one conventional machine learning approach. In addition, to sense the alterations in the sentiment patterns at distinct places and times, Google Trends was used to analyze the searches related to the coronavirus. From the findings, it was found that COVID-19 has drawn the attention of the public from distinct countries with differing intensities at different times. Moreover, the news and events that took place in their countries, such as the number of deaths, infected cases and recoveries, were also matching with the sentiment in their tweets. To classify tweets into positive, negative or neutral sentiments, three distinct ensemble classifiers known as the stacking classifier, bagging classifier and voting classifier were proposed by Rahman and Islam ^[33]. For the classification of tweets into sentiments, 12, 000 tweets collected from the United Kingdom (UK) were meticulously annotated by three separate reviewers. Finally, from the study, it was concluded that the stacking classifier obtained better performance (83.5% F1-score) as compared to the F1-score of the bagging and voting classifiers.

Rustam et al. ^[34] performed the analysis of tweets related to COVID-19 by using supervised learning approaches such as a support vector classifier, an RF, a DT, XGBoost and extra trees classifiers. Then, data preprocessing techniques and the TextBlob library were used to clean the data and extract the sentiments. In addition, the authors have also suggested a feature set for evaluating the models, which was obtained by integrating term frequency-inverse document frequency and bag-of-words. Further, the suggested approaches and the LSTM approach were also evaluated by using various performance metrics, and the results indicate that the extra tree classifier surpassed the other supervised machine learning models and the LSTM approach by attaining an accuracy of 93%. Moreover, the results were compared with the Vader sentiment analysis technique to show the efficiency of the suggested feature set. To identify misleading information regarding COVID-19 on Twitter, modified LSTM and modified GRU deep learning approaches have been recommended by Abdelminaam et al. ^[35]. The parameters of the proposed models were optimized by using a Keras tuner. The suggested approaches, along with machine learning approaches such as the DT, KNN, SVM, RF, logistic regression and naïve Bayes were evaluated on four standard datasets, and two feature extraction methods were used to extract features from the standard four datasets. From the results, it was concluded that the suggested framework obtained better accuracy in terms of detecting misleading tweets related to COVID-19 as compared with the standard machine learning approaches. In ^[36], the authors used a bidirectional encoder representation from transformers (BERT) bidirectional LSTM (Bi-LSTM) ensemble learning approach to perform sentiment analysis of COVID-19-related tweets. The proposed approach comprises two stages. The BERT model ^[37], in the first stage, acquires the domain knowledge with data on COVID-19 and adjusts with a sentiment word dictionary. To process the data in a bidirectional way, a Bi-LSTM approach ^[38] was used in the second stage. Finally, the BERT and Bi-LSTM approach was combined by the ensemble approach to classify the sentiment into positive and negative categories. Experiments were conducted, and the results indicate that the proposed approach attains better results as compared with other standard approaches. The following table, Table 1, represents other works carried out on the analysis of sentiments regarding COVID-19.

Table 1.  Other works performed on the analysis of sentiments related to COVID-19.

Author & Year	Approach	Dataset used	Aim of the work	Results	Limitation	Ref
Chintalapudi et al. 2021	Bidirectional Encoder Representations from Transformers (BERT) model	3090 tweets obtained from github.com within the time span of March 23, 2020 to July 15, 2020	Introducing fact-checkers on social media platform to overwhelm misleading propaganda about COVID-19 infection	Accuracy = 89%	Addresses only single-country people's emotions on social media sites on present pandemic	[39]
Sitaula et al. 2021	Fusion of fastText-based CNN model, domain-specific CNN model, and domain-agnostic-based CNN model.	Tweets collected from Feb. 11, 2020 to Jan. 10, 2021 through the geo-location of Nepal using keyword #COVID-19	To classify COVID-19 tweets related to Nepali	Accuracy = 68.7%	Subsequent approach of tokens has not been considered	[40]
Chandra and Aswin 2021	LSTM and BERT language model	Tweets related to COVID-19 within the time span of March 2020 to September 2020 from Maharashtra and Delhi states having highest COVID-19 confirmed cases	To analyze sentiment of people in regions having highest COVID-19 cases	Most of the tweets classified as positive tweets with highest confidence level	Does not consider tweets from distinct regions and countries	[41]
Tripathi 2021	NB, SVM and LSTM	4035 pre-processed Nepali tweets from https://www.kaggle.com/milan400/original	Analysis of Nepali tweets related to COVID-19	Accuracy of LSTM = 79%	Small dataset	[42]
Malla & Alphonse 2021	MVEDL (majority voting technique-based ensemble deep learning) model	COVID-19 English dataset from IEEE Dataport	To prevent spreading of negative sentiments related to COVID-19	Accuracy = 91.75%	Time complexity of the model is very high as CT-BERT, BERTweet and RoBERTa are pretrained models with large memory	[43]

---

 | Show Table

DownLoad: CSV

3.   Methodology

This section outlines the architecture of the proposed LSTM approach and firefly optimization algorithm used for the classification of tweets related to COVID-19 as positive and negative sentiments.

3.1.   LSTM architecture

The LSTM network developed by Hochreiter and Schmidhuber ^[44] is a special type of recurrent neural network which is capable of processing sequences of data. LSTM contains memory cells which are used for holding data for a long period. Due to the recursive nature of the cells, LSTM networks can store and connect previous information to the data acquired in the present. Basically, LSTM networks consist of three gates, known as the input gate, forget gate and output gate. The internal architecture of the LSTM network is shown in Figure 1.

Figure 1.  Internal architecture of LSTM network.

DownLoad: Full-Size Img PowerPoint

In the above figure, ${x}_{t}$ represents the current state, the previous cell state is represented by ${C}_{t-1}$ and the new state is represented by ${C}_{t}$. The previous and current outputs are respectively represented by ${h}_{t-1}$ and ${h}_{t}$. The content of cell states is modified by the forget gate, which is placed beneath the cell state. The output value of the forget gate represents a number between 1 and 0. If the output of the forget gate is 0, then the information is not kept in the cell. Otherwise, information is kept in the cell. The information that enters into the cell state is determined by the input gate. Lastly, the information that passes on to the subsequent hidden state is determined by the output gate. The following Eq (1) to Eq (6) are the mathematical formulas used to represent the LSTM network.

The information that needs to be stored has been selected by the input gate by using Eq (1):

The information that needs to be abandoned or kept has been selected by the forget gate by using Eq (2):

Finally, the output is determined by using Eq (3):

where the sigmoid function is represented by $\sigma$, the weights of the input, forget and output gates are represented by ${W}_{i}$, ${W}_{f}$ and ${W}_{0}$, respectively, the input at timestamp $t$ is represented by ${x}_{t}$, the biases of the input, forget and output gates are respectively represented by ${b}_{i}$, ${b}_{f}$ and ${b}_{0}$ and the input, forget and output gates are respectively denoted by ${i}_{t}$, ${f}_{t}$ and ${O}_{t}$, respectively.

The mathematical formula for the candidate cell state ${\stackrel{~}{c}}_{t}$ is given by Eq (4):

where information to add or remove to the previous input by the LSTM network is allowed by $tanh$, the candidate at timestamp t is denoted by ${C}_{t}$. ${b}_{c}$ and ${W}_{c}$ denote the control and weight parameters at the $tanh$ layer.

The cell state ${C}_{t}$ is represented by the mathematical formula shown in Eq (5):

The final output state ${h}_{t}$ is represented by the mathematical formula shown in Eq (6):

where the vector's element-wise multiplication is denoted by *. ${h}_{t}$ denotes the predicted output obtained from the current LSTM block, and ${C}_{t}$ represents the memory cell state at timestamp $t$.

3.2.   Firefly algorithm

In 2007, a multimodal nature-inspired metaheuristic algorithm known as the firefly algorithm was developed by Yang ^[45] at Cambridge University. Basically, the algorithm was developed based on the inspiration of the flashing behavior of tropical fireflies. The fundamental property that is enables the attraction of fireflies is its flashing light ^[46]. In general, fireflies stay more active during summer nights ^[47,48]. The process of coupling takes place between fireflies when a firefly finds another firefly nearby. Usually, male fireflies, through their signals, try to attract the female fireflies on the ground ^[49,50]. Then, the female fireflies discharge their flashing light in response to the signals of male fireflies, which results in the generation of flashing patterns of both males and females. Normally, female fireflies are more fascinated by male fireflies with brighter flashing lights. Based on the source distance, the intensity of flashing may vary. However, in some rare situations, female fireflies are unable to differentiate between male fireflies having the brightest and weakest flashes. According to Yang ^[51], the following are the idealized principles of the firefly algorithm:

• Regardless of the gender, the fireflies are attracted to each other.

• Fireflies with less brightness will be attracted by the brighter fireflies, as the attractiveness of fireflies is proportional to the brightness of the fireflies. Anyhow, the brightness of the fireflies decreases with increase in distance.

• The backdrop of the objective function is used to determine the illumination of the firefly.

The two phases of the firefly algorithm have been defined as follows:

i) Generally, the brightness of the firefly algorithm is based on the light intensity discharged by the firefly. Assume there are n number of fireflies. Then, the fitness value of the ${i}_{th}$ firefly is denoted by $f\left({x}_{i}\right)$. Now, the brightness of firefly $\beta$ is chosen as shown in Eq (7):

ii) As each and every firefly has their special attractiveness, they can strongly attract other fireflies in the swarm. The attractiveness between two similar fireflies will diverge with distance factor ${d}_{ij}$, as shown in Eq (8):

Then, the firefly's attractiveness is determined by using Eq (9):

where $\gamma$ denotes the coefficient of light absorption and $\beta$ denotes the attractiveness of the firefly at $r = 0$.

Then, the movement of the firefly with less brightness toward a firefly with more brightness is determined by using Eq (10):

Based on the above assumptions, the following pseudocode is used to represent the basic steps of the firefly algorithm.

Algorithm 1: Firefly Algorithm

Step 1: Set size of the population as $N$ and number of iterations as ${Max}_{Gen}$ Step 2: Set the parameters $\alpha, {\beta }_{0}\wedge \gamma$ Step 3: Define the objective function $f\left(x\right)$, where x = $\left({x}_{1}, {x}_{2}, \dots., {x}_{n}\right)$ Step 4: Formulate the fitness value of the fireflies from the objective function Step 5: for k = 1 to ${Max}_{Gen}$ do Step 6: for i = 1 to $N$ do Step 7: for j = 1 to $N$ do Step 8: if $f\left({x}_{j}\right) < f\left({x}_{i}\right)$, then Step 9: move ${x}_{i}$ toward ${x}_{j}$ using Eq (7) Step 10: end if Step 11: Attractiveness changes with distance r between fireflies by $exp\left(-\gamma {r}^{2}\right)$ Step 12: calculate the fitness value of a new firefly by updating its light intensity. Step 13: end for j Step 14: end for i Step 15: Rank fireflies based on their fitness Step 16: Find the best one Step 17: end for k Step 18: Stop

---

 | Show Table

DownLoad: CSV

3.3.   Proposed methodology

In the proposed model, the firefly algorithm has been introduced to optimize the parameters of the LSTM network to enhance the classification accuracy. The firefly algorithm is a nature-inspired metaheuristic algorithm with stochastic behavior. Due to the stochastic nature of the firefly algorithm, it is adequate to search a solution set by means of randomization. In heuristic algorithms, the probability of obtaining the optimal solution in a realistic amount of time is less because these algorithms use a trial-and-error base to search the solutions of the problems. Whereas, in metaheuristic algorithms, the probability of obtaining the optimal solution is higher, as the search is based on two approaches. One approach is exploration, which is the process of finding diversified solutions in the search space. The other approach is exploitation, which is the process of finding the best solution among neighbors to elicit the information. As the firefly algorithm makes use of both exploration and exploitation, it generates the best fit solution at lower levels from the recently produced solutions in the search space. Moreover, the firefly algorithm is also not trapped at local minima because of its randomization character. Therefore, the firefly algorithm is used to overcome the drawbacks of both randomization and local searches at higher levels. Furthermore, the firefly algorithm is a population-based algorithm, as it uses cross-operation to find the solutions in the search space. However, the population-based algorithms generate good parameter values and sustain the stability among exploration and exploitation only through the proper adjustment of the parameters. The only parameters that need to be adjusted in the firefly algorithm are the intensities of light and attractiveness. Initially, one should set the parameters of the firefly to produce the initial population of the fireflies randomly, and then calculate the fitness of each firefly by using Eqs (7) to (9). In the training phase, the LSTM network is trained on the dataset with the hyperparameters selected by using proposed firefly algorithm. Then, the fitness of the LSTM is computed by using the sum of the root mean square error (RMSE) as shown in Eq (11):

Finally, the loss function of the model is calculated by using Eq (12):

where the final required output at $\left(t-1\right)$ is denoted by ${d}_{j}\left(t\right)$. The process terminates if the objective function is satisfied. The overall framework of the proposed LSTM and firefly algorithm for the classification of COVID-19 tweets is depicted in Figure 2.

Figure 2.  Overall framework of the proposed methodology.

DownLoad: Full-Size Img PowerPoint

The specific steps for implementing the LSTM using firefly algorithm is listed below.

Step 1: Initially, set the parameters of the firefly algorithm, such as the number of fireflies, randomness ($\alpha$), initial attractiveness, maximum generation, absorption coefficient and number of iterations.

Step 2: Determine the light intensity of each firefly and then evaluate the firefly's attractiveness by using Eq (9).

Step 3: Then, determine the movement of the firefly with less brightness toward the firefly with more brightness by using Eq (10).

Step 4: Further, apply the firefly algorithm to optimize the parameters of the LSTM network, such as the learning rate, number of hidden layers, batch size, activation function and epochs.

Step 5. Train the LSTM network by using the optimized parameters and evaluate the fitness function.

Step 6: Repeat Steps 2 to 5 until the termination condition is satisfied.

Step 7: In the final step, compare the performance of the optimal proposed model with other standard machine and ensemble machine learning approaches by using evaluation metrics such as the accuracy, F1-score, precision, area under the curve of receiver operator characteristic (AUC-ROC) and recall.

4.   Experimental setup and dataset

All of the experiments in this study were performed by using a Jupyter Notebook with Python programming language which makes use of the Anaconda distribution package; the following is the system configuration used for carrying out the experiments: HP Pavilion system with Windows 10 operation system, 16 GB RAM, 160 GB programming space allocated by the NVIDIA Corporation, Intel core i7+10 generation processor with 1.80 GHz and 2.30 GHz speed. Moreover, seaborn, matplotlib and statsmodel packages were used to perform data visualization and statistical analysis of the Twitter data.

4.1.   Dataset description

In this study, the dataset related to COVID-19 tweets was incorporated from IEEE DataPort ^[52]. For this dataset, entity recognition and translation were done by using a fully automated algorithm that makes use of artificial intelligence with sentiment analysis. The dataset contains 5, 016 entities obtained from 1, 866 messages collected within the time span of July 15, 2021 to August 10, 2021. Out of 1, 866 tweet messages, 990 tweets contain one or more location entities, and 1, 322 location entities were identified in city, continent, country region, language and state entity categories. Initially, data pre-processing was applied to the dataset to enhance the optimization of the training process. Data pre-processing consisted of applying label encoding to the target column to convert the labels into numeric form and applying min-max scaler transformation to the independent columns.

4.2.   Data partitioning

To avoid overfitting on the dataset, the dataset was divided into two portions, i.e., training and testing. The trained tweets were used to identify the patterns of data and minimize the error rates. The testing dataset was used for the evaluation of model performance. Of the tweets, 80% were used for training and 20% of the tweets were used for testing.

5.   Result analysis

This section describes the performance metrics used in the evaluation of the model and the analysis of the results obtained using the proposed approach.

5.1.   Performance metrics

The performance of the proposed model has been evaluated by using various performance metrics, such as accuracy, precision, F1-score, recall and AUC-ROC. The mathematical formulas used for the representation of the accuracy, precision, recall and F1-score are shown in Eqs (13) to (16):

In the above equations, true positive, false positive, true negative and false negative are represented by TP, FP, TN and FN, respectively. P and R are used to represent the precision and recall.

5.2.   Comparative analysis of the results

In this section, the performance of the proposed approach is analyzed, along with other state-of-the-art models, such as the DT, the MLP, KNN, the RF, AdaBoost, gradient boost, bagging, extreme gradient boost and the LSTM network. The hyperparameters of the proposed approach were fine-tuned using the firefly algorithm. Table 2 presents detailed descriptions of the parameters used in model training of the proposed and other state-of-the-art models.

Table 2.  Descriptions of parameters used for training the model.

Model	Parameters
DT	criterion='gini', splitter='best', min_samples_split=2, min_samples_leaf=1
MLP	hidden_layer_sizes=(100, ), activation='relu', solver='adam', alpha=0.0001, learning_rate_init=0.001, max_iter=200
KNN	n_neighbors=5, weights='uniform', algorithm='auto', leaf_size=30, p=2, metric='minkowski'
RF	n_estimators=100, criterion='gini', min_samples_split=2, min_samples_leaf=1
AdaBoost	base_estimator=DecisionTreeClassifier(), n_estimators=50, learning_rate=1.0, algorithm='SAMME.R
GBoost	loss='deviance', learning_rate=0.1, n_estimators=100, subsample=1.0, criterion='friedman_mse', min_samples_split=2, min_samples_leaf=1, max_depth=3
Bagging	base_estimator=DecisionTreeClassifier(), n_estimators=10, *, max_samples=1.0, max_features=1.0
XGBoost	learning_rate =0.1, n_estimators=100, max_depth=5, min_child_weight=1
LSTM	Dropout=0.2, num_hidden_layer=300, number_neurons=128, Learning_rate=0.1, activation_fun_hidden layer='relu'
LSTM+ Firefly	Dropout=0.1, num_hidden_layer=100, number_neurons=32, Learning_rate=0.001, activation_fun_hidden layer='tanh'

---

 | Show Table

DownLoad: CSV

Figure 3(a) and 3(b) respectively represent the accuracy curve and loss curve for the LSTM and proposed approach. From Figure 3(a), it is evident that the proposed LSTM + Firefly approach attained better training and testing accuracy from the initial epoch as compared with the training and testing accuracy of the LSTM approach. Also, it is evident from Figure 3b that the training and testing loss of LSTM + Firefly was less as compared to the training and testing loss of LSTM model.

Figure 3.  (a) Accuracy curve of LSTM and proposed approach. (b) Loss curve of LSTM and proposed approach.

DownLoad: Full-Size Img PowerPoint

Figure 4 represents a comparative analysis of the training and testing accuracy of the proposed approach and other state-of-the-art models. From the figure, it can be concluded that the proposed approach obtains better training and testing accuracy as compared with the DT, MLP, KNN, RF, Adaboost, GBoost, XGBoost, Bagging and LSTM approaches.

Figure 4.  Comparative analysis of training and testing accuracy of proposed approach and other state-of-the-art models.

DownLoad: Full-Size Img PowerPoint

Table 3 and Figure 5 illustrate the comparative analysis of different performance metrics, such as the accuracy, precision, recall, F1-score and AUC-ROC considered for the evaluation of the performance of the proposed model and other considered models. From the table, it is clear that there is certain uncertainty in the best and average performances of various performance metrics used to evaluate all of the models. It can also be noticed from the table that deep learning approaches are dominant over machine learning and ensemble learning approaches in the classification of sentiment. The proposed LSTM + Firefly algorithm outperformed other models with 0.9951, 0.9956, 0.9951. 0.9979 and 99.59 for the precision, recall, F1-score, AUC-ROC and accuracy, respectively. The LSTM approach attained the second highest accuracy of 99.14 over other conventional models. It is also observed that the AdaBoost approach obtained the lowest accuracy of 80.13%, followed by the MLP with 81.45% accuracy. The other models, such as the DT, KNN, RF, GBoost, Bagging and XGBoost models, obtained accuracy values of 92.32%, 88.30%, 89.44%, 91.62%, 91.62% and 92.89%, respectively. From Table 3, it is also observed that the ensemble learning approaches obtained better accuracy as compared with the machine learning approaches. As the ensemble approaches combine multiple models and assist in reducing the bias or variance, they result in higher predictive accuracy models over machine learning approaches. It is also observed that the deep learning approach obtained superior performance over the ensemble approaches because deep learning approaches optimize the features while extracting. Hence, it can be concluded from the results that the proposed method is able to produce efficient results because of the fine-tuning of the hyperparameters by using the firefly algorithm, which signifies that optimization plays an important role in obtaining better results.

Table 3.  Comparative analysis of the performance metrics of the proposed model and other models.

Classification model	Precision	Recall	F1-Score	AUC-ROC	Accuracy
DT	0.9233	0.9240	0.9232	0.9250	92.32
MLP	0.8167	0.8145	0.8145	0.8176	81.45
KNN	0.8846	0.8819	0.8829	0.8897	88.30
RF	0.8944	0.8967	0.8956	0.9010	89.44
AdaBoost	0.8012	0.8023	0.8012	0.8054	80.13
GBoost	0.9165	0.9162	0.9160	0.9189	91.62
Bagging	0.9165	0.9162	0.9160	0.9189	91.62
XGboost	0.9289	0.9287	0.9270	0.9290	92.89
LSTM	0.9910	0.9912	0.9911	0.9944	99.14
LSTM + Firefly	0.9951	0.9956	0.9951	0.9979	99.59

---

 | Show Table

DownLoad: CSV

Figure 5.  (a)–(e) Comparative analysis of performance metrics of proposed approach and other models.

DownLoad: Full-Size Img PowerPoint

Figure 6(a) to (j) depicts the results for the AUC-ROC curves of the proposed model and other standard approaches. From the figure, it is observed that the proposed LSTM + Firefly attained an AUC-ROC of 1.00 for both the class labels as compared to other conventional methods, which indicates that the suggested approach can test all the instances in the data effectively as compared with others. Even the micro- and macro-average ROC curve values for the proposed approach were equal to 1.00, which indicates that all instances were correctly classified. The efficiency of the proposed model is mainly due to the fine-tuning of the hyperparameters of the LSTM model by using the firefly algorithm. Figure 7 shows how the fitness of the individuals evolves over the course of numerous generations for the LSTM model and LSTM + Firefly model. From all of the experimental results, it can be concluded that the efficiency of the machine learning and ensemble learning approaches used for the classification of sentiments related to COVID-19 can be enhanced if the hyperparameters are fine-tuned using optimization algorithms.

Figure 6.  AUC-ROC curves for a) DT, b) MLP, c) KNN, d) RF, e) AdaBoost, f) Gboost, g) Bagging, h) XGBoost, i) LSTM and j) LSTM + Firefly.

DownLoad: Full-Size Img PowerPoint

Figure 7.  Changes in fitness.

DownLoad: Full-Size Img PowerPoint

Moreover, the study also included a comparative analysis of the results obtained by using the proposed method, along with the results obtained in other studies on the sentiment analysis of COVID-19-related tweets, as shown in Table 4. From the table, it can be concluded that our proposed methodology has obtained better performance in terms of accuracy as compared with the performance of the LSTM models used in other studies for the classification of COVID-19 tweets.

Table 4.  Analysis of results of proposed method with results of other works on the sentiment analysis of COVID-19 related tweets.

Author & Ref	Approach used	Dataset used	Aim of the research	Results
Rustam et al. 2021 [34]	LSTM	7, 528 tweets from IEEE DataPort	Sentiment analysis of COVID-19 tweets	Accuracy = 57.7%
Tripathi 2021 [42]	NB, SVM and LSTM	4, 035 Nepali tweets from Kaggle	Analysis of Nepali tweets related to COVID-19	Accuracy of LSTM = 79%
Proposed Study	LSTM	5, 016 entities from IEEE DataPort	Classification of COVID-19-related tweets into sentiments	Accuracy of LSTM = 99.14
	LSTM + Firefly			Accuracy of LSTM + Firefly = 99.59

---

 | Show Table

DownLoad: CSV

6.   Conclusion

Since its origin, COVID-19 has been considered as one of the biggest challenges to human life all over the world. Researchers and government agencies across the world are continuously working on this disease to control the spread of this disease. Social media has created a great impact on the lives of people, as most people use these media to update the information related to COVID-19. In most of the situations, the information shared on the social media platform has misguided the people about the COVID-19 pandemic. which in turn created an adverse impact on the mental and well-being of the people. Therefore, in this study, we proposed a deep learning approach based on LSTM for the classification of tweets related to COVID-19 as positive and negative sentiments. The proposed approach also makes use of the firefly algorithm to fine-tune the hyperparameters of the LSTM approach. Further, the proposed model and other state-of-the-art models have been evaluated by using various performance metrics; the experimental results revealed that the proposed LSTM + Firefly model outperformed other approaches with an accuracy of 99.59%, precision and F1-score of 99.51%, recall of 99.56% and AUC-ROC of 99.79%. However, in recent days, many advanced machine learning-based approaches have been found to be successful in solving pattern recognition problems ^[53]. As future work, a fair comparative analysis may be conducted among the classical machine learning approaches, advanced approaches and the proposed framework to verify the superiority of adapting various related complex real-world problems. Moreover, the suggested approach can be extended to analyze the reactions regarding COVID-19 vaccinations due to the increase of anti-vaccine sentiments.

Conflict of interest

The authors declare that this manuscript has no conflict of interest with any other published source and has not been published previously (partly or in full). No data have been fabricated or manipulated to support our conclusions. There is no funding agencies involved in this research.