A survey of gradient methods for solving nonlinear optimization

  • Received: 01 October 2020 Revised: 01 November 2020 Published: 09 November 2020
  • Primary: 65K05, 90C30; Secondary: 90C06, 90C26

  • The paper surveys, classifies and investigates theoretically and numerically main classes of line search methods for unconstrained optimization. Quasi-Newton (QN) and conjugate gradient (CG) methods are considered as representative classes of effective numerical methods for solving large-scale unconstrained optimization problems. In this paper, we investigate, classify and compare main QN and CG methods to present a global overview of scientific advances in this field. Some of the most recent trends in this field are presented. A number of numerical experiments is performed with the aim to give an experimental and natural answer regarding the numerical one another comparison of different QN and CG methods.

    Citation: Predrag S. Stanimirović, Branislav Ivanov, Haifeng Ma, Dijana Mosić. A survey of gradient methods for solving nonlinear optimization[J]. Electronic Research Archive, 2020, 28(4): 1573-1624. doi: 10.3934/era.2020115

    Related Papers:

    [1] Loris Nanni, Michelangelo Paci, Gianluca Maguolo, Stefano Ghidoni . Deep learning for actinic keratosis classification. AIMS Electronics and Electrical Engineering, 2020, 4(1): 47-56. doi: 10.3934/ElectrEng.2020.1.47
    [2] Tamizhelakkiya K, Sabitha Gauni, Prabhu Chandhar . Modulation classification analysis of CNN model for wireless communication systems. AIMS Electronics and Electrical Engineering, 2023, 7(4): 337-353. doi: 10.3934/electreng.2023018
    [3] Suriya Priya R Asaithambi, Sitalakshmi Venkatraman, Ramanathan Venkatraman . Proposed big data architecture for facial recognition using machine learning. AIMS Electronics and Electrical Engineering, 2021, 5(1): 68-92. doi: 10.3934/electreng.2021005
    [4] Abdullah Yahya Abdullah Amer, Tamanna Siddiqu . A novel algorithm for sarcasm detection using supervised machine learning approach. AIMS Electronics and Electrical Engineering, 2022, 6(4): 345-369. doi: 10.3934/electreng.2022021
    [5] Habib Hadj-Mabrouk . Analysis and prediction of railway accident risks using machine learning. AIMS Electronics and Electrical Engineering, 2020, 4(1): 19-46. doi: 10.3934/ElectrEng.2020.1.19
    [6] Manisha Bangar, Prachi Chaudhary . A novel approach for the classification of diabetic maculopathy using discrete wavelet transforms and a support vector machine. AIMS Electronics and Electrical Engineering, 2023, 7(1): 1-13. doi: 10.3934/electreng.2023001
    [7] J. Rajeshwari, M. Sughasiny . Modified PNN classifier for diagnosing skin cancer severity condition using SMO optimization technique. AIMS Electronics and Electrical Engineering, 2023, 7(1): 75-99. doi: 10.3934/electreng.2023005
    [8] Deven Nahata, Kareem Othman . Exploring the challenges and opportunities of image processing and sensor fusion in autonomous vehicles: A comprehensive review. AIMS Electronics and Electrical Engineering, 2023, 7(4): 271-321. doi: 10.3934/electreng.2023016
    [9] Amleset Kelati, Hossam Gaber, Juha Plosila, Hannu Tenhunen . Implementation of non-intrusive appliances load monitoring (NIALM) on k-nearest neighbors (k-NN) classifier. AIMS Electronics and Electrical Engineering, 2020, 4(3): 326-344. doi: 10.3934/ElectrEng.2020.3.326
    [10] Abdul Yussif Seidu, Elvis Twumasi, Emmanuel Assuming Frimpong . Hybrid optimized artificial neural network using Latin hypercube sampling and Bayesian optimization for detection, classification and location of faults in transmission lines. AIMS Electronics and Electrical Engineering, 2024, 8(4): 508-541. doi: 10.3934/electreng.2024024
  • The paper surveys, classifies and investigates theoretically and numerically main classes of line search methods for unconstrained optimization. Quasi-Newton (QN) and conjugate gradient (CG) methods are considered as representative classes of effective numerical methods for solving large-scale unconstrained optimization problems. In this paper, we investigate, classify and compare main QN and CG methods to present a global overview of scientific advances in this field. Some of the most recent trends in this field are presented. A number of numerical experiments is performed with the aim to give an experimental and natural answer regarding the numerical one another comparison of different QN and CG methods.



    In late 2019, the world witnessed one of the most severe health disasters that affected all countries [1,2,3,4,5,6]. Spotted in Wuhan City, Hubei Province, China, Covid-19 is a respiratory disease that is caused by the new coronavirus SARS-CoV-2 (Severe Acute Respiratory Syndrome Coronavirus 2) [7,8,9,10]. Coronavirus is a pathogenic disease that starts by attacking the respiratory tract and then directs itself into the cells using an enzyme called furin and impairs the immune system [10]. The primary signs and symptoms of COVID-19 include fever (38℃ or higher), dry cough, dyspnea, sore throat, tiredness, pains, and, in certain cases, diarrhea. Loss of taste and smell is reported as Covid-19 symptoms. Respiratory problems are observed in those developing more severe forms, and these can result in hospitalization in critical care as well as death [7,11,12,13]. The World Health Organization declared, on February 11, 2020, this new disease to be a pandemic after it took a few weeks to spread rapidly throughout the world [7]. While writing this review, the accumulated positive cases is 698,035,072, including 6,940,130 deaths, and 669,603,810 recovered [14] (Last updated: November 20, 2023, 17:56 GMT).

    Faced with this complicated situation, many precautions have been taken to minimize the spread of Covid-19: Washing hands with soap frequently, using hydroalcoholic solution, keeping social distance (2 meters), covering mouth while sneezing, and wearing a mask [7,15,16]. The precautions taken did not stop there. Rather, movement between cities and countries was prevented, and a lockdown was imposed [17,18]. Due the demand to effective solutions of this disease and after deep research in laboratories, researchers developed Covid-19 vaccines. Indeed, different types of vaccines are available: BNT162 (Pfizer BioNTech, USA), ChAdOx1 (AstraZeneca, UK), mRNA1273 (Moderna, USA), Ad26.COV2-S (Johnson & Johnson, USA), BIBP-CorV (Sinopharm, China), CoronaVac (Sinovac, China), and Sputnik V (Gamaleya, Russia) [19,20,21,22].

    Considered a gold standard to detect Covid-19, Reverse Transcription-Polymerase Chain Reaction (RT-PCR) can detect the virus's genetic material, offering high accuracy. However, the need for specialized equipment and well-trained personnel and the time to confirm the patients as well as its expensiveness are the considerable obstacles of its use for testing in this pandemic [10,23]. To overcome this issue, medical images analysis is used massively in this field and allows doctors and specialists to make accurate diagnoses and eliminate the need for laborious manual review processes [24]. Indeed, X-Ray and CT (Computed Tomography) images are the most used in processing of medical images. The utilization of these images required a very good expertise to detect Covid-19 which is taking a lot of time [8]. So, the necessity of new techniques allowing the detection of the virus effectively with minimal time is crucial. Hence, the combination of Deep Learning and Machine Learning methods with X-Ray and CT images is one of the solutions that helps to beat this problem [25]. In fact, Machine Learning and Deep Learning have recently played a major role for automated medical diagnostics by providing new solutions overcoming the shortcomings of the classical methods. Therefore, developing automated analysis system based on X-Ray and CT images using Machine Learning and Deep Learning techniques, that allow to detect the anomalies in the images, helped physicians and healthcare professionals to make accurate decision and save a lot of time [26].

    We aim to focus on classification and detection of Covid-19 using X-Ray and CT images through Deep Learning and Machine Learning techniques. Based on a qualitative and quantitative analysis of published documents on this field and the use of new technologies to improve the diagnosis of Covid-19, we highlight the role of Deep Learning and Machine Learning methods on recognizing Covid-19 from medical images. Data analysis is used to get more insights from the data to determine some information's such as: The distribution of published papers in all used databases, and the most used techniques in the contributions. The participation of each country in terms of the number of published documents was considered as well as the type of publication. The rest of this paper is organized as follows. In Section 2, we describe the methodology of the bibliometric analysis employed, along with the tools used for this analysis. Section 3 delves into various aspects of bibliometric analysis, including the techniques used, annual trends, types of publications, geographic dispersion, leading authors, and predominant journals. Finally, Section 4 provides a detailed breakdown analysis for each database.

    The first and the most important step in any bibliometric analysis is the selection of the relevant databases which pertain to the objectives of the study [27]. The scope of bibliometric analysis is limited by the type of accessible information [28]. It is therefore imperative that the information sources be trusted and adapted to conduct such an analysis and make the most efficient decisions [29]. For our study, we used six well known databases: IEEE Xplore, ACM, MDPI, PubMed, Springer, and ScienceDirect.

    Once the databases have been selected, the second step is to pick out the appropriate indicators for evaluating the resulting samples. The published papers articles showcase a range of bibliometric measures [29]. We can distinguish three different types of bibliometric indicators: (1) Quality Indicators, which can measure the impact, (2) Quantity Indicators that allow us to measure the productivity, and (3) Structural Indicators that measure the connection [30]. Figure 1 shows the types of bibliometric indicators. In our study, we focused on quantity indicators.

    Figure 1.  Bibliometric indicator types.

    Our purpose of this study is to investigate the research area of classification and detection based on X-Ray and CT images using Deep Learning and Machine Learning techniques. We conducted our review using IEEE Xplore, ACM, MDPI, PubMed, Springer, and ScienceDirect databases. In our bibliometric analysis, we covered the most prevalent areas of knowledge, the prolific authors, the journals and the most publications, the most productive countries, the number of citations, the trends of publications, document types, and countries or territories of origin. To carry out our research, we used the same advanced search string for all the selected databases:

    ("Covid" OR "Covid19" OR "Covid 19" OR "Covid-19" OR "Coronavirus" OR "Corona Virus") AND ("Classification" OR "Image Classification") AND ("Detection" OR "Image Detection") AND ("Deep Learning" OR "Deep Neural Networks" OR "DL" OR "Convolutional Neural Network" OR "CNN") AND ("X-Ray" OR "X-Ray Image" OR "CT" OR "Computed Tomography") AND ("Machine Learning" OR "Machine Learning Algorithms" OR "ML")

    The methodology followed in our bibliometric analysis is described in Figure 2. First, it begins with selecting the databases (IEEE Xplore, ACM, MDPI, PubMed, Springer, and ScienceDirect). An advanced string search query is applied across these databases to gather research papers. Then, the results from this search undergo a cleaning process to refine the selection of documents. The specific counts of papers retrieved from each database are indicated: IEEE Xplore (128), ACM (58), MDPI (13), PubMed (35), Springer (217), and ScienceDirect (132). The bibliometric analysis software VOSviewer and Numpy, Pandas, and Matplotlib Python packages are used to conduct the data analysis and visualizations. The cleaning process involves eliminating duplicated documents and documents that do not address classification and detection of Covid-19 based on X-Ray and CT images.

    Figure 2.  Methodology of bibliometric analysis diagram.

    After the cleaning process, we extracted the relevant information for each document and created an excel file to process the data analysis. In the excel file, we stored the following information: Title of the document, year of publication, first author, country (of the first author; in some documents, we found many authors with different countries), type of document (conference paper, journal article, and book chapter), publisher (title of conference or journal), database, number of citations (the last upgrade of number of citation was 6 November 2023), abstract, key words, and the technique used (Deep Learning, Machine Learning or Deep Learning and Machine Learning). The Excel file was used as a primary tool to conduct an initial analysis of the data, which helped us understand the dataset better. This preliminary analysis opened the point for more interpretation and advanced data visualization. With the help of Python and its powerful libraries such as Pandas, Numpy, and Matplotlib, we created detailed graphs and charts that helped us visualize our findings and understand complex patterns within the data more effectively.

    The research resulted in 139, 80, 15, 39,239, and 161 documents in IEEE Xplore, ACM, MDPI, PubMed, Springer, and ScienceDirect, respectively. The cleaning process consists of removing duplicated and documents that do not exactly address the classification and detection of Covid-19 based on X-Ray and CT images using Deep Learning and Machine Learning. The total number of documents, after the cleaning process, for the databases IEEE Xplore, ACM, MDPI, PubMed, Springer, and ScienceDirect is 128, 58, 13, 35,217, and 132 documents, respectively. Figure 3 shows the distribution of total documents of each database over the years.

    Figure 3.  Total documents for each database over the years.

    We used VOSviewer version 1.6.20 as bibliometric analysis software due to its efficiency in analyzing clustered search results. Indeed, VOSviewer is a software tool designed for creating and visualizing networks within bibliometric data. Such networks include journals, researchers, or individual publications, and are formed through various relationships such as citation, bibliometric coupling, co-citation, or co-authorship relations. It also provides text mining capabilities, allowing us to construct and visualize the patterns of the networks of important terms drawn from a body of scientific literature [31,32]. VOSviewer analyzes the text within abstracts and titles of published papers by segmenting the paragraphs into individual words and phrases and connecting them with the citation data of the corresponding papers. The outcomes are presented as a density map or term bubbles. As we can see in the figure below (Figure 4), the density map highlights key areas of research related to Covid-19, including terms like "Covid-19", "deep learning, " and "machine learning" as central nodes. These nodes are surrounded by related terms such as "classification", "computed tomography image", "lung detection, " and "chest x-ray, " suggesting they are common subjects of study in the field. This visualization helps to identify the most prominent topics and techniques addressed in the existing body of research on Covid-19.

    Figure 4.  Cluster density map in the PubMed database.

    To facilitate the bubble map, words/terms occurring at a minimum of 1 time in the publications were examined and visualized, and 148 keywords met the selected threshold. Figure 5 illustrates the obtained results that are in the form of network visualization and density diagrams. Different colors indicate thematic concentrations in the literature, and lines between terms depict the relationships and collaborative nature of these research areas. VOSviewer has generated 16 clusters of terms.

    Figure 5.  Terms map in the PubMed database.

    The total documents used in this review is 584 documents. While extracting the relevant information from the documents, we took into consideration the contribution of each document and the technique used by the authors for classification and detection of Covid-19 based on X-ray and CT image. As a result, authors have used three techniques: Deep Learning, Machine Learning, or the combination of these two techniques. Figure 6 illustrates the distribution of techniques applied within these documents. It reveals that most of the documents, precisely 378 or 64.7%, used Deep Learning techniques, highlighting the predominant role that this method plays in the field. Moreover, Deep Learning and Machine Learning techniques were jointly utilized in 135 documents. A total of 71 documents were reviewed that used Machine Learning alone, representing 12.2% of all documents reviewed. This distribution underscores how important it is for research trends to focus on advanced computational methods, especially Deep Learning.

    Figure 6.  Distribution of techniques used.

    Since the beginning of the Covid-19 pandemic in 2020, academic publications about the virus have increased significantly, especially those using Deep Learning and Machine Learning methods. Indeed, the trend shown by Figure 7 presents the yearly distribution of total documents categorized by the utilized techniques from 2020 to 2024. Deep Learning techniques demonstrated its presence with 39 documents, and a significant increase in 2021 (100 documents), 2022 (125 documents), 2023 (111 documents), and 6 documents have already been published as of 2024, even though we are in 2023. A modest count of 6 documents for Machine Learning in 2020 had a low increase for 2021, 2022, and 2023 with 22, 26, and 16 documents, respectively. During the observed period, the combination of Deep Learning and Machine Learning remains relatively stable, indicating a consistent integration of these two approaches in research practices. As the years progressed, the adoption of Deep Learning and Machine Learning continued to grow. This continual increase underlines the critical role that Deep Learning and Machine Learning have played in understanding and combating the Covid-19 pandemic through innovative research and analysis.

    Figure 7.  Total documents for each technique used per year.

    In our study, we carefully paid attention to the type of document, distinguishing three major categories: Conference paper, journal article, and book chapter. We were able, through this classification, to analyze and understand the dissemination patterns and academic impact of each type of publication within the research environment. Regularly, authors present their initial findings in conference papers and their detailed contributions to the body of knowledge in journal articles, while comprehensive insights into specific areas of study are typically reserved for book chapters. Figure 8 represents the distribution of different publication types. As we can see, with 337 (57.7%) of all publications being journal articles, they make up the majority and play a major role in publishing research findings. Conference papers contain 211 (36.1%) of the publications, which is a notable proportion, indicating their significance as a platform for showcasing preliminary research findings and promoting academic discussion. Although they are fewer in number, book chapters make up 6.2% (36) of the publication because they provide more in-depth analysis and discussion on field-related topics. This distribution demonstrates the variety of channels that the academic community uses to publish research findings.

    Figure 8.  Distribution of type of publication.

    We take the first author's country into account to determine the country of origin for the research contribution in our study. Geographic dispersion of documents across the globe is illustrated in Figure 9. India tops the countries with an impressive count of 196 documents, highlighting its robust research in the topic. Followed by China in second with 61 contributions, illustrates the volume of its academic output. Other notable contributions come from Turkey with 29 papers, and Pakistan and the USA, each contributed with 23 papers. Other noteworthy contributions come from Egypt and Bangladesh, with 21 documents each. A substantial representation of Middle Eastern nations come from Iran and Saudi Arabia with 18 and 15 articles, respectively. Morocco from north Africa contributed 14 documents. The rising forces in economy and technology have challenged the established powers' traditional dominance in the field of global research and are making significant contributions. These emergent forces are leading the way in both academic and practical advances because of their remarkable agility and innovation.

    Figure 9.  Geographic dispersion of documents worldwide.

    The top 5 authors who have made major contributions to the corpus of documents are the focus of this analysis. We explore the specifics of their contributions, looking at the geographical context of their affiliations as well as the volume of published works they have contributed. The number of papers published, the esteemed journals that have published their work, and the corresponding databases are used to gauge their impact. This allows us to recognize the depth of their academic impact and the size of knowledge they contribute to the forefront of their respective fields.

    As mentioned in Figure 10, K. Shankar from India has published four important journal articles in three years. Shankar's research papers received 18 and 38 citations in 2021 when they were published in "ACM Transactions on Internet Technology" [32] and "Applied Soft Computing" [33], respectively, both published by ACM and ScienceDirect. His work continued to garner attention in "Multimedia Systems" [34] published by Springer the following year. Shankar's 2023 publication in "Cognitive Neurodynamics" [35] was also published by Springer. All these articles, which have appeared in respectable journals and databases, have accumulated 118 citations.

    Figure 10.  Top 5 leading authors in the field.

    The Pakistani author Imran Ahmed published two articles in 2022: "Interdisciplinary Sciences: Computational Life Sciences" [36] published by Springer, and one in "Virtual Reality & Intelligent Hardware" [37] hosted on ScienceDirect. Imran made another significant contribution in 2023 when he published an article in "Computing" [38], which was also published by Springer. The total citation of Imran reached 63 citations.

    Santosh Kumar, an Indian researcher, published two journal articles in 2022, having a total of 36 citations. One article was published in "Computer Methods and Programmes in Biomedicine" [39] which is indexed in PubMed Database. His second work was featured in "Computers and Electrical Engineering" [40] on ScienceDirect.

    The Iranian author Mustafa Ghaderzadeh had a remarkable year in 2021. His research was published as a conference paper by ACM and presented at the 5th International Conference on Medical and Health Informatics in 2021 [41]. Furthermore, he published an article in the "Journal of Medical Internet Research" [42] that was indexed in PubMed. He received a total count of 59 citations.

    In 2023, Aysen Degerli, from Finland, made a significant contribution by publishing one journal article in "Health Information Science and Systems" [43] indexed in Springer with 76 citations. In 2022, he published a conference paper in the IEEE International Conference on Image Processing (ICIP), which is indexed in the IEEE Xplore database and earned 9 citations [44].

    The prominence of the rising academic forces is demonstrated by the four leading authors who are from India, Pakistan, and Iran. Indeed, this is demonstrated by showcasing the increasing intellectual talent of their respective nations. These researchers have highlighted an academic ascendancy trend in regions that have been strongly pushing the boundaries of innovation and research.

    A comprehensive overview of the 20 most productive journals in this study with details of corresponding databases, total citations, and average citation per journal is presented in Table 1. The analysis showcases that "Multimedia Tools and Applications" (29 documents), "Computers in Biology and Medicine" (19 documents), and "Expert Systems with Applications" (13 documents) are the most top three journals within their corresponding databases (PubMed for the first journal and ScienceDirect for the two others). These articles earned total citations of 237, 3711, and 399, respectively. With their large number of citations, they established a standard for scholarly impact. A more thorough analysis identifies a trend of significant contributions on several platforms, including ScienceDirect, Springer, and IEEE Xplore. This trend across databases demonstrates how research publications are dynamic and how journals bear witness to how academic discourse is changing. These results highlight the critical role that highly regarded journals play in promoting knowledge advancement among the international research community.

    Table 1.  Top 20 Journals ranked by productivity in the study.
    Journal Title Database Total Documents Total Citation Average Citation per Journal
    Multimedia Tools and Applications PubMed 29 237 8.17
    Computers in Biology and Medicine ScienceDirect 19 3711 195.32
    Expert Systems with Applications ScienceDirect 13 399 30.69
    Applied Soft Computing ScienceDirect 12 632 52.67
    Biomedical Signal Processing and Control ScienceDirect 12 181 15.08
    Journal of Ambient Intelligence and Humanized Computing Springer 10 290 29.00
    Scientific Reports Springer 9 462 51.33
    Applied Intelligence Springer 8 1422 177.75
    IEEE Access IEEE Xplore 7 2637 376.71
    Informatics in Medicine Unlocked ScienceDirect 7 757 108.14
    Neural Computing and Applications Springer 7 106 15.14
    SN Computer Science Springer 5 78 15.60
    ACM Transactions on Multimedia Computing, Communications, and Applications ACM 4 132 33.00
    Computers and Electrical Engineering ScienceDirect 4 29 7.25
    Computer Methods and Programs in Biomedicine PubMed 4 615 153.75
    Procedia Computer Science ScienceDirect 4 39 9.75
    Neurocomputing ScienceDirect 4 48 12.00
    Pattern Recognition ScienceDirect 4 314 78.50
    Soft Computing Springer 4 185 46.25
    Knowledge-Based Systems ScienceDirect 3 74 24.67

     | Show Table
    DownLoad: CSV

    From Figure 11, we can notice that there is no clear trend indicating a direct proportionate relationship between the number of documents published and average citations. This implies that more publications do not necessarily lead to a higher average citation count per document.

    Figure 11.  A 3D scatter plot for journal metrics.

    The total documents that have been published in IEEE Xplore database is 128 documents. As depicted in Figure 12, most published papers used Deep Learning techniques, accounting for 77 documents of research, which is showing its significant role in studies. Machine Learning techniques and the combined use of Deep Learning and Machine Learning are almost in the same range with 23 and 28 documents, respectively.

    Figure 12.  Distribution of techniques used for the IEEE Xplore database.

    For the distribution of publication type in IEEE Xplore, as shown in Figure 13, the first remark that we can raise is the absence of book chapters. Only conference papers, which are predominant, with 113 documents, and journal articles with 15 documents and are published. This distribution clearly highlights conference proceedings within the IEEE community, reflecting the typical patterns of academic distribution in the field.

    Figure 13.  Distribution of publication type for the IEEE Xplore database.

    The graph in Figure 14 represents the top 10 contributing countries to the IEEE Xplore database. India is in the lead with a substantial 51 documents, underscoring the country's significant research output. Indonesia came in second with 10 publications, while the USA, Bangladesh, and Canada each contributed 7 documents. With four publications, Turkey follows closely in demonstrating its scholarly presence. Notable contributors include Saudi Arabia, Qatar, Morocco, and the UK, each of which has three publications in the database.

    Figure 14.  Top 10 Countries contributed to publication for IEEE Xplore database.

    For the IEEE Xplore Database, all authors have published one single document in the IEEE Xplore database. Indeed, Abbas Mazrouei Sebdani, from Iran, published a conference paper [45] on the 5th International Conference on Pattern Recognition and Image Analysis (IPRIA) in 2021. He proposed two classification algorithms of Covid-19 using Machine Learning. In 2020, the south Korean researcher, Abdul Waheed, has published a journal article [46] where he proposed a method to generate synthetic chest X-ray (CXR) images by developing an Auxiliary Classifier Generative Adversarial Network (ACGAN). Using a deep unsupervised framework to classify lung diseases from chest CT and X-ray images, Pooja Yadav (India) published in IEEE Access [47]. Oussama El Gannourm, from Morocco, presented a Deep Learning based system for the diagnosis of COVID19 disease at the IEEE 2nd International Conference on Electronics, Control, Optimization, and Computer Science (ICECOCS) [48]. Using Deep Learning and Machine Learning, a framework named CovFrameNet., which consist of a pipelined image pre-processing method and a deep learning model for feature extraction, classification, and performance measurement, has been published by Olaide Nathaniel Oyelade from South Africa on IEEE Access [49].

    According to Table 2, "IEEE Access" is the prominent journal in IEEE Xplore with a total of 7 documents and an impressive total of 2637 citations. Followed by "IEEE Journal of Biomedical and Health Informatics", "IEEE Transactions on Engineering Management", and "IEEE Transactions on Medical Imaging" participating with 2 documents each with total citations of 72, 33, and 186, respectively. With only one document, "IEEE Transactions on Big Data" shows its weight in the field with 70 citations.

    Table 2.  Top 5 journals ranked by productivity for IEEE Xplore database.
    Journal Title Database Total Documents Total Citation Average Citation per Journal
    IEEE Access IEEE Xplore 7 2637 376.71
    IEEE Journal of Biomedical and Health Informatics IEEE Xplore 2 72 36.00
    IEEE Transactions on Engineering Management IEEE Xplore 2 33 16.50
    IEEE Transactions on Medical Imaging IEEE Xplore 2 186 93.00
    IEEE Transactions on Big Data IEEE Xplore 1 70 70.00

     | Show Table
    DownLoad: CSV

    58 documents have been published in ACM database. Figure 15 illustrates that most documents used Deep Learning technique (46 documents) while Machine Learning and combined Deep Learning and Machine Learning techniques are less used accounting for 8 and 4, respectively. This makes us conclude the importance of Deep Learning techniques in ACM platform.

    Figure 15.  Distribution of techniques used for ACM database.

    For the ACM database (Figure 16), only conferences articles and journal articles have been published with a dominance of conference articles around 76% (44 documents) and only 24% (14 documents) for journal articles. This distribution highlights the dual focuses of ACM database, which are to contribute to the body of knowledge with journal articles and to promote the academic record through the conferences.

    Figure 16.  Distribution of publication type for the ACM database.

    Figure 17 is demonstrating China's remarkable advantage with 14 documents. India comes in second with 9 documents, while the USA contributes 6 publications. Greece, Thailand, Taiwan, and Saudi Arabia participated with 4 documents. With 2 documents, the UK contributed to the publication, although Brazil and Canada marked their presence with 1 document. This data shows a geographically varied collection of contributions to the ACM database, illustrating the international scope and cooperative character of modern research initiatives among different countries.

    Figure 17.  The top 10 Countries that contributed to publications for the ACM database.

    As for the ACM database, we do not have duplicated authors. Indeed, the paper of Abhishek Kumar, from India, was published for the 13 Indian Conference on Computer Vision, Graphics, and Image Processing (ICVGIP) [50]. He presented a comparative analysis of the most well-known CNN models to classify X-Ray images. The 12th ACM Conference on Bioinformatics, Computational Biology, and Health Informatics was an opportunity for the Georgian Tarun Naren [51] to publish his work where he developed a new variant of Model-Agnostic Meta-Learning (MAML) algorithm, named MAML++, in order to classify X-Ray images. From Vietnam, Pham Ngoc Ha [52] in 2023, participated in the 8th International Conference on Intelligent Information Technology and published his paper. He used Vision transformer architecture to extract data characteristics and classify images. In 2020, a system for automatic detection of COVID-19 from CXR images using machine learning techniques was proposed by Phongsathorn Kittiworapanya [53] on the Eleventh International Conference on Computational Systems-Biology and Bioinformatics. The year 2023 was when Robert Hertel [54] published a hit paper in ACM Computing Surveys. He presented an in-depth review of deep learning techniques on the purpose of diagnosis and prognosis of COVID-19 patients.

    Table 3 presents the top 5 Journals Ranked by Productivity for ACM database. Indeed, "ACM Transactions on Multimedia Computing, Communications, and Applications" stands out as the most productive journal in the ACM database with 4 papers and 132 citations overall, followed by "ACM Transactions on Management Information Systems" having 3 publications and a total of 47 citations. Publishing 2 documents, "ACM Transactions on Internet Technology" and "IEEE/ACM Transactions on Computational Biology and Bioinformatics" got overall citations of 33 and 957, respectively. "ACM Computing Surveys" has completed the list of the top five productive journals with one contribution totaling five citations.

    Table 3.  Top 5 Journals ranked by productivity for ACM database.
    Journal Title Database Total Documents Total Citation Average Citation per Journal
    ACM Transactions on Multimedia Computing, Communications, and Applications ACM 4 132 33.00
    ACM Transactions on Management Information Systems ACM 3 47 15.67
    ACM Transactions on Internet Technology ACM 2 33 16.50
    IEEE/ACM Transactions on Computational Biology and Bioinformatics ACM 2 957 478.50
    ACM Computing Surveys ACM 1 5 5.00

     | Show Table
    DownLoad: CSV

    Participating with only 13 documents in this study, MDPI possesses the smallest collection of documents. The distribution of published documents (Figure 18) according to the techniques used is that more than 61% (8 documents) of the techniques used are Deep Learning, and the mixture of Deep Learning and Machine Learning is highlighted in four documents (30.8%). The use of Machine Learning was used only in one document.

    Figure 18.  Distribution of techniques used for the MDPI database.

    Journal articles make up 100% of the publications and are the only contributor. The complete dominance of journal articles across all 13 documents in the dataset suggests that the ACM database places a high value on formal, peer-reviewed articles. The focused nature of this collection is highlighted by the absence of other publication types like conference papers or book chapters.

    The top 10 nations that have contributed to the MDPI database are shown in Figure 19. Turkey leads the count with 3 documents. Following with 2 contributions, China and Pakistan show that they are actively involved in academic publishing within MDPI. Additionally, Bangladesh, Egypt, Iraq, Jordan, Saudi Arabia, and South Korea, contributed one document each.

    Figure 19.  The top 10 Countries that contributed to publications for the MDPI database.

    There was one contribution per author from the MDPI database. Indeed, Ali Tariq Nagi [55], from Pakistan, has a paper that involves an evaluation of the performance of deep learning models for COVID-19 diagnosis using chest X-ray images from a dataset containing the largest number of COVID-19 images ever used in the literature. The Iraqi, Awf Ramdhan [56] proposed an approach that uses image cropping methods and a deep learning technique that classifies three public datasets. In [57], Bayan Alsaaidah from Jordan conducted a comprehensive review of Deep Learning and Machine Learning methods for detection and classification of Covid-19. Using Deep Learning techniques, a computer-aided diagnostic system for chest X-ray and CT images for Covid-19 classification was developed by Heba M. Emara [58] from Egypt. A Turkish researcher, Jawad Rasheed [59], proposed a computer vision and artificial-intelligence-based hybrid approach aid in efficient detection and control of the COVID-19 disease.

    The top 5 Journals Ranked by Productivity for the MDPI database are presented in Table 4. With 3 published documents, "Diagnostics" leads the 5 productive journals in the MDPI database, and having 10 citations, was followed by "Applied Sciences" and "Sensors" with a count of 2 documents and an overall of citation of 33 and 27, respectively. "Bioengineering" with 15 citations and "Cancers" had 6 citations as their publications, with one document each.

    Table 4.  Top 5 Journals ranked by productivity for the MDBI database.
    Journal Title Database Total Documents Total Citation Average Citation per Journal
    Diagnostics MDPI 3 10 3.33
    Applied Sciences MDPI 2 33 16.50
    Sensors MDPI 2 27 13.50
    Bioengineering MDPI 1 15 15.00
    Cancers MDPI 1 6 6.00

     | Show Table
    DownLoad: CSV

    In this study, 35 documents were selected in the PubMed database. Figure 20 displays the distribution of techniques used. Having almost 50% (17 documents), Deep Learning was used as a technique for detection and classification of Covid-19 in the PubMed database. Deep Learning and Machine Learning were utilized jointly with a count of 14 documents (40%). The use of Machine Learning as a technique appeared only 4 times with a percentage of 11.4%. This brings us to the superiority of the use of Deep Learning techniques in the PubMed database.

    Figure 20.  Distribution of techniques used for the PubMed database.

    Same as for the ACM database, only journal articles were published in the PubMed database. With the absence of other publication types, such as book chapters or conference papers, we can underline that peer-reviewed articles are the focused nature of the PubMed database.

    Leading the top 10 countries that contributed to publication, India published 8 documents, followed by the UK with 5 publications. Three published documents were the participation of China and Iran each. Not far from them is South Africa, with 3 published documents. Other countries include Pakistan and the USA with 2 publications, and Australia, Greece, and Italy contributed with 1 document each. Figure 21 displays the top 10 Countries that contributed to publications for the PubMed database.

    Figure 21.  The top 10 Countries that contributed to publications for the PubMed database.

    As displayed in Figure 22, from Saudi Arabia, Mahmoud Ragad published 2 journal articles in 2021 and 2022. In his first paper [60], he presented an artificial intelligence-based ensemble model for detection and classification (AIEM-DC) that aims to accurately detect and classify the COVID-19 using an ensemble of DL models. In [61], Mahmoud Ragad proposed a new Quantum Seagull Optimization Algorithm with a DL-based COVID-19 diagnosis model, named the QSGOA-DL technique to detect and classify COVID-19 with the help of CXR images. Abdullah Alen, from Turkey, proposed a Residual Convolutional Neural Network (ResCNN) used as a deep learning method, with the assistance of machine learning algorithms and extracted features from images [62]. An effective COVID-19 detection and classification approach using the Shufflenet CNN by employing three types of images (chest radiograph, CT-scan, and ECG-trace images) was proposed by the Pakistani Naeem Ullah [63]. The main idea of Linh T Duong, from Vietnam, in [64] is exploiting cutting-edge Machine Learning techniques for the detection of Covid-19 from chest X-ray (CXR) and lung computed tomography (LCT) images. From Italy, Luca Saba [65] presented a comparative study between Deep Learning and Machine Learning algorithms for the detection of Covid-19 based on radiological computed tomography (CT) lung scans.

    Figure 22.  Leading authors in the field of the PubMed database.

    Table 5 displays the 5 most productive journals in the PubMed database. With 3 documents, "Diagnostics (Basel)" ranks first with 29 citations, followed by "Computational Intelligence and Neuroscience" with 2 published papers, which got 8 citations. "Mathematical Biosciences and Engineering", "Journal of Healthcare Engineering", and "Journal of Medical Internet Research" have participated in this field by publishing 1 article each with total citations of 8, 10, and 59, respectively.

    Table 5.  Top 5 Journals ranked by productivity for the PubMed database.
    Journal Title Database Total Documents Total Citation Average Citation per Journal
    Diagnostics (Basel) PubMed 3 29 9.67
    Computational Intelligence and Neuroscience PubMed 2 8 4.00
    Mathematical Biosciences and Engineering PubMed 1 8 8.00
    Journal of Healthcare Engineering PubMed 1 10 10.00
    Journal of Medical Internet Research PubMed 1 59 59.00

     | Show Table
    DownLoad: CSV

    Springer is the database that has the highest number of documents in this study by contributing 217 documents. With 148 journal articles, Deep Learning dominated the techniques used in Springer database with a percentage of 68.2%. Merged Deep Learning and Machine Learning techniques have been used in 55 documents, achieving 25.3% of all documents. Moreover, the use of Machine Learning techniques occurred only in 14 documents (6.5%). This distribution used techniques that signify the research trends and priorities within the Springer database. Figure 23 shows the distribution of techniques used for the Springer database.

    Figure 23.  Distribution of techniques used for the Springer database.

    With a percentage of 60.8% (132 documents), the publication type of published documents in Springer are journal articles with a huge part (see Figure 24). A considerable number of types of publications are conference papers, accounting for 54 documents or 24.9%. Book chapters got a significant part of publication type with 31 documents (14.3%). This means that springer is a database that opens all the opportunities to researchers to publish their works.

    Figure 24.  Distribution of publication type for the Springer database.

    As we can see in Figure 25, Indian researchers lead the top 10 countries that contributed to publication with a very high number of documents (90 publications). This carries us to India's superiority in the field of scientific research in recent years. With a very big gap between India, China came second with 18 documents, followed by Pakistan, Egypt, and Turkey in the same range of published documents, with a count of 13, 11, and 10 documents, respectively. Morocco and the USA marked their presence with 8 publications each. Furthermore, Bangladesh, Iraq, and Algeria have participated with 5 documents each.

    Figure 25.  The top 10 countries that contributed to publications in the Springer database.

    The top 5 researchers in Springer participated with 2 documents each. Indeed, an Indian researcher, K. Shankar, who came first with publications [66], presented a new metaheuristic-based fusion model for COVID-19 diagnosis using chest X-ray images using Weiner filtering for preprocessing, grey level co-occurrence matrix for feature extraction, and artificial neural network for classification. In 2023, an automated COVID-19 diagnosis process using Convolutional Neural Network (CNN) with a fusion-based feature extraction model (FM-CNN) was developed in his second paper [67]. In 2021, Sourabh Shastri, from India, designed a nested ensemble model using deep learning methods based on long short-term memory (LSTM) [68]. A CheXImageNet model has been introduced for detecting Covid-19 disease using digital images of Chest X-ray with the help of an openly accessible dataset in his second paper [69]. The work done by the Pakistani Kainat Khero in [70] was a review of ML and DL-based techniques for early detection of Covid using chest X-rays. A comparison study of 9 state-of-the-art CNN architectures through a transfer learning approach for detection of Covid-19 was presented in his second work [71]. Siddharth Gupta, from India, presented pre-trained CNN models (VGG16 and VGG19) that were used for transfer learning and several machine learning classifiers for the classification of images, such as Covid19 [72]. In [73], he employed DL-based automated techniques (VGG16, VGG19, and Inception V3) to process the chest X-ray images for the detection of Covid-19. The Indian author Ebenezer Jangam [74] proposed a stacked ensemble of heterogenous pre-trained computer vision models where 4 pre-trained DL models were considered (VGG19, ResNet101, DenseNet169, and WideResNet502). His second work [75] was about a novel multi-class classification framework that can minimize either false positives or false negatives, and is useful in computer aided diagnosis or computer aided detection, respectively.

    Publishing 28 documents with a total citation of 222 citations, "Multimedia Tools and Applications" came out on top of the productive journal in Springer (see Table 6). With a considerable difference of 18 documents, "Journal of Ambient Intelligence and Humanized Computing" took the second rank, with 290 citations. "Applied Intelligence" and "Scientific Reports" have published 8 documents each with 1422 and 462 citations respectively. Six publications and 106 citations allowed "Neural Computing and Applications" to be in the top 5 productive journal in Springer.

    Table 6.  Top 5 Journals ranked by productivity for the Springer database.
    Journal Title Database Total Documents Total Citation Average Citation per Journal
    Multimedia Tools and Applications Springer 28 222 7.93
    Journal of Ambient Intelligence and Humanized Computing Springer 10 290 29.00
    Applied Intelligence Springer 8 1422 177.75
    Scientific Reports Springer 8 462 57.75
    Neural Computing and Applications Springer 6 106 17.67

     | Show Table
    DownLoad: CSV

    With 132 collected documents, ScienceDirect comes in second of all the databases concerned in this study. From Figure 26, we can conclude that Deep Learning techniques are the most used with a total count of 81 documents, which is almost a huge portion of all technique used. The use of Deep Learning and Machine Learning together appeared in 30 (22.7%) papers while Machine Learning as type of technique occurred in 21 (15.9%) documents.

    Figure 26.  Distribution of techniques used for ScienceDirect database.

    As illustrated in Figure 27, more than 96% (127 papers) of documents published in ScienceDirect are journal articles. This shows the total dominance of this type of publication, and it is the main way for researchers to publish their works in ScienceDirect. Even though they are less in number, book chapters make up 3.8% of all publications (5 documents).

    Figure 27.  Distribution of publication type for ScienceDirect database.

    Like almost all the databases, India tops the 10 countries that contributed to publications in ScienceDirect with 38 documents, which confirms its dominance in this study. India is followed by China, with a significant gab accounting for 21 published papers. Turkey maintained its presence among the countries contributing to publication in this database. Bangladesh and Iran contributed 8 documents, while Egypt and Pakistan h 7 and 5 times, respectively. South Korea, UAE, and Saudi Arabia insisted on being included among the countries with 3 documents each. Figure 28 displays the top countries that contributed to the publications for the ScienceDirect database.

    Figure 28.  The top 10 countries that contributed to publications for the ScienceDirect database.

    The year 2021 was a special year for the Pakistani Saddam Hussain Khan when he published two papers (Figure 29). In [76], he proposed two new deep learning frameworks: Deep Hybrid Learning (DHL) and Deep Boosted Hybrid Learning (DBHL) for effective COVID-19 detection in X-ray dataset. He developed, in his second work [77], two novel custom CNN architectures (COVID-RENet-1 and COVID-RENet-2) for COVID-19 specific pneumonia analysis. The Indian researcher, Aakarsh Malhotra [78], presented a COVID-19 Multi-Task Network (COMiT-Net), which is an automated end-to-end network for COVID-19 screening and that can perform semantic segmentation of the regions of interest to make the model explainable. Originally from India, Sakthivel R [79] built an efficient hardware architecture based on an ensemble deep learning model to identify the COVID-19 using chest X-ray (CXR) records. Saeed Iqbal [80] suggested a technique allowing a class balancing algorithm to understand and detect lung disease signs from chest X-ray and CT images. He used Deep learning techniques to train and evaluate images, enabling the extraction of basic visual attributes. Roa'a Mohammedqasem [81], from Turkey, proposed a real-time detection system based on the Internet of Things framework. The system can collect real-time data from users to determine potential coronavirus cases, analyze treatment responses for people who have been treated, and accurately collect and analyse the datasets.

    Figure 29.  Leading authors in the field for the ScienceDirect database.

    As presented in Table 7, the journal "Computers in Biology and Medicine" came in the top 5 productive journals with 19 published documents and accumulating a total of 3711 citations. With 13 papers having 221 citations, "Expert Systems with Applications" ranked in second place, followed by "Biomedical Signal Processing and Control" and "Applied Soft Computing" with 12 and 11 documents and receiving 181 and 622, respectively. "Informatics in Medicine Unlocked" announced its position within the productive journals with 7 documents accounting for a total of 757 citations.

    Table 7.  Top 5 Journals ranked by productivity for the ScienceDirect database.
    Journal Title Database Total Documents Total Citation Average Citation per Journal
    Computers in Biology and Medicine ScienceDirect 19 3711 195.32
    Expert Systems with Applications ScienceDirect 13 221 17.00
    Biomedical Signal Processing and Control ScienceDirect 12 181 15.08
    Applied Soft Computing ScienceDirect 11 622 56.55
    Informatics in Medicine Unlocked ScienceDirect 7 757 108.14

     | Show Table
    DownLoad: CSV

    In this review, we highlighted the use of Deep Learning and Machine Learning techniques to detect and classify Covid-19 using X-Ray and CT images. The bibliometric analysis carried out for this study revealed a significant number of documents describing the importance of the use of Deep Learning in this field compared to Machine learning methods. Moreover, one conclusion we can draw from this study is the rising forces in economy and technology, especially India, China, Turkey, and Pakistan, which began to compete in the field of scientific research. Additionally, journal articles, as a publication type, dominated the type of publication compared to Conference Papers and Book Chapters. This means that authors prefer to publish their detailed contributions to the field of knowledge in journals than publish their initial findings in conference papers or their comprehensive insights into specific areas in Book Chapters. Furthermore, authors focused on the Springer database to submit their works for publication, which is explained by the high number of published documents in this database compared to other databases.

    The authors declare that they have not used Artificial Intelligence (AI) tools in the creation of this article.

    The authors declare that there are no conflicts of interest in this paper.



    [1] A new reprojection of the conjugate directions. Numer. Algebra Control Optim. (2019) 9: 157-171.
    [2] Descent property and global convergence of the Fletcher-Reeves method with inexact line search. IMA J. Numer. Anal. (1985) 5: 121-124.
    [3] An unconstrained optimization test functions collection. Adv. Model. Optim. (2008) 10: 147-161.
    [4] An acceleration of gradient descent algorithm with backtracking for unconstrained optimization. Numer. Algorithms (2006) 42: 63-73.
    [5] A Dai-Liao conjugate gradient algorithm with clustering of eigenvalues. Numer. Algorithms (2018) 77: 1273-1282.
    [6] N. Andrei, Relaxed gradient descent and a new gradient descent methods for unconstrained optimization, Visited August 19, (2018).
    [7] N. Andrei, Nonlinear Conjugate Gradient Methods for Unconstrained Optimization, 1st edition, Springer International Publishing, 2020. doi: 10.1007/978-3-030-42950-8
    [8] Minimization of functions having Lipschitz continuous first partial derivatives. Pacific J. Math. (1966) 16: 1-3.
    [9] The Dai-Liao nonlinear conjugate gradient method with optimal parameter choices. European J. Oper. Res. (2014) 234: 625-630.
    [10] B. Baluch, Z. Salleh, A. Alhawarat and U. A. M. Roslan, A new modified three-term conjugate gradient method with sufficient descent property and its global convergence, J. Math., 2017 (2017), Article ID 2715854, 12 pages. doi: 10.1155/2017/2715854
    [11] Two-point step-size gradient method. IMA J. Numer. Anal. (1988) 8: 141-148.
    [12] M. Bastani and D. K. Salkuyeh, On the GSOR iteration method for image restoration, Numerical Algebra, Control and Optimization, (2020). doi: 10.3934/naco.2020013
    [13] An evaluation of quasi-Newton methods for application to FSI problems involving free surface flow and solid body contact. Computers & Structures (2016) 173: 71-83.
    [14] CUTE: Constrained and unconstrained testing environments. ACM Trans. Math. Softw. (1995) 21: 123-160.
    [15] A classification of quasi-Newton methods. Numer. Algorithms (2003) 33: 123-135.
    [16] A conjugate gradient algorithm and its applications in image restoration. Appl. Numer. Math. (2020) 152: 243-252.
    [17] A two-term PRP-based descent method. Numer. Funct. Anal. Optim. (2007) 28: 1217-1230.
    [18] A sufficient descent conjugate gradient method and its global convergence. Optim. Methods Softw. (2016) 31: 577-590.
    [19] Stepsize analysis for descent methods. J. Optim. Theory Appl. (1981) 33: 187-205.
    [20] Convergence of quasi-Newton matrices generated by the symmetric rank one update. Math. Programming (1991) 50: 177-195.
    [21] Y.-H. Dai, Nonlinear Conjugate Gradient Methods, Wiley Encyclopedia of Operations Research and Management Science, (2011). doi: 10.1002/9780470400531.eorms0183
    [22] Y.-H. Dai, Alternate Step Gradient Method, Report AMSS–2001–041, Academy of Mathematics and Systems Sciences, Chinese Academy of Sciences, 2001.
    [23] A nonmonotone conjugate gradient algorithm for unconstrained optimization. J. Syst. Sci. Complex. (2002) 15: 139-145.
    [24] Y.-H. Dai and R. Fletcher, On the Asymptotic Behaviour of some New Gradient Methods, Numerical Analysis Report, NA/212, Dept. of Math. University of Dundee, Scotland, UK, 2003.
    [25] A nonlinear conjugate gradient algorithm with an optimal property and an improved wolfe line search. SIAM. J. Optim. (2013) 23: 296-320.
    [26] New conjugacy conditions and related nonlinear conjugate gradient methods. Appl. Math. Optim. (2001) 43: 87-101.
    [27] R-linear convergence of the Barzilai and Borwein gradient method. IMA J. Numer. Anal. (2002) 22: 1-10.
    [28] Testing different conjugate gradient methods for large-scale unconstrained optimization. J. Comput. Math. (2003) 21: 311-320.
    [29] Another improved Wei–Yao–Liu nonlinear conjugate gradient method with sufficient descent property. Appl. Math. Comput. (2012) 218: 7421-7430.
    [30] A nonlinear conjugate gradient method with a strong global convergence property. SIAM J. Optim. (1999) 10: 177-182.
    [31] An efficient hybrid conjugate gradient method for unconstrained optimization. Ann. Oper. Res. (2001) 103: 33-47.
    [32] Y. H. Dai and Y. Yuan, A class of Globally Convergent Conjugate Gradient Methods, Research report ICM-98-030, Institute of Computational Mathematics and Scientific/Engineering Computing, Chinese Academy of Sciences, 1998.
    [33] A class of globally convergent conjugate gradient methods. Sci. China Ser. A (2003) 46: 251-261.
    [34] Modified two-point step-size gradient methods for unconstrained optimization. Comput. Optim. Appl. (2002) 22: 103-109.
    [35] Alternate minimization gradient method. IMA J. Numer. Anal. (2003) 23: 377-393.
    [36] Analysis of monotone gradient methods. J. Ind. Manag. Optim. (2005) 1: 181-192.
    [37] Y.-H. Dai and H. Zhang, An Adaptive Two-Point Step-size gradient method, Research report, Institute of Computational Mathematics and Scientific/Engineering Computing, Chinese Academy of Sciences, 2001.
    [38] The conjugate gradient method for linear and nonlinear operator equations. SIAM J. Numer. Anal. (1967) 4: 10-26.
    [39] S. Delladji, M. Belloufi and B. Sellami, Behavior of the combination of PRP and HZ methods for unconstrained optimization, Numerical Algebra, Control and Optimization, (2020). doi: 10.3934/naco.2020032
    [40] Investigation of quasi-Newton methods for unconstrained optimization. International Journal of Computer Application (2010) 29: 48-58.
    [41] Two modifications of the method of the multiplicative parameters in descent gradient methods. Appl. Math, Comput. (2012) 218: 8672-8683.
    [42] S. S. Djordjević, Unconstrained Optimization Methods: Conjugate Gradient Methods and Trust-Region Methods, Book Chapter, 2019. doi: 10.5772/intechopen.84374
    [43] Benchmarking optimization software with performance profiles. Math. Program. (2002) 91: 201-213.
    [44] The application of quasi-Nnewton methods in fluid mechanics. Internat. J. Numer. Methods Engrg. (1981) 17: 707-718.
    [45] D. K. Faddeev and I. S. Sominskiǐ, Collection of Problems on Higher Algebra. Gostekhizdat, 2nd edition, Moscow, 1949.
    [46] A. G. Farizawani, M. Puteh, Y. Marina and A. Rivaie, A review of artificial neural network learning rule based on multiple variant of conjugate gradient approaches, Journal of Physics: Conference Series, 1529 (2020). doi: 10.1088/1742-6596/1529/2/022040
    [47] R. Fletcher, Practical Methods of Optimization, Unconstrained Optimization, 1st edition, Wiley, New York, 1987.
    [48] Function minimization by conjugate gradients. Comput. J. (1964) 7: 149-154.
    [49] Global convergence properties of conjugate gradient methods for optimization. SIAM J. Optim. (1992) 2: 21-42.
    [50] On steepest descent. J. SIAM Control Ser. A (1965) 3: 147-151.
    [51] A nonmonotone line search technique for Newton's method. SIAM J. Numer. ANAL. (1986) 23: 707-716.
    [52] A class of nonmonotone stability methods in unconstrained optimization. Numer. Math. (1991) 59: 779-805.
    [53] A truncated Newton method with nonmonotone line search for unconstrained optimization. J. Optim. Theory Appl. (1989) 60: 401-419.
    [54] Nonmonotone globalization techniques for the Barzilai-Borwein gradient method. Comput. Optim. Appl. (2002) 23: 143-169.
    [55] A new conjugate gradient method with guaranteed descent and an efficient line search. SIAM J. Optim. (2005) 16: 170-192.
    [56] Algorithm 851: CG_DESCENT, a conjugate gradient method with guaranteed descent. ACM Trans. Math. Software (2006) 32: 113-137.
    [57] A survey of nonlinear conjugate gradient methods. Pac. J. Optim. (2006) 2: 35-58.
    [58] Combining quasi-Newton and Cauchy directions. Int. J. Appl. Math. (2003) 12: 167-191.
    [59] A new type of quasi-Newton updating formulas based on the new quasi-Newton equation. Numer. Algebra Control Optim. (2020) 10: 227-235.
    [60] Methods of conjugate gradients for solving linear systems. J. Res. Nat. Bur. Standards (1952) 49: 409-436.
    [61] Global convergence result for conjugate gradient methods. J. Optim. Theory Appl. (1991) 71: 399-405.
    [62] Fixed points by a new iteration method. Proc. Am. Math. Soc. (1974) 44: 147-150.
    [63] B. Ivanov, P. S. Stanimirović, G. V. Milovanović, S. Djordjević and I. Brajević, Accelerated multiple step-size methods for solving unconstrained optimization problems, Optimization Methods and Software, (2019). doi: 10.1080/10556788.2019.1653868
    [64] Applications of the conjugate gradient method in optimal surface parameterizations. Int. J. Comput. Math. (2010) 87: 1032-1039.
    [65] A hybrid conjugate gradient method with descent property for unconstrained optimization. Appl. Math. Model. (2015) 39: 1281-1290.
    [66] S. H. Khan, A Picard-Mann hybrid iterative process, Fixed Point Theory Appl., 2013 (2013), Article number: 69, 10 pp. doi: 10.1186/1687-1812-2013-69
    [67] Novel hybrid algorithm in solving unconstrained optimizations problems. International Journal of Novel Research in Physics Chemistry & Mathematics (2017) 4: 36-42.
    [68] Implementation of gradient methods for optimization of underage costs in aviation industry. University Thought, Publication in Natural Sciences (2016) 6: 71-74.
    [69] A continuous-time approach to online optimization. J. Dyn. Games (2017) 4: 125-148.
    [70] On a two phase approximate greatest descent method for nonlinear optimization with equality constraints. Numer. Algebra Control Optim. (2018) 8: 315-326.
    [71] A modified BFGS method and its global convergence in nonconvex minimization. J. Comput. Appl. Math. (2001) 129: 15-35.
    [72] A modified PRP conjugate gradient algorithm with trust region for optimization problems. Numer. Funct. Anal. Optim. (2011) 32: 496-506.
    [73] A projected preconditioned conjugate gradient method for the linear response eigenvalue problem. Numer. Algebra Control Optim. (2018) 8: 389-412.
    [74] A modified Fletcher-Reeves-type derivative-free method for symmetric nonlinear equations. Numer. Algebra Control Optim. (2011) 1: 71-82.
    [75] Approximate greatest descent in neural network optimization. Numer. Algebra Control Optim. (2018) 8: 327-336.
    [76] J. Liu, S. Du and Y. Chen, A sufficient descent nonlinear conjugate gradient method for solving M-tensor equations, J. Comput. Appl. Math., 371 (2020), 112709, 11 pp. doi: 10.1016/j.cam.2019.112709
    [77] A projection method for convex constrained monotone nonlinear equations with applications. Comput. Math. Appl. (2015) 70: 2442-2453.
    [78] Efficient generalized conjugate gradient algorithms, part 1: Theory. J. Optim. Theory Appl. (1991) 69: 129-137.
    [79] A risk minimization problem for finite horizon semi-Markov decision processes with loss rates. J. Dyn. Games (2018) 5: 143-163.
    [80] I. E. Livieris and P. Pintelas, A descent Dai-Liao conjugate gradient method based on a modified secant equation and its global convergence, ISRN Computational Mathematics, 2012 (2012), Article ID 435495. doi: 10.5402/2012/435495
    [81] M. Lotfi and S. M. Hosseini, An efficient Dai-Liao type conjugate gradient method by reformulating the CG parameter in the search direction equation, J. Comput. Appl. Math., 371 (2020), 112708, 15 pp. doi: 10.1016/j.cam.2019.112708
    [82] Hybrid approach for solving systems of nonlinear equations using chaos optimization and quasi-Newton method. Applied Soft Computing (2008) 8: 1068-1073.
    [83] Mean value methods in iterations. Proc. Amer. Math. Soc. (1953) 4: 506-510.
    [84] Scalar correction method for solving large scale unconstrained minimization problems. J. Optim. Theory Appl. (2011) 151: 304-320.
    [85] S. K. Mishra and B. Ram, Introduction to Unconstrained Optimization with R, 1st edition, Springer Singapore, Springer Nature Singapore Pte Ltd., 2019. doi: 10.1007/978-981-15-0894-3
    [86] I. S. Mohammed, M. Mamat, I. Abdulkarim and F. S. Bt. Mohamad, A survey on recent modifications of conjugate gradient methods, Proceedings of the UniSZA Research Conference 2015 (URC 5), Universiti Sultan Zainal Abidin, 14–16 April 2015.
    [87] A brief survey of methods for solving nonlinear least-squares problems. Numer. Algebra Control Optim. (2019) 9: 1-13.
    [88] A survey of sufficient descent conjugate gradient methods for unconstrained optimization. SUT J. Math. (2014) 50: 167-203.
    [89] J. L. Nazareth, Conjugate-Gradient Methods, In: C. Floudas and P. Pardalos (eds), Encyclopedia of Optimization, 2nd edition, Springer, Boston, 2009. doi: 10.1007/978-0-387-74759-0
    [90] J. Nocedal and S. J. Wright, Numerical Optimization, Springer, New York, 1999. doi: 10.1007/b98874
    [91] W. F. H. W. Osman, M. A. H. Ibrahim and M. Mamat, Hybrid DFP-CG method for solving unconstrained optimization problems, Journal of Physics: Conf. Series, 890 (2017), 012033. doi: 10.1088/1742-6596/890/1/012033
    [92] Initial improvement of the hybrid accelerated gradient descent process. Bull. Aust. Math. Soc. (2018) 98: 331-338.
    [93] A modified conjugate gradient algorithm. Oper. Res. (1978) 26: 1073-1078.
    [94] An accelerated double step-size method in unconstrained optimization. Applied Math. Comput. (2015) 250: 309-319.
    [95] Determination of accelerated factors in gradient descent iterations based on Taylor's series. University Thought, Publication in Natural Sciences (2017) 7: 41-45.
    [96] Hybridization of accelerated gradient descent method. Numer. Algorithms (2018) 79: 769-786.
    [97] A note on hybridization process applied on transformed double step size model. Numerical Algorithms (2020) 85: 449-465.
    [98] M. J. Petrović and P. S. Stanimirović, Accelerated double direction method for solving unconstrained optimization problems, Math. Probl. Eng., 2014 (2014), Article ID 965104, 8 pages. doi: 10.1155/2014/965104
    [99] M. J. Petrović, P. S. Stanimirović, N. Kontrec and J. Mladenović, Hybrid modification of accelerated double direction method, Math. Probl. Eng., 2018 (2018), Article ID 1523267, 8 pages. doi: 10.1155/2018/1523267
    [100] A new class of efficient and globally convergent conjugate gradient methods in the Dai-Liao family. Optim. Methods Softw. (2015) 30: 843-863.
    [101] Memoire sur la theorie des equations aux derivees partielles et la methode des approximations successives. J. Math. Pures Appl. (1890) 6: 145-210.
    [102] E. Polak and G. Ribière, Note sur la convergence des méthodes de directions conjuguées, Rev. Française Imformat Recherche Opértionelle, 3 (1969), 35–43.
    [103] The conjugate gradient method in extreme problems. USSR Comput. Math. and Math. Phys. (1969) 9: 94-112.
    [104] Algorithms for nonlinear constraints that use Lagrangian functions. Math. Programming (1978) 14: 224-248.
    [105] On the Barzilai and Borwein choice of steplength for the gradient method. IMA J. Numer. Anal. (1993) 13: 321-326.
    [106] The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem. SIAM J. Optim. (1997) 7: 26-33.
    [107] Relaxed steepest descent and Cauchy-Barzilai-Borwein method. Comput. Optim. Appl. (2002) 21: 155-167.
    [108] A new family of conjugate gradient coefficient with application. International Journal of Engineering & Technology (2018) 7: 36-43.
    [109] Convergence of line search methods for unconstrained optimization. Appl. Math. Comput. (2004) 157: 393-405.
    [110] The application of new conjugate gradient methods in estimating data. International Journal of Engineering & Technology (2018) 7: 25-27.
    [111] Multi-step spectral gradient methods with modified weak secant relation for large scale unconstrained optimization. Numer. Algebra Control Optim. (2018) 8: 377-387.
    [112] New hybrid conjugate gradient and Broyden-Fletcher-Goldfarb-Shanno conjugate gradient methods. J. Optim. Theory Appl. (2018) 178: 860-884.
    [113] Computation of {2,4} and {2,3}-inverses based on rank-one updates. Linear Multilinear Algebra (2018) 66: 147-166.
    [114] Computing {2,4} and {2,3}-inverses by using the Sherman-Morrison formula. Appl. Math. Comput. (2015) 273: 584-603.
    [115] Accelerated gradient descent methods with line search. Numer. Algor. (2010) 54: 503-520.
    [116] P. S. Stanimirović, G. V. Milovanović, M. J. Petrović and N. Z. Kontrec, A transformation of accelerated double step-size method for unconstrained optimization, Math. Probl. Eng., 2015 (2015), Article ID 283679, 8 pages. doi: 10.1155/2015/283679
    [117] Global convergence of nonmonotone descent methods for unconstrained optimization problems. J. Comp. Appl. Math. (2002) 146: 89-98.
    [118] Identification of Hessian matrix in distributed gradient based multi agent coordination control systems. Numer. Algebra Control Optim. (2019) 9: 297-318.
    [119] W. Sun and Y.-X. Yuan, Optimization Theory and Methods: Nonlinear Programming, 1st edition, Springer, New York, 2006. doi: 10.1007/b106451
    [120] Non–monotone trust–region algorithm for nonlinear optimization subject to convex constraints. Math. Prog. (1997) 77: 69-94.
    [121] Efficient hybrid conjugate gradient techniques. J. Optim. Theory Appl. (1990) 64: 379-397.
    [122] A class of gradient unconstrained minimization algorithms with adaptive step-size. J. Comp. Appl. Math. (2000) 114: 367-386.
    [123] New quasi-Newton methods for unconstrained optimization problems. Appl. Math. Comput. (2006) 175: 1156-1188.
    [124] New nonlinear conjugate gradient formulas for large-scale unconstrained optimization problems. Appl. Math. Comput. (2006) 179: 407-430.
    [125] The convergence properties of some new conjugate gradient methods. Appl. Math. Comput. (2006) 183: 1341-1350.
    [126] Convergence conditions for ascent methods. SIAM Rev. (1969) 11: 226-235.
    [127] Survey of derivative-free optimization. Numer. Algebra Control Optim. (2020) 10: 537-555.
    [128] A conjugate gradient method to solve convex constrained monotone equations with applications in compressive sensing. J. Math. Anal. Appl. (2013) 405: 310-319.
    [129] Global convergence properties of nonlinear conjugate gradient methods with modified secant condition. Comput. Optim. Appl. (2004) 28: 203-225.
    [130] X. Yang, Z. Luo and X. Dai, A global convergence of LS-CD hybrid conjugate gradient method, Adv. Numer. Anal., 2013 (2013), Article ID 517452, 5 pp. doi: 10.1155/2013/517452
    [131] S. Yao, X. Lu and Z. Wei, A conjugate gradient method with global convergence for large-scale unconstrained optimization problems, J. Appl. Math., 2013 (2013), Article ID 730454, 9 pp. doi: 10.1155/2013/730454
    [132] Convergence analysis of a modified BFGS method on convex minimizations. Comp. Optim. Appl. (2010) 47: 237-255.
    [133] A notes about WYL's conjugate gradient method and its applications. Appl. Math. Comput. (2007) 191: 381-388.
    [134] A new stepsize for the steepest descent method. J. Comput. Math. (2006) 24: 149-156.
    [135] A conjugate gradient algorithm for large-scale nonlinear equations and image restoration problems. Appl. Numer. Math. (2020) 147: 129-141.
    [136] G. Yuan, T. Li and W. Hu, A conjugate gradient algorithm and its application in large-scale optimization problems and image restoration, J. Inequal. Appl., 2019 (2019), Article number: 247, 25 pp. doi: 10.1186/s13660-019-2192-6
    [137] Modified limited memory BFGS method with nonmonotone line search for unconstrained optimization. J. Korean Math. Soc. (2010) 47: 767-788.
    [138] An improved Wei-Yao-Liu nonlinear conjugate gradient method for optimization computation. Appl. Math. Comput. (2009) 215: 2269-2274.
    [139] Two new Dai-Liao-type conjugate gradient methods for unconstrained optimization problems. J. Optim. Theory Appl. (2017) 175: 502-509.
    [140] Semi-local convergence of the Newton-HSS method under the center Lipschitz condition. Numer. Algebra Control Optim. (2019) 9: 85-99.
    [141] A nonlinear conjugate gradient method based on the MBFGS secant condition. Optim. Methods Softw. (2006) 21: 707-714.
    [142] G. Zoutendijk, Nonlinear programming, computational methods, In: J. Abadie (eds.): Integer and Nonlinear Programming, North-Holland, Amsterdam, (1970), 37–86.
    [143] A new gradient method for solving linear regression model. International Journal of Recent Technology and Engineering (2019) 7: 624-630.
  • This article has been cited by:

    1. Shaima Ibraheem Jabbar, Abathar Qahtan Omran Aladi, 2024, Fuzzy inference system for detection CT and X-Ray image’s edges, 979-8-3503-8753-7, 1, 10.1109/AICT61888.2024.10740426
  • Reader Comments
  • © 2020 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)
通讯作者: 陈斌, bchen63@163.com
  • 1. 

    沈阳化工大学材料科学与工程学院 沈阳 110142

  1. 本站搜索
  2. 百度学术搜索
  3. 万方数据库搜索
  4. CNKI搜索

Metrics

Article views(7412) PDF downloads(826) Cited by(22)

Figures and Tables

Figures(18)  /  Tables(13)

/

DownLoad:  Full-Size Img  PowerPoint
Return
Return

Catalog