Real-time artificial intelligence based health monitoring, diagnosing and environmental control system for COVID-19 patients

1.   Introduction

Chronic disease (for example, cardiovascular disease, diabetes, Alzheimer's) has increased, with mortality associated with a rise in coronial health disease to 66% by 2030. Telehealth, which includes virtual healthcare, makes use of technology to improve communication between doctors, clinics, and patients. Clinics, doctors, and patients can share information, monitor, and follow up on care plans using electronic communication technology, which allows for maximum virtual participation throughout medical treatment. We now live in a globe where technology can execute procedures, perform pre-operative planning, and monitor results from afar ^[1], in accordance with the World Health Organization (WHO). It is necessary to monitor patients continuously with chronic illnesses to avert life-threatening scenarios.

This decade, the Internet of Things has been hailed as the beginning of a new era of interconnectedness for everyday devices everywhere. Remote health monitoring mechanisms ^[2,3], parking management ^[4], smart houses ^[5], smart cities ^[6], smart environment ^[7], industrial sites ^[8], and agricultural lands ^[9]. Health and environmental conditions can be tracked using IoT in healthcare management. The importance of IoT systems has been increased in real-time applications because of their simple structure. IoT connects computers to the internet through sensors and networks in order to process data in real-time ^[10,11]. Figure 1 shows the IoT applications in real-time systems.

Figure 1.  IoT applications in daily life.

DownLoad: Full-Size Img PowerPoint

In this article, a system for remote health surveillance is developed by utilizing locally available sensors. This system will provide the live location, GSM messages, and an email to the doctors during emergency conditions. The remainder of the paper is laid out as follows: Section 2 contains the literature survey of previously existing techniques and problem statements. Section 3 contains the materials and methodology of the proposed system. Section 4 gives the general description of intelligent neural networks and defines the training of the proposed intelligent system. Section 5 contains the real-time implementation results of the proposed system and provides a discussion of obtained results. Section 5 also gives a comparison among proposed and different existing IoT-based health monitoring systems. The conclusions and future works are described in the last.

2.   Literature review

A health monitoring gadget might make use of interconnected pieces of equipment. It is then sent to M2M, which are machines for people, machines for computers, portable devices, or cellphones ^[12]. Health care tracking and optimization may be done in a far more efficient, intelligently scalable, and interoperable manner with this method. Modern systems now include a flexible user interface ^[13], helper devices ^[14], and mental health management ^[15] to help people live smarter lives. The IoT is essential for monitoring health because it allows us to link equipment for medication to gather health data from patients and work on it to avoid catastrophic occurrences ^[16]. It is also possible to connect medical sensors and process them. In every expansion sign of progress of humanity in technology, the critical issue in life is achieving good health. A sensing device may be considered "smart" if it is combined with a network-capable application processor (NCAP) that provides local control, and feedback and connects each sensing device to the high-level network. ^[17]. The situation in the hospitals is getting worst by the viral infections of COVID-19, which create space shortage for new patients. Numerous hospitals have already initiated to invest in technology acceptance. For example, to reduce administrative burdens while improving healthcare outcomes. Saratoga Hospital purchased IoT-enabled patient monitoring devices ^[18].

In healthcare hardware and software, the medical IoT is keenly integrated to back clinicians and patients. Continuous patient surveillance provisions are in place. When they come home, no arrangement is made to verify the parameters. And thus, there's a risk that the illness might come back. Therefore, the Internet of Things health surveillance system is the answer. Data from a patient is often measured and uploaded to the server (temperature, heart rate, SpO2, ECG). The physician might even have not to live with the patient to check the patient's status. The doctor can look at the health state of his patient using his android mobile application or the smartphone of the patient. If any parameter exceeds the threshold value, the doctor receives a message of an emergency alert. The optimal control of humidity and temperature of the room also keeps a safe environment in the patient room ^[19].

According to the WHO, COVID-19 has harmed more than 10 million individuals in the world. And there is also a high overall death rate. During COVID-19, many doctors who met patients to secure COVID-19 patients were also affected ^[20]. In various monitoring applications, wireless sensor networks (WSNs) have been widely employed because of the advantages of low cost and easy deployment, including battlefield surveillance, environmental monitoring, and bio identification ^[21,22], etc.

The key function of IoT in an emergency is to monitor and notify the patient. Wearable gadgets, medical tracking systems, supply chains, remote patient monitoring, and more are examples of IoT in the medical and healthcare sectors. IoT can help improve diagnostic accuracy by continuously monitoring health changes. Users can stay in touch with doctors or nurses via IoT healthcare apps. They assist healthcare workforces to work more efficiently with the patients in working place ^[23]. An IoT platform will help the doctor and the patient. The doctor examines the health indicators of patients through the IoT platform (Blynk). It is necessary to constantly monitor patients with chronic illnesses in order to avert life-threatening scenarios. The IoT is essential for monitoring health because it allows linking health parameter measuring instruments to gather patient health data and proceed with it to avoid catastrophic occurrences. It is also possible to connect medical sensors and process them. The design of the wireless sensor network consists of nodes that are used to monitor environments such as temperature. This may be assessed through the physicalizing of the Internet. This technology is exciting for an extensive range of applications such as medical, environmental, transit, military, entertainment, home defense, crisis management, and smart spaces ^[24,25].

Normally, many researchers have considered the machine learning techniques as efficient diagnostic tool health monitoring of patients ^[26,27,28]. For the best solution of real-time problems, the most suitable AI techniques is the neural network ^[29].

3.   Materials and methodology

In this section, we provided the materials and methodology of the proposed system. The complete description of the system using a flowchart is also presented.

3.1.   General context

There are three primary problems with existing IoT-based health surveillance systems. First, the communication lines such as 3G/4G are used commonly under relatively high-cost conditions. Secondly, in general, data privacy problems are not addressed. And at last, most of them are not analyzed or monitored. Our main work is to make a low-cost and more robust system via ESP32 board. One of the key elements of this project is Blynk, a development platform for e-health monitoring that works along with Arduino Uno. Up to 15 sensors can be connected and measured using Blynk. The Blynk platform is utilized for biomedical sensors such as ECG sensor, body temperature sensor and oxygen saturation and pulse rate sensor. LoRa communication technology, a novel, private and broadcast modulation approach, enables the transmission of data at extremely low data rates to very long ranges are utilized for the wireless transfer of measured health data to the cloud.

When patients have to take a medicine at a certain point during the day, the patient will be notified of the medication and the condition. Blynk IoT platform is the best solution for monitoring parameters from remote location. A complete flow chart of the proposed real-time health monitoring system is shown in Figure 2.

Figure 2.  Flow chart for real time monitoring.

DownLoad: Full-Size Img PowerPoint

3.2.   CAD model design and hardware implementation

The CAD modal of an auto-injector device that injects the dose automatically is developed and implemented as shown in Figures 3–5. There are three slots in this device, and two prefilled syringes are placed at the 1st and 3rd slot, and the 2nd or central space is empty. The first syringe belongs to emergency treatment for the heart disease patients, and the other needle is for the treatment of patients with breathing disorder. When the parameter is gone critical, it gets a notification to take-action. When the heart disease is detected, the first servo SG90 starts to take the heart's string from 1st slot and take it to the space. When the syringe is placed, the stepper motors begin to rotate the shaft, and then the slider moves on the post to push the barrel, and then the needle injects into the patient's muscle. A spring mechanism is present there to take out the injection. Below Figures represent the CAD design and hardware implementation.

Figure 3.  1^st CAD model.

DownLoad: Full-Size Img PowerPoint

Figure 4.  2^nd CAD model.

DownLoad: Full-Size Img PowerPoint

Figure 5.  Hardware implementation of slected CAD model design.

DownLoad: Full-Size Img PowerPoint

3.3.   General description of proposed methodology

An innovative health management system has been created that is intelligent enough to monitor IoT and Cloud patients' vital health metrics automatically and link their doctor through live contact with the patient. This collects the patient's heart rate, heart rate, level SpO2, ECG, and temperature using sensors connected to the patient's body and communicates the current state of the emergency condition if the threshold of the parameter is reported and comprehensive medical data to the patient's doctor. The doctor will continue to follow the patient and urge the patient to observe and interact directly with their state of health without going to the hospital. All the sensors are connected to the patient's body to measure the real-time data and then the observed values are sent to the microcontroller. The working principle of NodeMCU microcontroller, in the proposed methodology, is shown in Figure 6. The NodeMCU microcontroller is attached to the internet module ESP32. An IoT platform (Blynk) collects all the data from this controller and displays it to the doctor.

Figure 6.  Working principle of IoT-based NodeMCU in the proposed methodology.

DownLoad: Full-Size Img PowerPoint

If any of the parameter values changed from the threshold, the doctor would receive an emergency email. A doctor can communicate with their patient via live calling. Figure 7 represents the step-by-step working principle of the proposed methodology.

Figure 7.  Steps involved in working of proposed methodology.

DownLoad: Full-Size Img PowerPoint

4.   The intelligent neural networks

In this section, we provided the intelligent neural networks, their selection accuracy improvement trainings and testing.

4.1.   General context

Many classifiers and machine learning techniques were created to categorize data sets for classification issues. Linear classificatory are employed when input data is linearly separable. Relatively raw electrical networks of neurons based on the neural construction of the brain are artificial neural networks (ANC). The process records one by one and learns by comparing their record categorization with the actual record classification known. The mistakes from the original categorization of the first record are reproduced in the network and are utilized for subsequent iterations to change the network algorithm. A neuron is a collection of input values (xi) and associated weights in an artificial neural network (wi). Function (g), which accumulates the consequences and maps the output results (y).

4.2.   Selection of neural network

The neural network of three fundamental kinds of artificial neural network (ANN), deep neural network (DNN), and convolutional (CNN). ANN for pattern recognition issues, CNN for HD, and DNN for multiple hidden layers are frequently applied. DNN is used for several hidden layers in this paper. At its most straightforward, the deep network (DNN) or the deep network for the shortest is the neural network with a certain complexity, generally two layers. Deep networks use advanced mathematical models to handle data in complicated ways.

Figure 8.  The evolution to deep neural networks (DNN).

DownLoad: Full-Size Img PowerPoint

First, it was necessary to create machine learning. ML is a framework for automating statistical models (via algorithms), such as a linear regression model, to improve predictions. A single model is a model that predicts something. These forecasts are accurately produced. The learning model – machine learning takes all its poor predictions and changes the weight of the model in order to build a more miniature error model. The learning part of the model building has created artificial neural networks. The essence of neural network training is back-propagation. It is the way weight in a neural network may be adjusted based on the error rate achieved during the previous epoch. Proper adjustment of weights helps you to minimize error rates and make the model more generalized and dependable. Neural network background propagation is a brief "backward error propagation" form. It is an artificial neural networking standard technique. In this technique, the loss function gradient for all weights in the network may be calculated.

Figure 9.  Visual representation of ANN with backpropagation.

DownLoad: Full-Size Img PowerPoint

ANNs use the hidden layer as a storage location and assess the importance of one of the inputs to the output. The hidden layer contains information about the relevance of input and links the importance of input combinations.

4.2.1.   Improving accuracy

Deep networks allow the performance of a model to improve precision. You can take a collection of inputs and produce output for a model. The usage of a deep network is so simple as to copy and paste a code line for each layer. The ML platform you use does not matter; it is as easy to type in the letters 2 or 2000 to guide the model to two or 2000 nodes in each layer.

4.2.2.   Training and testing

Dataset from the EMG body data, utilized as input for classification using ANN feedforward with backpropagation technology. 80:20 ratios are used to train and evaluate the neural network as shown in Figure 10. For neural network training, 80% of data set values are utilized. It's more accurate to train a neural network several times ^[30]. An epoch is called the number of times the entire training data set is passed on. The optimum scenario is that the training and validation error continues to decrease in a number of periods. When the model ends at 1 stage, the model parameters are modified according to the results.

Figure 10.  Training progress.

DownLoad: Full-Size Img PowerPoint

A part of date set is considered as validation data set. For the validation data set, the fitted model is used to predict reactions to these observations. For the testing of the neural network, the remaining 20% are employed. Usually, it is a holdout dataset that has not been utilized for training or cross-validation. The test analyses the final model fit the data set of training while adjusting the hyperparameters of the model. Classification results are acquired through training and testing.

5.   Results and discussion

In this section, we report the results of the real-time implementation of the proposed system, discuss the obtained results, and compare the proposed and different existing methods.

5.1.   Implementation of IOT

The Blynk application is designed to monitor the heart rate, SpO2 level, and body temperature and control the room's environment by maintaining the temperature and humidity. The Blynk application is also capable of showing the live location of the patient in case of emergency. This location will be shared with doctors and family members. The ESP32 sends an email to the doctor The ESP32 is also able to call and send messages using the SIM900A module. The design gets the data from the ESP32 and sends it to the laptop which is viewed on a serial monitor.

Figure 11.  Hardware implementation.

DownLoad: Full-Size Img PowerPoint

IOT enables the data transfer to Blynk application using ESP8266. The heart rate, SpO2 level and temperature data sends to first screen of Blynk interface. The room's temperature and humidity transferred to the second page of Blynk interface. The application is also able to set the predefined levels for heart rate, SpO2 level, and body temperature, once the parameters meet the current level the doctor receives an email alert about the critical parameter of patient health. The live location of the patient is also available on the interface.

For the second interface, one can set the humidity and temperature levels to the desired value for the patient's room. That helps to maintain the environment around the patient. Once we set a level using the slider, Blynk sends the data to the Blynk cloud using API and ESP8266 gets that data from Blynk cloud and sets in the hardware to compare it with current values around the patient. Figure 12 shows the interface first and second for vital features and room environmental control.

Figure 12.  Blynk IoT system.

DownLoad: Full-Size Img PowerPoint

5.2.   Circuit design

Fans, relays, and humidifiers are the major components in this circuit schematic which are shown in Figure 13. With the aid of two relays stated above, the fan and the humidifier are regulated. For humidity and temperature in °F, the real data or input is obtained from the DHT11 sensor in mmHg. These criteria are specified using sliders, Blynk's GUI feature. Both parameters' thresholds have previously been specified and sliders match and react to these values. Examples,

Figure 13.  The schematic diagram of fans, relays and humidifiers circuit.

DownLoad: Full-Size Img PowerPoint

The fan had to hold a temperature of 20°F and the sensor transfers that information's to the fan quicker in order to retain the necessary temperature when it surpasses the set threshold.

5.3.   Implementation of neural network

A technique of identifying QRS complex (R-Peaks) ECG signal is presented using an undecimated discrete wavelet transform (DWT) ^[31]. It is shown how DWT is used to filter an ECG signal through the bandpass to maintain the R-peaks, which is best for selected BPF frequencies. The heart ratio is also calculated (in beats per minute) using the calculation of total R peaks. The proposed robustness of the Scheme is further proven using different types of ECG signals from MIT-BIH and ECG-ID databases.

The QRS complex consists of the conjunction of three deflections (Q, R and S) (ECG). It referred to the depolarization of the ventricles right and left of the heart and the contraction of the big ventricular muscles. Average QRS amplitude is 5 to 30 mm and duration between 0.06 and 0.12 seconds, respectively ^[32]. QRS complex width, amplitude, or form is essential for the diagnosis of coronary rhythm, aberrant behavior, VHH, myocardial infarction, electrolyte disorder, and other illness conditions. An ECG signal from the MIT-BIH database is shown below in Figure 14.

Figure 14.  ECG signal output.

DownLoad: Full-Size Img PowerPoint

The form of an aberrant QRS complex varies from near-normal to strange and/or slurred and nodules. The enlargement of one or both ventricles, or the faulty pacemaker or aberrantly performed beat, is often responsible for large QRS complexes. In obese, hyperthyroid and pleural effusion individuals, low voltage or unusually tiny QRS complexes might be detected.

Now, we describe the symlet4 wavelet for an ECG signal. Note that the rationale for choosing symlet4 is that the "sym4" wavelet is like the QRS complex and is a suitable option for QRS detection. Symlet4 of an ECG signal is shown in Figure 15.

Figure 15.  Symlet4 wavelet for an ECG signal.

DownLoad: Full-Size Img PowerPoint

As a technique of identifying QRS complex (R-Peaks) ECG signal is presented using a decimated discrete wavelet transform (DWT). It is shown how DWT is used to filter an ECG signal through the bandpass to maintain the R-peaks, which is best for selected BPF frequencies. The heart ratio is also calculated (in beats per minute) using the calculation of total R peaks. The proposed robustness of the Scheme is further proven using different types of ECG signals from MIT-BIH and ECG-ID databases.

Figure 16 demonstrates the loss of training and validation and precision. Our objective was to obtain as low a validation loss as feasible. The loss of assurance in Figure 16 is higher than the loss of training, which is also shown in the confusion matrix of Figure 17. It is referred to as oversize. If the data set comprises any scenario, the overfit may be advantageous for the neural network's performance.

Figure 16.  Loss and accuracy of training and test datasets.

DownLoad: Full-Size Img PowerPoint

Figure 17.  Confusion matrix.

DownLoad: Full-Size Img PowerPoint

5.4.   Comparison

A brief comparison between different techniques in terms of sensors and features used for different health monitoring research is given in above Figure 18. Below there is Table 1 has a brief comparison between eight different research.

Figure 18.  Comparison of proposed research with existing research having no of features.

DownLoad: Full-Size Img PowerPoint

Table 1.  Comparison of proposed and the existing techniques.

Ref.	NET	System	Heart rate + SpO2	App	ECG	GSM	LCD	Email alert	Room control	Injector	GPS
Proposed	WIFI	AI	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$
[33]	IP Based		$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$
[34]	WIFI	Fuzzy	$\checkmark$	$\checkmark$			$\checkmark$	$\checkmark$
[35]	LoraWAN		$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$
[36]	BT + Wifi		$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$
[37]	BT + Wifi		$\checkmark$	$\checkmark$			$\checkmark$
[38]	Wifi		$\checkmark$	$\checkmark$			$\checkmark$
[39]	LoRa	AI	$\checkmark$		$\checkmark$		$\checkmark$
[40]	Wifi	AI	$\checkmark$		$\checkmark$

---

 | Show Table

DownLoad: CSV

6.   Summary

6.1.   Conclusions

By applying the deep neural network, the training data is processed in MATLAB, and the output class is generated that shows the probabilities of the output class with respect to the target class. The confusion matrix is made up of marginal and joint probabilities. This matrix shows the loss and validation of the training data. The overall result from DNN is generated, and our system becomes intelligent by training the system. Our designed IOT-based system can automatically send data to the doctor via the web page. Also, an AI-based medicine prescription system can give a dose to the patient in the absence of a doctor during a severe emergency. This system can also be used as a product for patients living in rural areas as helping them to avoid hospital expenses; they can use the device and check their vital signs and get the correct medicine for them via doctor or the AI-based system ^[41].

6.2.   Limitations

Our AI-Based Patient Monitoring System has been designed to help doctors observe the vital signs of patients remotely via a webpage and to help people who are in need of a hospital and living in rural areas but cannot reach the hospital and also in case of the absence of a doctor, to prescribe medicines to the patients accordingly. The fully developed proposed system has many applications. There is no need for the doctor to visit the patient, again and again, the doctor can monitor the patient's vital signs via a webpage from anywhere in the world. This system can be used in hospitals so that doctors can simply observe the patient while sitting in their office. This system can also be used as a product for patients living in rural areas as helping them to avoid hospital expenses, they can simply use the device and check their vital signs and can get the right medicine for them via doctor or the AI-based system ^[42].

6.3.   Future work and suggestions

In future research, the real-time health monitoring system will be installed in the isolated sections of the local hospitals. In this way, the monitoring of patients without medical staff will be quite easy. Also, the health monitoring data of the whole remote areas population will be interlinked to the server cloud, and it can be observed by using the mobile application of this system.

Conflict of interest

The authors declare that there is no conflict of interest.

Acknowledgement

The authors extend their appreciation to the Deanship of Scientific Research at Imam Mohammad Ibn Saud Islamic University for funding this work through Research Group no. RG-21-07-03.