Figure 1.
Algorithm flowchart.
In this review, we explore the advances, setbacks, and future possibilities of directed acyclic graphs (DAGs) as conceptual and analytical tools in applied and theoretical epidemiology. DAGs are literal, theoretical or speculative, and diagrammatic representations of known, uncertain, or unknown data generating mechanisms (and dataset generating processes) in which the causal relationships between variables are determined on the basis of two over-riding principles—"directionality" and "acyclicity". Among the many strengths of DAGs are their transparency, simplicity, flexibility, methodological utility, and epistemological credibility. All these strengths can help applied epidemiological studies better mitigate (and acknowledge) the impact of avoidable (and unavoidable) biases in causal inference analyses based on observational/non-experimental data. They can also strengthen the credibility and utility of theoretical studies that use DAGs to identify and explore hitherto hidden sources of analytical and inferential bias. Nonetheless, and despite their apparent simplicity, the application of DAGs has suffered a number of setbacks due to weaknesses in understanding, practice, and reporting. These include a failure to include all possible (conceivable and inconceivable) unmeasured covariates when developing and specifying DAGs; and weaknesses in the reporting of DAGs containing more than a handful of variables and paths, and where the intended application(s) and rationale(s) involved is necessary for appreciating, evaluating, and exploiting any causal insights they might offer. We proposed two additional principles to address these weaknesses and identify a number of opportunities where DAGs might lead to further advancements: The critical appraisal and synthesis of observational studies; the external validity and portability of causality-informed prediction; the identification of novel sources of bias; and the application of DAG-dataset consistency assessment to resolve pervasive uncertainty in the temporal positioning of time-variant and time-invariant exposures, outcomes, and covariates.
Citation: George TH Ellison, Hanan Rhoma. Directed acyclic graphs as conceptual and analytical tools in applied and theoretical epidemiology: advances, setbacks and future possibilities[J]. Mathematical Biosciences and Engineering, 2025, 22(6): 1280-1306. doi: 10.3934/mbe.2025048
| [1] | Jinyun Jiang, Jianchen Cai, Qile Zhang, Kun Lan, Xiaoliang Jiang, Jun Wu . Group theoretic particle swarm optimization for gray-level medical image enhancement. Mathematical Biosciences and Engineering, 2023, 20(6): 10479-10494. doi: 10.3934/mbe.2023462 |
| [2] | Kunpeng Li, Zepeng Wang, Yu Zhou, Sihai Li . Lung adenocarcinoma identification based on hybrid feature selections and attentional convolutional neural networks. Mathematical Biosciences and Engineering, 2024, 21(2): 2991-3015. doi: 10.3934/mbe.2024133 |
| [3] | Jun Gao, Qian Jiang, Bo Zhou, Daozheng Chen . Convolutional neural networks for computer-aided detection or diagnosis in medical image analysis: An overview. Mathematical Biosciences and Engineering, 2019, 16(6): 6536-6561. doi: 10.3934/mbe.2019326 |
| [4] | Yufeng Li, Chengcheng Liu, Weiping Zhao, Yufeng Huang . Multi-spectral remote sensing images feature coverage classification based on improved convolutional neural network. Mathematical Biosciences and Engineering, 2020, 17(5): 4443-4456. doi: 10.3934/mbe.2020245 |
| [5] | Hong Yu, Wenhuan Lu, Qilong Sun, Haiqiang Shi, Jianguo Wei, Zhe Wang, Xiaoman Wang, Naixue Xiong . Design and analysis of a robust breast cancer diagnostic system based on multimode MR images. Mathematical Biosciences and Engineering, 2021, 18(4): 3578-3597. doi: 10.3934/mbe.2021180 |
| [6] | Binju Saju, Neethu Tressa, Rajesh Kumar Dhanaraj, Sumegh Tharewal, Jincy Chundamannil Mathew, Danilo Pelusi . Effective multi-class lungdisease classification using the hybridfeature engineering mechanism. Mathematical Biosciences and Engineering, 2023, 20(11): 20245-20273. doi: 10.3934/mbe.2023896 |
| [7] | Sushovan Chaudhury, Kartik Sau, Muhammad Attique Khan, Mohammad Shabaz . Deep transfer learning for IDC breast cancer detection using fast AI technique and Sqeezenet architecture. Mathematical Biosciences and Engineering, 2023, 20(6): 10404-10427. doi: 10.3934/mbe.2023457 |
| [8] | S. Dinesh Krishnan, Danilo Pelusi, A. Daniel, V. Suresh, Balamurugan Balusamy . Improved graph neural network-based green anaconda optimization for segmenting and classifying the lung cancer. Mathematical Biosciences and Engineering, 2023, 20(9): 17138-17157. doi: 10.3934/mbe.2023764 |
| [9] | Yingjian Yang, Wei Li, Yingwei Guo, Nanrong Zeng, Shicong Wang, Ziran Chen, Yang Liu, Huai Chen, Wenxin Duan, Xian Li, Wei Zhao, Rongchang Chen, Yan Kang . Lung radiomics features for characterizing and classifying COPD stage based on feature combination strategy and multi-layer perceptron classifier. Mathematical Biosciences and Engineering, 2022, 19(8): 7826-7855. doi: 10.3934/mbe.2022366 |
| [10] | Haiyan Song, Cuihong Liu, Shengnan Li, Peixiao Zhang . TS-GCN: A novel tumor segmentation method integrating transformer and GCN. Mathematical Biosciences and Engineering, 2023, 20(10): 18173-18190. doi: 10.3934/mbe.2023807 |
In this review, we explore the advances, setbacks, and future possibilities of directed acyclic graphs (DAGs) as conceptual and analytical tools in applied and theoretical epidemiology. DAGs are literal, theoretical or speculative, and diagrammatic representations of known, uncertain, or unknown data generating mechanisms (and dataset generating processes) in which the causal relationships between variables are determined on the basis of two over-riding principles—"directionality" and "acyclicity". Among the many strengths of DAGs are their transparency, simplicity, flexibility, methodological utility, and epistemological credibility. All these strengths can help applied epidemiological studies better mitigate (and acknowledge) the impact of avoidable (and unavoidable) biases in causal inference analyses based on observational/non-experimental data. They can also strengthen the credibility and utility of theoretical studies that use DAGs to identify and explore hitherto hidden sources of analytical and inferential bias. Nonetheless, and despite their apparent simplicity, the application of DAGs has suffered a number of setbacks due to weaknesses in understanding, practice, and reporting. These include a failure to include all possible (conceivable and inconceivable) unmeasured covariates when developing and specifying DAGs; and weaknesses in the reporting of DAGs containing more than a handful of variables and paths, and where the intended application(s) and rationale(s) involved is necessary for appreciating, evaluating, and exploiting any causal insights they might offer. We proposed two additional principles to address these weaknesses and identify a number of opportunities where DAGs might lead to further advancements: The critical appraisal and synthesis of observational studies; the external validity and portability of causality-informed prediction; the identification of novel sources of bias; and the application of DAG-dataset consistency assessment to resolve pervasive uncertainty in the temporal positioning of time-variant and time-invariant exposures, outcomes, and covariates.
In recent years, remote sensing (RS) imagery has been widely applied in various fields, with the continuous development of RS technology. The fields include environmental monitoring [1], natural disaster monitoring [2,3,4], mineral resource development [5] and cartography [6]. However, a large part of Earth's surface is covered by clouds [7], which results in a significant amount of ground information being inaccessible to satellites. Additionally, the texture and spectral information of images are inconstant. There are substantial challenges in subsequent object detection and analysis tasks. Therefore, accurate and effective cloud detection is crucial due to being a preprocessing step for getting over the challenges before the analysis of satellite images [8].
Currently, cloud detection methods can be categorized into multi-spectral threshold methods and machine learning-based methods. Multi-spectral threshold methods use spectral differences between clouds and the underlying surface in different wavelength bands to select thresholds for cloud detection. Rossow et al. used spectral thresholding based on the 11 μm brightness temperature data of cloud tops in the international satellite cloud climatology project [9]. Xiong et al. proposed a cloud detection approach based on hybrid multi-spectral features with dynamic thresholds [10]. The multi-spectral threshold methods have a simple structure and are easy to implement. However, it is challenging for them to distinguish between clouds and the underlying surface for them when the ground is covered with ice, snow, desert or a cloud cover that is thin.
Many cloud detection algorithms based on machine learning have emerged with the rapid development of computer science, including the support vector machine (SVM) [11,12,13], random forest (RF) [14,15], convolutional neural network (CNN) [16,17] and others. Li et al. [11] utilized brightness temperature and texture features, employing SVM as a classifier for cloud detection. Zhang et al. proposed a cloud detection method based on the multi-feature embedded learning SVM [12]. Fu et al. used the results of the ensemble threshold as training samples for the RF classifier [15]. Zhang et al. designed a cascaded feature attention module based on the encoder-decoder structure to obtain and enhance useful feature information [16]. Xie et al. proposed a CNN with two branches to detect different types of clouds [17]. However, all the machine learning methods used by them encounter the following challenges:
(1) Cloud edges are often semi-transparent, making it difficult for them to separate the underlying surface based solely on visual features. Therefore, it is essential to make full use of the multi-channel information from RS satellites.
(2) These machine learning methods require a sufficient amount of training data to achieve reliable performance [18]. However, fragmented cloud samples are often sparse and scattered, demanding a significant amount of effort and time for manual labeling.
The GMM, as an unsupervised learning method, does not require manual sample labeling. Therefore, it requires less preparation work before detection [19]. Furthermore, visual features can have difficulty in distinguishing between cloud edges and the underlying surface during manual labeling, and incorrect labeling may result in model inaccuracies. In contrast, the GMM algorithm use satellite multi-channel reflectance and brightness temperature as clustering features and can effectively leverage the spectral differences between clouds and the underlying surface for classification.
In summary, we propose a cloud detection algorithm based on a feature enhancement module and GMM clustering. The proposed method utilizes clustering-based feature analysis to select appropriate feature schemes for different time periods, enabling cloud detection throughout the day. Additionally, we employ the Laplacian operator to enhance the spectral characteristics of the cloud edges and broken cloud regions and improve the accuracy of the GMM classifier for cloud detection.
The paper is structured as follows: Section 2 presents the various applications of cloud detection. Section 3 discusses data preprocessing, with a focus on the selection of clustering features. Section 4 presents a modular description of the proposed algorithm, highlighting its key components and functionalities. Section 5 provides a comprehensive account of all experiments conducted and their corresponding observations, shedding light on the algorithm's performance and efficacy. Section 6 discusses the ethical considerations and potential societal impacts of enhanced cloud detection. Finally, Section 7 offers a summary of the paper, summarizing the major contributions.
(1) Wildfire detection
When utilizing satellite data for wildfire monitoring, clouds' mid-infrared reflectance might cause them to be mistakenly identified as fire. Hence, cloud detection is crucial. Zhang et al. set different thresholds during daytime and nighttime to detect clouds [20]. Kang et al. employed an algorithm utilizing Imager's reflectance and BT to eliminate clouds in aerosol optical depth retrieval [21].
(2) Cartography in agriculture
To reduce spectral redundancy in images and improve method efficiency, cloud detection is often a crucial step in data preprocessing during agricultural mapping. Xia et al. utilized cloud masks generated by the C Function of Mask algorithm [22]. Li et al. calculated cloud scores according to the spectral data from six bands to achieve cloud removal [23].
(3) Vegetation change analysis
Vegetation change analysis is particularly sensitive to cloud contamination because unmarked clouds above the vegetation can potentially be mistaken for change. Even minor errors in cloud detection can lead to significant inaccuracies in downstream vegetation change analysis. Huang et al. used clear view forest pixels as references to define cloud boundaries, separating clouds from the clear view surface in the spectral-temperature space [24]. Long et al. employed cloud identification algorithms provided by Google Earth Engine, utilizing the QA band to identify clouds and shadows in the images [25].
The cloud detection algorithm in this paper is based on multi-spectral data from the Himawari-8 satellite. As shown in Figure 1, the first step involves conducting a statistical analysis using probability density functions (PDFs) of pixel spectral data from clouds and underlying surfaces to select feature schemes for different time periods. The second step involves cloud detection based on the different feature schemes selected. During nighttime, the raw data of BT15 and BTD7−14 are directly subjected to GMM clustering to obtain the detection results. During the daytime, the detection process is divided into three parts. First, a feature enhancement module is applied to R3+4, and the enhanced data is clustered to detect cloud edges and broken clouds. Furthermore, the raw data is clustered to detect thick clouds. Finally, the results from the dual branches of clustering are combined to obtain the detection results.
The definition of clouds is crucial, and many scholars currently use a subjective approach to define different types of clouds. In this paper, cloud transparency and distribution are used as criteria for subjective definition.
(1) Thick clouds are opaque and typically appear over large areas.
(2) Cloud edges refer to clouds with transparency at the cloud boundaries. Cloud edges do not abruptly separate but gradually transition between the cloud layer and the non-cloud layer.
(3) Broken clouds exhibit varying degrees of transparency, with their most noticeable characteristic being their distribution in small patches across the sky.
Figure 2 illustrates various cloud types, with some cloud edges highlighted using red boxes.
The synchronous meteorological satellite Himawari-8 was officially launched in July 2015. The satellite is equipped with the advanced Himawari imager (AHI), which has 16 observation channels covering visible, near-infrared and thermal infrared ranges [26]. The central wavelengths of these channels for detecting various objects are shown in Table 1. Additionally, AHI can provide high-resolution observation data for the entire disk at intervals of 10 minutes, and its high temporal resolution data is advantageous for target detection and analysis [27]. The open accessibility of Himawari-8/AHI data, provided by the Japan Aerospace Exploration Agency, empowers researchers to extensively leverage this data for scientific inquiry and innovation. This ease of access and the widespread availability of this data have propelled its applications across the fields of Earth sciences, meteorology and environmental studies. For instance, it plays a pivotal role in monitoring natural disasters such as storm tracking, wildfire detection and climate change observations. Moreover, these datasets offer rich information for fields like agriculture, water resource management and urban planning.
| Channel | Central wavelength/μm | Detection targets |
| 1 | 0.46 | aerosol, coastline |
| 2 | 0.51 | phytoplankton |
| 3 | 0.64 | vegetation, aerosol |
| 4 | 0.86 | stratus cloud |
| 5 | 1.6 | cloud top phase, snow |
| 6 | 2.3 | surface, cloud, snow |
| 7 | 3.9 | surface, wildfire |
| 8 | 6.2 | upper atmospheric water vapor, rainfall |
| 9 | 7.0 | mid-level atmospheric water vapor, rainfall |
| 10 | 7.3 | sulfur dioxide |
| 11 | 8.6 | water, sulfur dioxide, rainfall |
| 12 | 9.6 | ozone, airflow, wind |
| 13 | 10.4 | surface, cloud |
| 14 | 11.2 | surface, cloud |
| 15 | 12.4 | water, sea surface temperature |
| 16 | 13.3 | temperature, cloud altitude, cloudiness |
DownLoad:
CSV
As shown in Figure 3, the data retrieved from the Himawari-8 satellite comes in the network common data form (NetCDF) format, requiring preprocessing before utilization. Initially, the NetCDF format will undergo conversion into physical variables. Channels 1 to 6 will denote reflectance data, while channels 7 to 16 will contain brightness temperature data. Following this, a sequence of geometric corrections will be applied, opting for the Geographic WGS84 Projection, succeeded by cropping procedures.
Moreover, different features may have varying measurement units or numerical ranges. Normalization can alleviate these effects, enabling the model to treat each feature more evenly. The normalization formula is as follows:
| Y=X−XmeanXstd | (1) |
Where the mean and standard deviation of X are indicated by Xmean and Xstd, respectively. X and Y represent the initial and the normalized feature values, respectively.
The spectral data from the Himawari-8 satellite is analyzed using PDF to generate PDF distributions for reflectance, brightness temperature and brightness temperature difference of pixels corresponding to both clouds and the underlying surface. The selection of cluster features is completed based on the analysis of these PDF distributions. As shown in Figure 4, a random sample of cloud and underlying surface pixels from a series of RS images is used for statistical analysis.
Figure 5 shows the PDF distributions of different cluster features for cloud and underlying surface pixels. The PDF distributions for three cluster features, R3+4, BT15 and BTD7−14, exhibit some overlap for both cloud and underlying surface pixels. As shown in Figure 5(a), the PDF distribution of R3+4 for both cloud and underlying surface pixels roughly follows a normal distribution. Cloud PDF mainly ranges from 0.8 to 1.3, with a peak around 1.2, while the underlying surface PDF predominantly ranges from 0.2 to 0.35, with a peak around 0.3.
Figure 5(b) displays the PDF distribution of BT15 for clouds and underlying surface pixels. Cloud BT15 PDF is mainly distributed between 220 K and 285 K, displaying two peaks near 250 K and 270 K, respectively. In contrast, the underlying surface BT15 PDF mainly ranges from 280 K to 300 K, with a peak of around 290 K.
Figure 5(c) demonstrates that the PDF distributions of BTD7−14 for both cloud and underlying surface pixels also roughly follow a normal distribution. Cloud BTD7−14 PDF mainly ranges from 20 K to 30 K, with a peak around 23 K, while the underlying surface BTD7−14 PDF mainly ranges from 0 K to 7 K, with a peak around 2 K.
Although there are significant differences in the R3+4 PDF distributions between clouds and the underlying surface, during nighttime periods, the visible light band of AHI struggles to capture clear visible light images. Therefore, in this work, we use different cluster feature schemes for cloud detection during day and night periods: R3+4 during the day and BT15 and BTD7−14 at night.
During the process of cloud detection, cloud edges (the overlapping parts of PDFs for various cluster features in Figure 5) are prone to being misclassified as non-cloud pixels. This is primarily due to the minimal spectral feature differences between cloud edges and the underlying surface. Additionally, broken clouds are often sparse and scattered, causing their detection to be challenging. Therefore, as shown in Figure 6, to enhance the algorithm's efficiency in detecting cloud edges and broken clouds, convolution operations are employed to enhance the spectral features of cloud edges and broken clouds. The convolution operation involves sliding the sharpening convolution kernel over each pixel of the original image and computing the weighted sum of the convolution kernel with the pixel values. The formula for this enhancement is as follows:
| g(x,y)=f(x,y)∗w(i,j) | (2) |
In the equation, f(x, y) represents the pixel value at (x, y), and w(i, j) represents the convolution kernel.
The Laplacian operator, often used in image processing, plays a crucial role in extracting gradient information from images [28,29]. It accentuates abrupt changes in pixel values, making it particularly useful in highlighting boundaries and fine details, such as edges of clouds or broken clouds. It is essentially a filter that amplifies high-frequency components, aiding in the detection and emphasis of subtle variations in the image. Compare the sharpening effects of the Laplacian operator with commonly used operators. As depicted in Figure 7, the sharpening results obtained using Roberts, Prewitt and Sobel operators exhibit missing cloud edges, whereas the sharpening results at the edges using the Laplacian operator demonstrate continuity. Therefore, the Laplacian operator is chosen to enhance the features of cloud edges and broken clouds. The definition of the Laplacian operator is as follows:
| ∇2f=∂2f∂2x+∂2f∂2y | (3) |
By separately taking the second derivatives of the Laplacian operator in the x and y directions through differencing, we obtain:
| ∇2f=f(x+1,y)+f(x−1,y)+f(x,y+1)+f(x,y−1)−4f(x,y) | (4) |
The above formulas can be transformed into the form of convolution kernels:
| (0101−41010) | (5) |
After the Laplacian operator processing, the spectral features of cloud edges and broken clouds are enhanced. However, the extensive areas of thick clouds that manifest as low-frequency signals are removed. This can lead to sample imbalance and affect clustering results. Therefore, overlaying the image processed with the Laplacian operator onto the original image is crucial. The resulting feature-enhanced image can highlight cloud edges and broken cloud details while preserving the thick cloud portions from the original image. The overlay formula is as follows:
| g(x,y)=f(x,y)∗w(i,j)=f(x,y)−[∇2f] | (6) |
Finally, based on Eq (6), we obtain the convolution kernel used in this paper:
| w(i,j)=(0 - 10 - 15 - 10 - 10) | (7) |
Figure 8 provides a comparison of the results before and after feature enhancement. From Figure 8(a), (b), a noticeable improvement in the clarity of cloud edges and broken clouds that is within the RS image is observed after the feature enhancement. Additionally, the R3+4 values for cloud edges and broken clouds are extracted from the image for PDF comparison. Figure 8(c) shows that feature enhancement significantly increases the R3+4 values for cloud edges and broken clouds. This will effectively reduce the occurrence of false negatives in the algorithm for cloud edge and broken cloud pixels.
During detection, the diversity of cloud and the underlying surface results in complex distributions of reflectance and brightness temperature. GMM can represent the PDF of data by the way of using a convex combination of multiple Gaussian functions (a linear combination of multiple Gaussian functions). Given a sufficient number of samples, GMM can model distributions of any shape [30]. Therefore, GMM can accurately describe the probability distributions of reflectance and brightness temperature for both clouds and the underlying surface. Additionally, some spectral characteristics of clouds and underlying surfaces may overlap (as shown in Figure 5). The advantage of GMM compared to clustering algorithms like K-means and mean-shift lies in its more flexible modeling capability. Unlike K-means and other hard clustering methods, GMM has the ability to perform soft clustering which allows data points to belong to different clusters with certain probabilities. This flexibility helps in better understanding and representing relationships between data points, making the model more adaptable to clusters that exhibit overlap or have fuzzy boundaries. Additionally, GMM uses probability distributions to describe clusters, which makes it better suited for handling complex data distributions, especially when dealing with non-spherical distributions where GMM often outperforms other algorithms [31]. Its PDF formula is as follows:
| P(x|θ)=K∑j=1βjf(x|μj,σj2) | (8) |
In the equation, x represents a vector X = {x1, x2, x3, ..., xn} consisting of n sample values, where X contains K categories. The weight of the jth class is denoted by βj, and the function f(x) is the PDF of a single Gaussian model with mean μj and variance σj for the jth class.
The set of data for the study area, R3+4, BT15 and BTD7-14, is denoted as X = {x1, x2, x3, ..., xn}, where xi is a sample drawn from the set X. The elements in the set are independent of each other and follow a GMM. As shown in Eq (7), the GMM density function contains a parameter set θ = {β, μ, σ2} and this parameter set is defined as θ. For a single Gaussian model, the parameters θ can be estimated using the Maximum Likelihood (ML) method, where the likelihood function is given by the PDF:
| L(θ)=n∏i=1P(x|θ)=n∏i=11√2πσ2exp(−(xi−μ)22σ2) | (9) |
Similarly, the likelihood function can be derived from the GMM PDF as follows:
| L(θ)=n∏i=1P(x|θ)=n∑i=1log[K∑j=1βjf(x|μj,σj2)] | (10) |
In the equation, there is a complex nonlinear relationship between the likelihood function L (θ) and the parameter set θ, making it impractical to directly maximize the likelihood function through differentiation. The Expectation-Maximization (EM) algorithm is a common method for maximum likelihood estimation and is suitable for data with hidden, unknown variables. It can appropriately estimate the optimal parameters of probability models [32,33]. The specific steps of the EM algorithm are as follows:
Step 1: Initialize the parameter set θ = {β, μ, σ2} for each category (hidden variables) based on prior knowledge.
Step 2: Calculate the probability that the data is generated by a Gaussian distribution as follows:
| γ(i,j)=βjP(xi|μj,σj2)K∑j=1βjP(xi|μj,σj2) | (11) |
Step 3: Update the μ and σ2 for each Gaussian model as follows:
| μj=n∑i=1γ(i,j)−xin∑i=1γ(i,j) | (12) |
| σj2=n∑i=1γ(i,j)(xi−μj)(xi−μj)Tn∑i=1γ(i,j) | (13) |
Step 4: Update the β for each Gaussian model as follows:
| βj=n∑n=1γ(i,j)n | (14) |
Repeat Steps 2 through Step 4 until the maximum likelihood value L (θ) approaches stability. Once this condition is met, the algorithm is complete.
Before applying the GMM algorithm for clustering, the value of K needs to be determined. In practical situations, the types of distributions contained in the data samples are unknown. As the number of sub-models in the observed samples increases, the mixture model better reflects the characteristics of the data samples. However, this also leads to a more complex model. A balance point should be established to maintain the simplicity of the mixture model and ensure the accuracy of the model fit. Therefore, how to determine the value of parameter K to ensure that the mixture model is both accurate and concise is a primary goal to optimize the model. We use the AIC and BIC criterions from information statistics to determine the parameter K and their expressions are as follows:
| VAIC=2m−lnL(x|K,β,μ,σ2)VBIC=mlnn−lnL(x|K,β,μ,σ2) | (15) |
In the equations, m is the number of parameters in the mixture model, n is the number of samples, and lnL (x|K, β, μ, σ2) represents the maximum likelihood value. The minimum value of VAIC and VBIC is taken as the criteria for selecting the optimal model.
First, four sets of RS images of R3+4 are selected as sample data, and the range for selecting K is set between 1 and 14. VAIC and VBIC are calculated for different values of K. As shown in Figure 9, with the increase of K, both VAIC and VBIC show an overall decreasing trend, and the trend gradually stabilizes when K > 8. To further determine the specific value of K, values of K are set to 5, 6, 7 and 8, and the GMM cloud detection results for different K values are compared to determine the appropriate value of K. Figure 10 shows the comparison of cloud pixel omissions for K = 5, 6 and 7. As the increase in the value of K, the number of omitted cloud pixels gradually decreases. When K = 7, the algorithm produces detection results for various types of clouds that closely align with the actual cloud coverage. Furthermore, as shown in Figure 11, there is tiny little difference in the detection results of cloud edge details between K = 7 and K = 8. Therefore, K = 7 ensures that the mixture model is both accurate and concise.
The Himawari-8 data was provided by the Japan Aerospace Exploration Agency (https://www.eorc.jaxa.jp/ptree/). In this study, a dataset was constructed for AHI, encompassing partial data from 2021 and 2022. This dataset includes both full disk images and study area images, with over 1,400,000 pixels annotated.
The computer used for the experiments has an AMD R5 5600G processor with a base frequency of 2900 MHz, a 1 TB solid-state drive and 16 GB of RAM. The experiment uses the OpenCV and Scikit-Learn packages to implement the proposed algorithm.
The GMM algorithm selected in this study requires pre-determination of the number of clusters, similar to the K-means algorithm. Therefore, to demonstrate the advantage of GMM's soft clustering capability over K-means, a comparison was made. The selected images were taken during early morning or evening when the reflectance of clouds is lower compared to daytime, posing a significant challenge to the clustering algorithms. As shown in Figure 12 within the red box, K-means wrongly classified many low-reflectance clouds as underlying surfaces. This is because K-means tends to rigidly assign data points to the nearest cluster center when dealing with non-convex shapes or overlapping regions. Consequently, some low-reflectance clouds were erroneously categorized as land features by K-means. On the other hand, GMM is capable of better reflecting the probability of data points belonging to different clusters. Therefore, in such scenarios, GMM might more accurately identify areas with ambiguous boundaries. It excels in capturing complex data distributions and delineating regions with unclear boundaries, which might lead to a more precise identification of low-reflectance clouds compared to K-means.
The proposed algorithm uses a feature enhancement module to enhance the features of cloud edges and broken clouds, and it integrates the detection results of cloud edges and broken clouds in the final detection results, thereby improving the overall accuracy of the algorithm. Figure 13 shows the heatmap of the proposed algorithm, which reflects the aggregation of broken clouds and cloud edges with their respective categories. In Figure 13(a), (b), the model's attention to broken clouds is evident (highlighted in yellow boxes). Figure 13(b), (c) depict the attention to cloud edges, and it can be observed that the cloud edges of thin clouds also receive significant attention (highlighted in yellow boxes).
Traditional methods such as the multi-spectral threshold method are widely used, and machine learning methods like RF, KNN and GMM have efficient classification capabilities. Therefore, this paper implemented the proposed algorithm and the aforementioned algorithm and implemented cloud detection experiments on Himawari-8 images from different time phases and regions to validate the applicability of the proposed algorithm.
Figure 14 presents the comparison of cloud detection results using the multi-spectral threshold method [10], RF [15], KNN [34], GMM and the proposed algorithm. In the figure, the true labels obtained through visual interpretation are depicted in yellow, while the detection results are shown in white rendering. Professionally trained experts or analysts, following the cloud definition method outlined in Section 3.1, utilized Photoshop software to meticulously annotate the cloud's location pixel by pixel.
In Figure 14(a), the detection results of GMM and the proposed algorithm are the most accurate, while the multi-spectral threshold method and RF exhibit a slight underestimation of cloud pixels, primarily at the cloud edges. KNN shows the highest number of cloud pixel omissions (highlighted within the green bounding boxes in the images). In Figure 14(b), GMM and the proposed algorithm provide the best detection results, with the multi-spectral threshold method and RF exhibit the greatest number of cloud pixels omission. KNN exhibits a large area of omissions in the lower right corner of the image and it also misclassifies some non-cloud pixels as clouds at the cloud edges (highlighted within the purple bounding boxes for omissions and the yellow bounding boxes for misclassifications). In Figure 14(c), (d), the cloud edges in the upper right corners of the images are somewhat blurred. Upon comparison, it is evident that the proposed algorithm produces the most accurate detection results. Moreover, the multi-spectral threshold method, RF and GMM all exhibit varying degrees of omissions at the cloud edges. Multi-spectral threshold method has the most pronounced omission phenomenon, followed by RF (highlighted within the blue bounding boxes for omissions and the yellow bounding boxes for misclassifications).
For the detection of broken clouds, as depicted within the red box in Figure 14(d), the multi-spectral threshold method exhibits the highest number of missed broken cloud pixels, followed by RF and GMM. Additionally, KNN misclassifies numerous pixels as clouds. Analysis of the above comparison results reveals that in sufficient sunlight conditions during the day, the reflectance of clouds in the visible light spectrum is relatively high. During this time, the multi-spectral threshold method is more suitable. However, as it approaches evening, decreasing light levels can lead to higher threshold values, which is not conducive to cloud detection. RF and KNN, as supervised learning algorithms, have high requirements for sample size and the balance of samples among different categories. However, the small and dispersed nature of scattered clouds poses challenges in creating training samples for RF and KNN, making it difficult for these algorithms to learn the necessary features effectively.
In contrast, as an unsupervised learning algorithm, GMM can automatically learn the inherent relationships between different categories and perform classification without the need for an extensive set of training samples. Because the spectral characteristics of cloud edges are somewhat similar to those of underlying surfaces, the threshold set by the multi-spectral threshold method is relatively high for cloud edge detection. While GMM has several advantages in detecting large-area thick clouds, it does not consider the spectral feature similarity between cloud edges and underlying surfaces. Therefore, the proposed algorithm achieves the best detection results for various cloud types.
Additionally, as shown in Figure 15, full disk RS data from September 24, 2022, at UTC 03:00 are selected for cloud detection. Among the five algorithms, the multi-spectral threshold method and GMM achieve relatively good detection results, but there are instances of cloud underestimation in some regions (highlighted within the green boxes). RF and KNN show varying degrees of cloud false positives (highlighted within the red boxes). The proposed algorithm, an enhancement of GMM, reduces the underestimation of cloud pixels, resulting in the best detection performance among the five algorithms, demonstrating that this algorithm provides reliable results even for large-scale areas.
To quantitatively evaluate these methods, this study uses two evaluation metrics, the hit rate (HR) and the Hansen-Kuiper skill score (KSS), to analyze 15 RS images from different times and regions. The HR represents the proportion of correctly identified cloud and clear-sky pixels to the total number of pixels. The KSS score is used to provide a comprehensive overall measure. The formulas for calculating HR and KSS are as follows:
| HR=TC+TUPA | (16) |
| KSS=TC×TU−TF×FT(TC+FT)(TU+TF) | (17) |
In the formulas, TC represents the number of pixels correctly detected as clouds, TU is the number of pixels that are actually non-cloud but are detected as clouds, FT is the number of pixels that are actually clouds but are detected as non-cloud and PA is the total number of pixels in the image.
Figure 16 shows the evaluation results of different algorithms for detecting thick clouds, cloud edges and broken clouds. The values of HR and KSS approaching 1 indicate superior algorithm performance. For thick cloud detection, the HR values of the multi-spectral threshold method, RF, KNN, GMM and the proposed algorithm are 0.793, 0.832, 0.799, 0.974 and 0.976, respectively, and the KSS values are 0.692, 0.763, 0.642, 0.973 and 0.974, respectively. It is evident that GMM and the proposed algorithm outperform the other three algorithms in thick cloud detection. For cloud edge and broken cloud detection, the HR values of the five algorithms are 0.619, 0.687, 0.624, 0.963 and 0.974, respectively, and the KSS values are 0.458, 0.561, 0.563, 0.947 and 0.973, respectively. Compared to thick cloud detection, the HR and KSS values of the multi-spectral threshold method, RF and KNN significantly decrease in cloud edge and broken cloud detection, while those of GMM and the proposed algorithm remain above 0.96 and 0.94, respectively. Moreover, the proposed algorithm, which incorporates the feature enhancement module, shows an improvement of 0.011 in HR and 0.026 in KSS compared to GMM. This indicates that the feature enhancement module aids the classification model in extracting deeper cloud edge and broken cloud features. Additionally, as shown in Table 2, 20 images sized at 256 × 256 were selected for quantitative analysis to demonstrate the overall performance of the algorithm. Compared to GMM, the proposed algorithm shows an improvement of 0.002 in HR and 0.015 in KSS.
| Algorithm | HR | KSS |
| Multi-spectral threshold method | 0.763 | 0.572 |
| RF | 0.811 | 0.65 |
| KNN | 0.753 | 0.606 |
| GMM | 0.973 | 0.958 |
| Proposed algorithm | 0.975 | 0.973 |
DownLoad:
CSV
When performing target detection and task analysis, the vast amount of RS data meant that the runtime of cloud detection algorithms is an important concern. To address this, RS images of sizes 1000 × 1100, 2401 × 2401 and 60016001 are selected to compare the runtime differences between different algorithms. As shown in Figure 17, when the image sizes are 1000 × 1100 and 2401 × 2401, the runtime differences among the four machine learning algorithms does not vary significantly, with GMM having the shortest runtime, followed by the proposed algorithm, RF and KNN. However, when the image size is 6001 × 6001, KNN's runtime is significantly longer than the other three algorithms (approximately 3.9 times longer than GMM and 3.65 times longer than RF), while the proposed algorithm with the feature enhancement module has a runtime only 7.45 seconds longer than GMM. Therefore, for cloud detection in large-sized RS images, the algorithm proposed in this study can maintain high accuracy while having a relatively short runtime.
While the proposed algorithm generally achieves satisfactory cloud detection results in most scenarios, it encounters errors, particularly when dealing with cloud overlap. This defect is especially obvious when dealing with cloud shadows. These shadows, appearing as dark regions in visible light images, share spectral characteristics similar to the underlying surfaces. The similarity poses a challenge in distinguishing clouds within these shadowed areas, leading to errors in identification. Figure 18 illustrates examples of cloud detection errors specifically within shadowed regions, highlighting numerous omitted pixels in the detection results. In addition, the proposed algorithm is tailored for satellites capable of providing 2 km resolution images. With higher image resolution, the diversity of surface types becomes more intricate, leading to fluctuations in the optimal K value. In such scenarios, extensive experimental analysis becomes crucial for re-evaluating the selection of the K value. Even then, pinpointing a stable K value can be challenging. For instance, in an image observed at 2 km resolution, a pixel m (x, y) might initially be identified as a cloud. However, as the resolution increases, the area represented by m (x, y) encompasses more pixels. Among these newly added pixels, some might reveal underlying surfaces that were previously unobservable at the 2 km resolution.
The future research goal is to combine thermal infrared spectroscopy with neural network algorithms to enhance the algorithm's transferability. Moreover, by exploring the imaging correlation between clouds and their shadows, there is potential to calculate the orientation and distance of shadows concerning cloud areas. This computational approach could be helpful in effectively filtering out these shadows from the detection results.
The improved cloud detection technology holds significant potential in disaster management. Accurate cloud coverage data is meaningful to monitoring and early warning systems for natural disasters such as storms and floods, aiding in the proactive implementation of response measures and minimizing potential losses. However, when applying these technologies, the critical importance of data accuracy and timeliness in decision-making must be considered to prevent the potential risks of misinformation or delays.
Furthermore, advancements in cloud detection technology may raise concerns regarding personal privacy, particularly in the monitoring and analysis of extensive areas. Therefore, safeguarding personal information and data privacy should be a crucial consideration when utilizing these algorithms, necessitating the establishment of stringent guidelines for data processing and sharing.
In this paper, we propose a multi-spectral data cloud detection algorithm based on a feature enhancement module and Gaussian mixture model (GMM) clustering. First, the spectral data of clouds and underlying surfaces are statistically analyzed. Based on the analysis results, an appropriate feature scheme is selected to achieve cloud detection throughout the day. Second, the Laplacian operator is used to enhance the spectral features of cloud edges and broken clouds. The enhanced data is then input into a GMM classifier for cloud edge and broken cloud detection. The original RS data is simultaneously subjected to GMM clustering to obtain thick cloud detection results. Finally, the results from the both branches are combined to obtain the final cloud detection result.
Comparing the results of the multi-spectral threshold methods, RF, KNN, GMM and the proposed algorithm, the following conclusions are drawn: The proposed algorithm achieves the highest accuracy with a hit rate (HR) of 0.976 for the case of thick cloud detection and a Hansen-Kuiper skill score (KSS) of 0.974 for the case of cloud edges and broken cloud detection. This means the proposed algorithm demonstrates significant advantages over other algorithms, with an HR and KSS improvement of 0.011 and 0.026, respectively, compared to GMM. Furthermore, the inclusion of the feature enhancement module in the proposed algorithm maintains a relatively short runtime. This further demonstrates that the proposed algorithm can achieve fast and high-precision cloud detection for different types of clouds. Therefore, the proposed algorithm can provide real-time cloud detection results for various tasks such as vegetation change analysis, disaster monitoring and cartography, offering a viable solution to alleviate the challenges posed by cloud interference.
The authors declare that they have not used Artificial Intelligence (AI) tools in the creation of this article.
This work is supported by the Major Scientific and Technological Projects of Yunnan Province: Research on Key Technologies of Ecological Environment Monitoring and Intelligent Management of Natural Resources in Yunnan (202202AD080010), the National Natural Science Foundation of China (52107017) and the Fundamental Research Fund of Science and Technology Department of Yunnan Province (202201AU070172).
The authors declare that there are no conflicts of interest.
| [1] | N. L. Biggs, E. K. Lloyd, R. J. Wilson, Graph Theory, 1736–1936, Oxford University Press, 1986. |
| [2] | G. R. Law, R. Green, G. T. H. Ellison, Confounding and causal path diagrams, in Modern Methods for Epidemiology, (eds. Y. K. Tu and D. C. Greenwood), Springer, (2012), 1–13. http://dx.doi.org/10.1007/978-94-007-3024-3_1 |
| [3] |
J. Zhou, M. Müller, Depth-first discovery algorithm for incremental topological sorting of directed acyclic graphs, Inf. Process. Lett., 88 (2003), 195–200. https://doi.org/10.1016/j.ipl.2003.07.005 doi: 10.1016/j.ipl.2003.07.005
|
| [4] | I. A. Kader, Path partition in directed graph-modeling and optimization, New Trend. Math. Sci., 1 (2013), 74–84. |
| [5] |
P. W. G. Tennant, E. J. Murray, K. F. Arnold, L. Berrie, M. P. Fox, S. C. Gadd, et al., Use of directed acyclic graphs (DAGs) to identify confounders in applied health research: Review and recommendations, Int. J. Epidemiol., 50 (2021), 620–632. https://doi.org/10.1093/ije/dyaa213 doi: 10.1093/ije/dyaa213
|
| [6] | G. T. H. Ellison, Using directed acyclic graphs (DAGs) to represent the data generating mechanisms of disease and healthcare pathways: A guide for educators, students, practitioners and researchers, in Teaching Biostatistics in Medicine and Allied Health Sciences, (eds. R. J. Medeiros Mirra and D. Farnell), Springer Verlag, (2023), 61–101. |
| [7] |
M. Lewis, A. Kuerbis, An overview of causal directed acyclic graphs for substance abuse researchers, J. Drug Alcohol. Res., 5 (2016), 1–8. https://doi.org/10.4303/jdar/235992 doi: 10.4303/jdar/235992
|
| [8] |
Z. M. Laubach, E. J. Murray, K. L. Hoke, R. J. Safran, W. Perng, A biologist's guide to model selection and causal inference, Proc. Biol. Sci., 288 (2021), 20202815. https://doi.org/10.1098/rspb.2020.2815 doi: 10.1098/rspb.2020.2815
|
| [9] | B. Sauer, T. J. VanderWeele, Use of directed acyclic graphs, in Developing a Protocol for Observational Comparative Effectiveness Research: A User's Guide, (eds. P. Velentgas, N. A. Dreyer, P. Nourjah, S. R. Smith and M. M. Torchia), Agency for Healthcare Research and Quality (2013), 177–183. |
| [10] |
J. C. Digitale, J. N. Martin, M. M. Glymour, Tutorial on directed acyclic graphs, J. Clin. Epidemiol., 142 (2021), 264–267. https://doi.org/10.1016/j.jclinepi.2021.08.001 doi: 10.1016/j.jclinepi.2021.08.001
|
| [11] |
H. Iwata, T. Wakabayashi, R. Kato, The dawn of directed acyclic graphs in primary care research and education, J. Gen. Fam. Med., 24 (2023), 274. https://doi.org/10.1002/jgf2.627 doi: 10.1002/jgf2.627
|
| [12] |
R. A. Rose, J. A. Cosgrove, B. R. Lee, Directed acyclic graphs in social work research and evaluation: A primer. J. Soc. Social. Work Res., 15 (2024), 391–415. http://dx.doi.org/10.1086/723606 doi: 10.1086/723606
|
| [13] | S. Fergus, DAGs in data engineering: A powerful, problematic tool, Shipyard. Blog., (2024). |
| [14] | P. Hünermund, Causal Data Science with Directed Acyclic Graphs, Copenhagen Business School, University of Copenhagen (2021). https://web.archive.org/web/20200523155727/https://www.udemy.com/course/causal-data-science/ |
| [15] | C. S. Aneshensel, Theory-Based Data Analysis for the Social Sciences, SAGE Publications, 2002. https://doi.org/10.4135/9781412986342 |
| [16] | J. Pearl, Probabilistic Reasoning in Intelligent Systems, Elsevier, 1988. https://doi.org/10.1016/C2009-0-27609-4 |
| [17] |
T. R. Frieden, Evidence for health decision making—beyond randomized, controlled trials, N. Engl. J. Med., 377 (2017), 465–475. https://doi.org/10.1056/nejmra1614394 doi: 10.1056/nejmra1614394
|
| [18] |
M. Piccininni, S. Konigorski, J. L. Rohmann, T. Kurth, Directed acyclic graphs and causal thinking in clinical risk prediction modelling, BMC Med. Res. Methodol., 20 (2020), 179. https://doi.org/10.1186/s12874-020-01058-z doi: 10.1186/s12874-020-01058-z
|
| [19] |
L. Lin, M. Sperrin, D. A. Jenkins, G. P. Martin, N. Peek, A scoping review of causal methods enabling predictions under hypothetical interventions, Diagn. Progn. Res., 5 (2021), 1–6. https://doi.org/10.1186/s41512-021-00092-9 doi: 10.1186/s41512-021-00092-9
|
| [20] |
P. Msaouel, J. Lee, J. A. Karam, P. F. Thall, A causal framework for making individualized treatment decisions in oncology, Cancers, 14 (2022), 3923. https://doi.org/10.3390/cancers14163923 doi: 10.3390/cancers14163923
|
| [21] |
J. Fehr, M. Piccininni, T. Kurth. S. Konigorski, Assessing the transportability of clinical prediction models for cognitive impairment using causal models, BMC Med. Res. Meth., 23 (2023), 187. https://doi.org/10.1186/s12874-023-02003-6 doi: 10.1186/s12874-023-02003-6
|
| [22] | CRRS (Committee on Reproducibility and Replicability in Science), Reproducibility and Replicability in Science, National Academies Press, (2019), 1–256. https://doi.org/10.17226/25303 |
| [23] | R. A. Alfawaz, Exploring the Relationship Between Metabolic Syndrome and Sleep Amongst Adults in the UK, Ph. D thesis, University of Leeds Medical School, (2017). |
| [24] |
J. Textor, B. van der Zander, M. S. Gilthorpe, M. Liśkiewicz, G. T. H. Ellison, Robust causal inference using directed acyclic graphs: the R package 'dagitty', Int. J. Epidemiol., 45 (2016), 1887–1894. https://doi.org/10.1093/ije/dyw341 doi: 10.1093/ije/dyw341
|
| [25] | M. Fiore, M. Devesas Campos, The algebra of directed acyclic graphs, in Computation, Logic, Games, and Quantum Foundations. The Many Facets of Samson Abramsky, (eds. B. Coecke, L. Ong and P. Panangaden), Springer, (2013), 37–51. https://doi.org/10.1007/978-3-642-38164-5_4 |
| [26] |
S. Geneletti, S. Richardson, N, Best, Adjusting for selection bias in retrospective, case-control studies, Biostatistics, 10 (2009), 17–31. https://doi.org/10.1093/biostatistics/kxn010 doi: 10.1093/biostatistics/kxn010
|
| [27] |
G. T. H. Ellison, Might temporal logic improve the specification of directed acyclic graphs (DAGs)?, J. Stat. Data Sci. Educ., 29 (2021), 202–213. https://doi.org/10.1080/26939169.2021.1936311 doi: 10.1080/26939169.2021.1936311
|
| [28] |
A. Tafti, G. Shmueli, Beyond overall treatment effects: Leveraging covariates in randomized experiments guided by causal structure, Inf. Syst. Res., 31 (2020), 1183–1199. https://dx.doi.org/10.2139/ssrn.3331772 doi: 10.2139/ssrn.3331772
|
| [29] | F. E. Raimondi, T. O'Keeffe, H. Chockler, A. R. Lawrence, T. Stemberga, A. Franca, et al., Causal analysis of the TOPCAT trial: Spironolactone for preserved cardiac function heart failure, preprint, arXiv: 2211.12983. https://doi.org/10.48550/arXiv.2211.12983 |
| [30] |
G. J. Griffith, T. T. Morris, M. J. Tudball, A. Herbert, G. Mancano, L. Pike, et al., Collider bias undermines our understanding of COVID-19 disease risk and severity, Nat. Commun., 11 (2020), 5749. https://doi.org/10.1038/s41467-020-19478-2 doi: 10.1038/s41467-020-19478-2
|
| [31] |
M. G. Hudgens, M. E. Halloran, Toward causal inference with interference, J. Am. Stat. Ass., 103 (2008), 832–842. https://doi.org/10.1198/016214508000000292 doi: 10.1198/016214508000000292
|
| [32] |
R. M. Baron, D. A. Kenny, The moderator–mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations, J. Pers. Soc. Psychol., 51 (1986), 1173–1182. https://doi.org/10.1037//0022-3514.51.6.1173 doi: 10.1037//0022-3514.51.6.1173
|
| [33] |
T. J. VanderWeele, A unification of mediation and interaction: A four-way decomposition, Epidemiology, 25 (2014), 749–761. https://doi.org/10.1097/ede.0000000000000121 doi: 10.1097/ede.0000000000000121
|
| [34] |
R. H. Groenwold, T. M. Palmer, K. Tilling, To adjust or not to adjust? When a "confounder" is only measured after exposure, Epidemiology, 32 (2021), 194–201. https://doi.org/10.1097/ede.0000000000001312 doi: 10.1097/ede.0000000000001312
|
| [35] | M. Viswanathan, N. D. Berkman, D. M. Dryden, L. Hartling, Assessing Risk of Bias and Confounding in Observational Studies of Interventions or Exposures: Further Development of the RTI Item Bank, Agency for Healthcare Research and Quality, 2013. |
| [36] |
T. S. Al-Jewair, N. Pandis, Y. K. Tu, Directed acyclic graphs: A tool to identify confounders in orthodontic research, Part Ⅱ, Am. J. Orthod. Dentofacial. Orthop., 151 (2017), 619–621. https://doi.org/10.1016/j.ajodo.2016.12.003 doi: 10.1016/j.ajodo.2016.12.003
|
| [37] | B. van der Zander, M. Liśkiewicz, J. Textor, Constructing separators and adjustment sets in ancestral graphs, in Proceedings of the UAI 2014 Conference on Causal Inference, 1274 (2014), 11–24. |
| [38] |
S. Greenland, J. Pearl, J. M. Robins, Causal diagrams for epidemiologic research, Epidemiology, 10 (1999), 37–48. https://doi.org/10.1097/00001648-199901000-00008 doi: 10.1097/00001648-199901000-00008
|
| [39] |
F. Elwert, C. Winship, Endogenous selection bias: The problem of conditioning on a collider variable, Ann. Rev. Sociol., 40 (2014), 31–53. https://doi.org/10.1146/annurev-soc-071913-043455 doi: 10.1146/annurev-soc-071913-043455
|
| [40] |
T. B. Dondo, M. Hall, T. Munyombwe, C. Wilkinson, M. E. Yadegarfar, A. Timmis, et al., A nationwide causal mediation analysis of survival following ST-elevation myocardial infarction, Heart, 106 (2020), 765–771. https://doi.org/10.1136/heartjnl-2019-315760 doi: 10.1136/heartjnl-2019-315760
|
| [41] |
D. A. Freedman, On regression adjustments to experimental data, Adv. Appl. Math., 40 (2008), 180–193. https://doi.org/10.1016/j.aam.2006.12.003 doi: 10.1016/j.aam.2006.12.003
|
| [42] |
M. Mueller, M. D'Addario, M. Egger, M. Cevallos, O. Dekkers, C. Mugglin, et al., Methods to systematically review and meta-analyse observational studies: A systematic scoping review of recommendations, BMC Med. Res. Methodol., 18 (2018), 44. https://doi.org/10.1186/s12874-018-0495-9 doi: 10.1186/s12874-018-0495-9
|
| [43] |
O. M. Dekkers, J. P. Vandenbroucke, M. Cevallos, A. G. Renehan, D. G. Altman, M. Egger, COSMOS-E: Guidance on conducting systematic reviews and meta-analyses of observational studies of etiology, PLoS Med., 16 (2019), 1002742. https://doi.org/10.1371/journal.pmed.1002742 doi: 10.1371/journal.pmed.1002742
|
| [44] |
G. Sarri, E. Patorno, H. Yuan, J. J. Guo, D. Bennett, X. Wen, et al., Framework for the synthesis of non-randomised studies and randomised controlled trials: A guidance on conducting a systematic review and meta-analysis for healthcare decision making, Brit. Med. J. Evid. Based Med., 27 (2020), 109–119. https://doi.org/10.1136/bmjebm-2020-111493 doi: 10.1136/bmjebm-2020-111493
|
| [45] |
A. Ankan, I. M. Wortel, J. Textor, Testing graphical causal models using the R package "dagitty", Curr. Protoc., 1 (2021), e45. https://doi.org/10.1002/cpz1.45 doi: 10.1002/cpz1.45
|
| [46] |
P. Ding, L. W. Miratrix, To adjust or not to adjust? Sensitivity analysis of M-bias and butterfly-bias, J. Caus. Inf., 3 (2015), 41–57. https://doi.org/10.1515/jci-2013-0021 doi: 10.1515/jci-2013-0021
|
| [47] |
T. J. VanderWeele, Commentary: Resolutions of the birthweight paradox: Competing explanations and analytical insights, Int. J. Epidemiol., 43 (2014), 1368–1373. https://doi.org/10.1093/ije/dyu162 doi: 10.1093/ije/dyu162
|
| [48] |
B. N. Detweiler, L. E. Kollmorgen, B. A. Umberham, R. J. Hedin, B. M. Vassar, Risk of bias and methodological appraisal practices in systematic reviews published in anaesthetic journals: A meta‐epidemiological study, Anaesthesia, 71 (2016), 955–968. https://doi.org/10.1111/anae.13520 doi: 10.1111/anae.13520
|
| [49] |
G. T. H. Ellison, COVID-19 and the epistemology of epidemiological models at the dawn of AI, Ann. Hum. Biol., 47 (2020), 506–513. https://doi.org/10.1080/03014460.2020.1839132 doi: 10.1080/03014460.2020.1839132
|
| [50] |
S. J. Pocock, T. J. Collier, K. J. Dandreo, B. L. de Stavola, M. B. Goldman, L. A. Kalish, et al., Issues in the reporting of epidemiological studies: A survey of recent practice, Brit. Med. J., 329 (2004), 883. https://doi.org/10.1136/bmj.38250.571088.55 doi: 10.1136/bmj.38250.571088.55
|
| [51] |
E. von Elm, M. Egger, The scandal of poor epidemiological research, Brit. Med. J., 329 (2004), 868–869. https://doi.org/10.1136/bmj.329.7471.868 doi: 10.1136/bmj.329.7471.868
|
| [52] |
A. Blair, P. Stewart, J. H. Lubin, F. Forastiere, Methodological issues regarding confounding and exposure misclassification in epidemiological studies of occupational exposures, Am. J. Ind. Med., 50 (2007), 199–207. https://doi.org/10.1002/ajim.20281 doi: 10.1002/ajim.20281
|
| [53] |
T. Kurth, Continuing to advance epidemiology, Front. Epidemiol., 1 (2021), 782374. https://doi.org/10.3389/fepid.2021.782374 doi: 10.3389/fepid.2021.782374
|
| [54] |
V. Tomić, I. Buljan, A. Marušić, Perspectives of key stakeholders on essential virtues for good scientific practice in research areas, Account. Res., 29 (2021), 77–108. https://doi.org/10.1080/08989621.2021.1900739 doi: 10.1080/08989621.2021.1900739
|
| [55] |
J. Textor, J. Hardt, S. Knüppel, DAGitty: A graphical tool for analyzing causal diagrams, Epidemiology, 22 (2011), 745. https://doi.org/10.1097/ede.0b013e318225c2be doi: 10.1097/ede.0b013e318225c2be
|
| [56] | J. Textor, DAGbase: A database of human-drawn causal diagrams, Proc. Eur. Causal. Inf. Mtg., (2020). |
| [57] |
T. Stacey, P. W. G. Tennant, L. M. E. McCowan, E. A. Mitchell, J. Budd, M. Li, et al., Gestational diabetes and the risk of late stillbirth: A case-control study from England, UK, Brit. J. Obstet. Gynaecol., 126 (2019), 973–982. https://doi.org/10.1111/1471-0528.15659 doi: 10.1111/1471-0528.15659
|
| [58] | N. Swartz, The Concept of Physical Law, 2nd Edition, Cambridge University Press, 2003. |
| [59] | G. E. P. Box, Science and statistics, J. Am. Stat. Ass., 71 (1976), 791–799. |
| [60] |
G. T. H. Ellison, R. B. Mattes, H. Rhoma, T. de Wet, Economic vulnerability and poor service delivery made it more difficult for shack-dwellers to comply with COVID-19 restrictions, S. Afr. J. Sci., 118 (2022), 1–5. https://doi.org/10.17159/sajs.2022/13301 doi: 10.17159/sajs.2022/13301
|
| [61] |
G. A. Escobar, D. Burks, M. R. Abate, M. F. Faramawi, A. T. Ali, L. C. Lyons, et al., Risk of acute kidney injury after percutaneous pharmacomechanical thrombectomy using AngioJet in venous and arterial thrombosis, Ann. Vasc. Surg., 42 (2017), 238–245. https://doi.org/10.1016/j.avsg.2016.12.018 doi: 10.1016/j.avsg.2016.12.018
|
| [62] | D. Hume, A Treatise of Human Nature, John Noon, 1738. |
| [63] | M. Barrows, G. T. H. Ellison, 'Belief-consistent information processing' vs. 'coherence-based reasoning': Pragmatic frameworks for exposing common cognitive biases in intelligence analysis, preprint, (2024), 2024011338. https://doi.org/10.20944/preprints202401.1338.v1 |
| [64] | R. Foraita, J. Spallek, H. Zeeb, Directed acyclic graphs, in Handbook of Epidemiology, (eds. W. Ahrens and I. Pigeot), Springer, (2014), 1481–1517. https://doi.org/10.1007/978-0-387-09834-0_65 |
| [65] |
M. J. Gardner, D. G. Altman, Confidence intervals rather than P values: Estimation rather than hypothesis testing, Brit. Med. J., 292 (1986), 746–750. https://doi.org/10.1136/bmj.292.6522.746 doi: 10.1136/bmj.292.6522.746
|
| [66] |
Y. K. Tu, D. Gunnell, M. S. Gilthorpe, Simpson's Paradox, Lord's Paradox, and suppression effects are the same phenomenon–the reversal paradox, Emerg. Themes Epidemiol., 5 (2008), 1–9. https://doi.org/10.1186/1742-7622-5-2 doi: 10.1186/1742-7622-5-2
|
| [67] |
E. F. Schisterman, S. R. Cole, R. W. Platt, Overadjustment bias and unnecessary adjustment in epidemiologic studies, Epidemiology, 20 (2009), 488–495. https://doi.org/10.1097/ede.0b013e3181a819a1 doi: 10.1097/ede.0b013e3181a819a1
|
| [68] |
L. Richiardi, R. Bellocco, D. Zugna, Mediation analysis in epidemiology: Methods, interpretation and bias, Int. J. Epidemiol., 42 (2013), 1511–1519. https://doi.org/10.1093/ije/dyt127 doi: 10.1093/ije/dyt127
|
| [69] | K. J. Rothman, T. L. Lash, Epidemiologic study design with validity and efficiency considerations, Chapter 6, in Modern Epidemiology, (eds. T. L. Lash, T. J. VanderWeele, S. Haneuse and K. J. Rothman), Wolters Kluwer, (2021), 161–213. |
| [70] |
F. Castelletti, Bayesian model selection of Gaussian directed acyclic graph structures, Int. Stat. Rev., 88 (2020), 752–775. https://doi.org/10.1111/insr.12379 doi: 10.1111/insr.12379
|
| [71] |
G. Li, Q. Zhou, Bayesian multi-task variable selection with an application to differential DAG analysis, J. Comput. Graph Stat., 33 (2024), 35–46. https://doi.org/10.1080/10618600.2023.2252023 doi: 10.1080/10618600.2023.2252023
|
Figures(4)
George TH Ellison, Hanan Rhoma. Directed acyclic graphs as conceptual and analytical tools in applied and theoretical epidemiology: advances, setbacks and future possibilities[J]. Mathematical Biosciences and Engineering, 2025, 22(6): 1280-1306. doi: 10.3934/mbe.2025048
| Channel | Central wavelength/μm | Detection targets |
| 1 | 0.46 | aerosol, coastline |
| 2 | 0.51 | phytoplankton |
| 3 | 0.64 | vegetation, aerosol |
| 4 | 0.86 | stratus cloud |
| 5 | 1.6 | cloud top phase, snow |
| 6 | 2.3 | surface, cloud, snow |
| 7 | 3.9 | surface, wildfire |
| 8 | 6.2 | upper atmospheric water vapor, rainfall |
| 9 | 7.0 | mid-level atmospheric water vapor, rainfall |
| 10 | 7.3 | sulfur dioxide |
| 11 | 8.6 | water, sulfur dioxide, rainfall |
| 12 | 9.6 | ozone, airflow, wind |
| 13 | 10.4 | surface, cloud |
| 14 | 11.2 | surface, cloud |
| 15 | 12.4 | water, sea surface temperature |
| 16 | 13.3 | temperature, cloud altitude, cloudiness |
DownLoad:
CSV
| Algorithm | HR | KSS |
| Multi-spectral threshold method | 0.763 | 0.572 |
| RF | 0.811 | 0.65 |
| KNN | 0.753 | 0.606 |
| GMM | 0.973 | 0.958 |
| Proposed algorithm | 0.975 | 0.973 |
DownLoad:
CSV
| Channel | Central wavelength/μm | Detection targets |
| 1 | 0.46 | aerosol, coastline |
| 2 | 0.51 | phytoplankton |
| 3 | 0.64 | vegetation, aerosol |
| 4 | 0.86 | stratus cloud |
| 5 | 1.6 | cloud top phase, snow |
| 6 | 2.3 | surface, cloud, snow |
| 7 | 3.9 | surface, wildfire |
| 8 | 6.2 | upper atmospheric water vapor, rainfall |
| 9 | 7.0 | mid-level atmospheric water vapor, rainfall |
| 10 | 7.3 | sulfur dioxide |
| 11 | 8.6 | water, sulfur dioxide, rainfall |
| 12 | 9.6 | ozone, airflow, wind |
| 13 | 10.4 | surface, cloud |
| 14 | 11.2 | surface, cloud |
| 15 | 12.4 | water, sea surface temperature |
| 16 | 13.3 | temperature, cloud altitude, cloudiness |
| Algorithm | HR | KSS |
| Multi-spectral threshold method | 0.763 | 0.572 |
| RF | 0.811 | 0.65 |
| KNN | 0.753 | 0.606 |
| GMM | 0.973 | 0.958 |
| Proposed algorithm | 0.975 | 0.973 |
