CDA-SKAG: Predicting circRNA-disease associations using similarity kernel fusion and an attention-enhancing graph autoencoder

1.   Introduction

Circular RNAs (circRNAs) constitute a type of covalently closed non-coding RNAs with a loop structure. CircRNAs are resistant to exonuclease digestion and more stable than linear RNAs ^[1]. Studies have shown that circRNAs have various biological functions, such as regulating the expression of miRNA target genes and affecting protein interactions ^[2,3,4]. The mutations or dysfunctions of circRNAs can lead to the development of various diseases ^[5]. Liang et al. ^[6] found that breast cancer proliferation and progression can be promoted by circCDYL binding of miR-1275-ATG7, suggesting that circCDYL can be a potential molecule for predicting prognosis and treatment response in breast cancer patients. Zhang et al. ^[7] found that circ_0005015 is significantly upregulated in the fibrovascular membranes of diabetic retinopathy patients, suggesting that circ_0005015 is one of the candidate biomarkers for monitoring diabetic retinopathy. With the development of circRNA research, a variety of circRNAs have been identified as being importantly associated with the generation and development of complex diseases such as gastric cancer, leukemia and diabetes ^[8,9,10,11]. Therefore, predicting potential circRNA–disease associations is useful for exploring the pathogenesis of complex diseases and identifying additional therapeutic targets and biomarkers to aid disease treatment.

The multi-source similarity fusion algorithm can achieve information complementation between circRNAs similarities and diseases similarities and enrich the features between them, which is important for enhancing the ability to predict circRNA–disease associations ^[12]. To achieve the fusion of multi-source similarity data, Deepthi and Jereesh ^[13,14] used an information-filling algorithm to integrate circRNA functional similarity, circRNA Gaussian interaction profile kernel similarity, disease semantic similarity and disease Gaussian interaction profile kernel similarity. The information-filling algorithm replaces missing values with the similarity values of the corresponding nodes in the other remaining similarity matrices for the prediction of circRNA–disease associations. Ma et al. ^[15] used a linear weighting algorithm to fuse multiple sources of similarity data to predict circRNA–disease associations. Studies have shown that retaining non-linear information of multi-source similarity data facilitates the accuracy of prediction models ^[16]. However, both the information-filling algorithm and linear weighting algorithm integrate circRNA and disease similarity data in the manner of linear operations. They ignore the information differences existing in circRNA and disease similarity data. Zheng et al. ^[17] quantified the non-linear relationship of circRNAs by using chaos game representation and demonstrated that capturing the non-linear relationship in data contributed to the prediction of circRNA–disease associations. The similarity kernel fusion algorithm can effectively capture the information differences between circRNAs similarity and diseases similarity by calculating both the neighbor information matrix and the iterative similarity matrix; it then obtains the complex non-linear information effectively ^[18]. Therefore, we introduce the similarity kernel fusion algorithm to integrate multi-source circRNA and disease similarity data and capture the non-linear information of circRNA and disease. The similarity kernel fusion algorithm can construct more reliable integrated similarity information and construct the circRNA information space and disease information space.

A graph autoencoder (GAE) aggregates neighbor node features to obtain topological information from the graph, and it has shown good performance in predicting circRNA–disease associations. Li et al. ^[19] proposed GGAECDA for circRNA–disease associations prediction. GGAECDA obtained low-dimensional representations of multi-source similarity data through the use of a graph attention network and random walk with restart algorithm; they used a GAE to extract feature representations of circRNAs and diseases. Wu et al. ^[20] proposed a deep learning model based on GAE and matrix complementation for the prediction of lncRNA–disease associations. In the above-mentioned model, the graph convolutional network (GCN) layer of the GAE aggregates neighbor information via the adjacency matrix, and it updates the node features to the sum of all neighbor nodes features ^[21]. However, since all neighbor nodes are given the same weight, important feature information of the higher-similarity neighbor nodes cannot obtain more attention, which restricts the GAE's ability to optimize the feature representation of circRNAs and diseases ^[22,23]. Therefore, we have embedded an attention-enhancing layer in the GAE to increase the attention of the GAE to high-similarity neighbor features. It enables the adaptive optimization of features and enhances the feature representation capability of the prediction model.

The confirmed human circRNA–disease associations in the CircR2disease dataset are only 650, while there are 50,830 unlabeled samples ^[13]. Training the model with all unlabeled samples as negative samples may lead to a bias in the prediction results toward classes with a higher number of samples. It will impact the generalization capability of the model and even cause prediction failure ^[24,25]. To address the imbalance of the samples, Deepthi and Jereesh ^[13] balanced the positive and negative samples by randomly sampling the negative samples and predicted circRNA–disease associations by using a random forest classifier. Zeng et al. ^[26] proposed the positive-unlabeled strategy, which uses all positive samples and a subset of unlabeled samples to train the classifier together, training the classifier multiple times to find more reliable negative samples. The above methods improve the recognition rate of positive samples by balancing the number of positive and negative samples. However, they only select the same number of negative samples as positive samples in the full set of negative samples, which cannot take full advantage of all of the data in the dataset; and, their performance relies on the random sampling results. Cost-sensitive neural networks adjust the ratio of majority class to minority class weights in the loss function by re-weighting, instead of balancing the positive and negative samples via random sampling, solving the problem of not fully using all sample data. It has been applied successfully to disease gene identification, compound-protein interactions and so on ^[27,28,29]. Thus, we introduce a cost-sensitive neural network to address the classification bias of positive and negative samples caused by class imbalance, adjust the weight ratio between samples and improve the generalization capability of the prediction model ^[30].

Based on the above problems, we propose a deep learning model, CDA-SKAG, with similarity kernel fusion and an attention-enhancing GAE to predict circRNA–disease associations. The framework is shown in Figure 1. In this model, we take circRNA functional similarity, circRNA Gaussian interaction profile kernel similarity, disease semantic similarity and disease Gaussian interaction profile kernel similarity as input. To construct the circRNA information space and disease information space, we introduce the similarity kernel fusion algorithm to integrate multi-source similarity data and obtain non-linear information from the circRNA and disease data. An attention-enhancing GAE is used to capture the circRNA and disease features, enhance the feature weight of neighbor nodes with high similarity and adaptively optimize the circRNA and disease feature components. A cost-sensitive neural network is included to adjust the weight ratio between samples. To validate the predictive performance of CDA-SKAG, we applied a five-fold cross-validation to the CircR2disease dataset and performed case studies on lung and cervical cancers. The experimental results show that the predictive performance of CDA-SKAG is superior to existing circRNA–disease association prediction models. Our model, CDA-SKAG, obtains complex association mechanisms between the circRNA and disease and can be used as an effective tool to assist in predicting circRNA–disease associations.

2.   Materials and methods

2.1.   Human circRNA–disease associations

We collected and downloaded experimentally validated circRNA–disease association data from the CircR2Disease database, which contains 661 circRNAs, 100 diseases and 739 validated circRNA–disease associations ^[31]. We retained circRNA-disease samples associated with humans and removed redundant entries, resulting in a dataset containing 585 circRNAs, 88 diseases and 650 circRNA–disease associations (hereafter referred to as the circRNA–disease dataset). Let C = {${c}_{1}$, ${c}_{2}$, …, ${c}_{m}$} and D = {${d}_{1}$, ${d}_{1}$, …, ${d}_{n}$} be the set of m circRNAs and n diseases in the dataset, respectively. We constructed a binary matrix ${Y\in R}^{m*n}$ of circRNA–disease interactions, where the circRNA–disease association value $Y\left(i, j\right)$ = 1 if the association between circRNA i and disease j has been experimentally confirmed, and the value $Y\left(i, j\right)$ = 0 if the association between i and j is unknown or unverified.

2.2.   Multi-source similarity data

In this part, we calculate the disease semantic similarity, circRNA functional similarity, circRNA Gaussian interaction profile kernel similarity and disease Gaussian interaction profile kernel similarity as input for the CDA-SKAG model. The dimensions of similarities are shown in Table 1.

2.2.1.   Disease semantic similarity

Disease semantic similarity is calculated by using the hierarchical ontology method, which calculates the degree of disease similarity based on their relative positional relationships in a directed acyclic graph (DAG) ^[32]. Based on the description information of the Mesh database for disease relationships, we constructed a DAG to describe the relationships of all diseases. The DAG of disease D includes the ancestor nodes of D and D itself, as well as all direct edges from parent-to-child nodes ^[32]. According to Wang's method for calculating semantic similarity scores for diseases, we can calculate the semantic similarity scores between each pair of diseases based on their DAGs ^[33,34]. The detailed formula is as follows:

where ${N}_{{d}_{i}}$ and ${N}_{{d}_{j}}$ denote the ancestral diseases in the graph for disease ${d}_{i}$ and disease ${d}_{j}$, respectively. ${S}_{{d}_{i}}\left(t\right)$ denotes the semantic value of disease $t\in {N}_{{d}_{i}}$ compared to disease ${d}_{i}$, and ${S}_{{d}_{j}}\left(t\right)$ denotes the semantic value of disease $t\in {N}_{{d}_{j}}$ compared to disease ${d}_{j}$. The semantic value of disease d, ${S}_{d}\left(t\right)$, is defined as follows:

2.2.2.   CircRNA functional similarity

CircRNA functional similarity is measured based on the assumption, whereby the higher semantic similarity of the disease group two circRNAs share, the more functionally similar the two circRNAs are ^[34]. We can calculate the functional similarity of each circRNA pair based on disease semantic similarity according to the method of Deepthi and Jereesh ^[13], assuming that ${D}_{i}$ and ${D}_{j}$ are the disease groups associated with circRNAs ${c}_{i}$ and ${c}_{j}$, respectively. The circRNA functional similarity matrix can be calculated based on Eqs (3) and (4):

$S\left({d}_{q}, {D}_{j}\right)$ represents the calculated similarity with disease ${d}_{q}$ and disease group ${D}_{j}$:

$SD\left({d}_{q}, {d}_{s}\right)$ represents the semantic similarity of ${d}_{q}$ and ${d}_{s}$, where disease ${d}_{q}$ is related to circRNA ${c}_{i}$ and disease group ${D}_{j}$ is related to circRNA ${c}_{j}$. The semantic similarity between diseases and disease groups is calculated by considering disease ontology terms as described in Section 2.2.1.

2.2.3.   Gaussian interaction profile kernel similarity between circRNA and disease

By obtaining topological information, Gaussian interaction profile kernel similarity is calculated based on the interactions in the circRNA–disease interaction network between nodes ^[35]. For circRNA ${c}_{i}$, the $R\left({c}_{i}\right)$ value is defined as the ith row of the circRNA–disease association matrix Y. The Gaussian interaction profile kernel similarity between each pair ${c}_{i}$ and ${c}_{j}$ is calculated as described by Eqs (5) and (6):

where $CGS\left({c}_{i}, {c}_{j}\right)$ denotes the Gaussian interaction profile kernel similarity between ${c}_{i}$ and ${c}_{j}$. $R\left({c}_{i}\right)$ and $R\left({c}_{j}\right)$ respectively denote the ith and jth rows of the association matrix Y; λ is used to control the bandwidth, which denotes the regularized Gaussian interaction profile kernel similarity coefficient constructed based on ${N}_{c}$ (the number of circRNAs).

Similarly, the disease Gaussian interaction profile kernel similarity $DGS\left({d}_{i}, {d}_{j}\right)$ between the two diseases ${d}_{i}$ and ${d}_{j}$ can be calculated according to Eq (7) and (8), as follows:

2.3.   CDA-SKAG architecture

In our work, we have applied the circRNA functional similarity, circRNA Gaussian interaction profile kernel similarity, disease semantic similarity, disease Gaussian interaction profile kernel similarity and circRNA–disease associations as the input of CDA-SKAG and used CDA-SKAG to learn the features of the circRNAs and diseases. Here, we introduce the similarity kernel fusion algorithm used to integrate the multi-source similarity data; it is purposed to obtain the non-linear information from the circRNA data and disease data. An attention-enhancing GAE is used to acquire the circRNA features and disease features. A cost-sensitive neural network has been introduced to adjust the weight ratio between samples and enhance the generalization capability of the model. The framework of our model, CDA-SKAG, is illustrated in Figure 1.

2.3.1.   Similarity kernel fusion

In this part, we denote the circRNA functional similarity and circRNA Gaussian interaction profile kernel similarity by ${S}_{c, m}\left(m = \mathrm{1, 2}\right)$, and ${S}_{d, n}\left(n = \mathrm{1, 2}\right)$ denotes the disease semantic similarity and disease Gaussian interaction profile kernel similarity; all four similarities are entered into our model in matrix form. The similarity kernel fusion algorithm fuses the four similarities non-linearly, constructing the circRNA integration similarity matrix ${S}_{c}^{\mathrm{*}}\in {R}^{c*c}$ and the disease integration similarity matrix ${S}_{d}^{\mathrm{*}}\in {R}^{d*d}$. We illustrate the calculation of the similarity kernel fusion algorithm by using the calculation of circRNA integration similarity as a case study, and it is implemented as follows.

First, each circRNA similarity matrix is normalized by using Eq (9).

${NS}_{c, m}\left({c}_{i}, {c}_{j}\right)$ denotes the circRNA normalized similarity matrix that satisfies ${\sum }_{{c}_{k}\in C}{NS}_{c, m}\left({c}_{k}, {c}_{j}\right) = 1$.

Then, we calculate the contribution of neighbor information for each node in the circRNA similarity matrix by using Eq (10); unrelated nodes are assigned zero to obtain ${F}_{c, m}$.

where ${F}_{c, m}$ is a sparse matrix that satisfies ${\sum }_{{c}_{j}\in C}{F}_{c, m}\left({c}_{k}, {c}_{j}\right) = 1$, ${N}_{i}$ is the set of all neighbors of ${c}_{i}$.

We use Eq (11) to calculate the two circRNA similarity matrices (m = 1, 2) separately after t iterations.

${SC}_{c, m}^{t+1}$ is the state matrix of the mth circRNA similarity matrix after $t+1$ iterations, and ${SC}_{c, r}^{0}$ denotes the initial state of ${SC}_{c, r}$. The transfer probability α is distributed between (0, 1), and the value of α is taken as 0.1.

After Step $t+1$, we use Eq (12) to calculate the circRNA integration similarity matrix:

In addition, to eliminate the noise in the similarity matrix ${S}_{c}$, we define the following weight matrix ${w}_{c}$ and calculate the fused circRNA similarity matrix by using Eq (14).

Similarly, we normalize the disease matrices to ${NS}_{d, n}\left({d}_{i}, {d}_{j}\right)$ and construct the sparse matrix ${F}_{d, n}\left({d}_{i}, {d}_{j}\right)$ for each disease matrix; the two similarity matrices for the diseases can be calculated as ${SD}_{d, n}^{t+1}\left(n = \mathrm{1, 2}\right)$, respectively. After $t+1$ iterations, we fuse the two disease similarity matrices as ${S}_{d}$ and multiply with the weight matrix ${w}_{d}$. We obtain the disease integration similarity, denoted by ${S}_{d}^{\mathrm{*}}\in {R}^{d*d}$.

2.3.2.   Attention-enhancing GAE

In this part, we design an attention-enhancing GAE for extracting circRNA and disease features. The encoder acquires the embedding feature representations of circRNAs and diseases; the attention-enhancing layer enhances the feature weight of high-similarity neighbors and obtains optimized embedding features; the decoder reconstructs the circRNA–disease association matrix based on the optimized embedding features. It is implemented as follows (extracting circRNA features as an example).

For this model, we use a one-layer GCN as an encoder to extract similarity information from the circRNA and disease information space into the embedding features. For the circRNA information space, we obtain the embedding feature ${E}_{c}$ of circRNAs via the encoder:

where $Y$ is the circRNA–disease association matrix, ${S}_{C}\in {R}^{nc*nc}$ denotes the circRNA integration similarity matrix, ${A}_{c}$ denotes the regularized circRNA similarity matrix, the normalization is calculated as shown in Eq (16) and ${D}_{c}\in {R}^{nc*nc}$ is the degree matrix of the similarity matrix ${S}_{C}$. ${W}^{\left(i\right)}$ denotes the weight matrix of the ith neural network layer. The activation function in the encoder is $tanh\left(\cdot \right)$.

The attention-enhancing layer optimizes the feature representation of node embeddings by calculating the important differences of different neighboring nodes through the use of an attention mechanism ^[36]. The attention-enhancing layer reinforces the feature representation between highly similar circRNA nodes (Figure 2); moreover, it enables smooth updates of the parameters of the embedding features. We define the loss function ${L}_{Att}$ for the attention-enhancing layer in Eq (18), following Gao et al. ^[37].

where ${E}_{i}\in {E}_{c}$ denotes the initial embedding features of node i obtained from the encoder and ${H}_{i}$ denotes the updated embedding features of node i in the attention-enhancing layer. In addition, λ denotes the attention score and ${\lambda }_{ij}$ measures the weight of neighbor node j to node i. ${N}_{i}$ is the set of all neighbor nodes of node i. The first term in Eq (17) is used for the smoothing update of the embedding features of node i, and the second term makes ${H}_{i}$ of node i similar to ${H}_{j}$ of neighbor node j. Parameters β and γ are used to balance the weighting factors of the first and second terms on the effect of the embedding features. The embedding features ${H}_{i}$ are updated according to the following rules:

The initial value of the embedding feature ${H}_{i}^{\left(1\right)}$ is set to ${E}_{i}$ and ${H}_{i}^{\left(k\right)}$ denotes the embedding updated in the kth iteration.

The attention weight between node i and node j is calculated as shown in Eq (21).

$Attention\left(\cdot \right)$ denotes the single-layer feed-forward network for calculating the attention values and ${W}_{t}$ denotes the trainable matrix of the hidden layer. We set ${H}_{i}$ = ${H}_{i}^{\left(k\right)}$ as the final representation of node i. The value of K was set to 2 in our experiments. The attention-enhancing layer concentrates on extracting features from the nodes of highly similar neighbors so that, as the number of iterations K in each layer increases, nodes will gain more and more information from their more similar neighbors ^[38].

The decoder consists of a one-layer GCN that reconstructs the prediction matrix with the embedding features obtained from the attention-enhancing layer. The feature embeds ${H}_{i}$ from the attention-enhancing layer is combined into a circRNA feature matrix ${H}_{c}$ and a disease feature matrix ${H}_{d}$, where ${H}_{c}\in {R}^{nc*nc}$ and ${H}_{d}\in {R}^{nd*nd}$, respectively. We feed the embedding feature ${H}_{c}$ into the decoder and reconstruct the circRNA–disease association score matrix ${D}_{c}\in {R}^{nc*nd}$. ${D}_{c}$ is the prediction of the attention-enhancing GAE based on the circRNA features. The score $\left({c}_{i}, {d}_{j}\right)$ of all items in the association matrix represents the probability of association between each circRNA $c\left(i\right)$ and each disease $d\left(j\right)$. The formula is as follows.

${W}^{\left(i\right)}$ denotes the weight matrix of the ith neural network layer. The activation function of the decoder is $sigmoid\left(\cdot \right)$.

Finally, we fuse the circRNA association matrix and the disease association matrix to obtain the circRNA–disease association prediction results.

F is the final prediction matrix, where δ $\in$ (0, 1) determines the scaling relationship between circRNA features and disease features.

2.3.3.   Cost-sensitive neural network

There are only 650 validated positive samples in the circRNA–disease dataset, accounting for 1.26% of all samples ^[15]. Class imbalance leads to differences in the recognition rates of positive and negative samples. Therefore, we have introduced a cost-sensitive neural network to offset the classification bias caused by class imbalance ^[30]; the idea is to weaken the impact of sample size differences on classification by increasing the misclassification cost of positive samples. When the model predicts the circRNA–disease associations, the cost-sensitive neural network increases the weight value of positive samples so that the loss value of positive sample misclassification is higher than the loss value of negative samples. It is possible to improve the recognition rate of positive samples in this way. Therefore, we modify the loss function as follows:

where ${W}_{ij}$ denotes the learnable label weight matrix with the same dimensions as the association matrix. The values of the weight matrix are updated by the feed-forward network.

3.   Results and discussion

3.1.   Experimental setup and evaluation criteria

In this study, we used five-fold cross-validation to evaluate the performance of CDA-SKAG. Five statistical metrics were used to evaluate the predictive performance of the proposed model and other comparative models. These metrics include the accuracy (Acc), precision, recall, F1 score and Matthew's correlation coefficient (MCC), which are defined as follows:

where TP and TN represent the number of correctly identified results in positive and negative samples, respectively. FP and FN represent the number of incorrectly identified results in positive and negative samples, respectively. Acc represents the proportion of the number of positive and negative samples correctly identified by the model relative to the size of all samples. Precision represents the probability that, of all of the samples with a positive prediction, the true class is also positive. Recall represents the proportion of all positive samples that are correctly predicted by the model, and it is used to assess the ability of the model to identify circRNA–disease association pairs. The circRNA–disease dataset has a problem of sample class imbalance, so we can calculate the F1 score and MCC to measure the accuracy of the predictive model. The F1 score and MCC combine the precision and recall of the model, as well as provide a more accurate evaluation of the predictive performance. In addition, we used the AUC (area under receiver operating characteristics (ROC)) curve and AUPR (area under precision-recall (PR)) curve to evaluate the predictive performance of our model.

3.2.   CDA-SKAG parameter selection

In the circRNA–disease association prediction problem, the model parameter settings can affect model predictive performance. In this study, the CDA-SKAG model was implemented using Pytorch 1.8.0. In the process of training the model, we used a cost-sensitive neural network as the loss function and Adam's algorithm as the optimizer to optimize the object function ^[39]. We set the learning rate to 0.001 and the epoch to 1200 to ensure the stability of the model training process. We performed five-fold cross-validation on the circRNA-disease dataset using CDA-SKAG and plotted the loss curves, the relevant experimental results are shown in Figure A1 of supplementary materials. We set the dropout to 0.5 and the hyperparameter of weight decay to 0.00001 to prevent overfitting of CDA-SKAG. CDA-SKAG consists of similarity kernel fusion, an attention-enhancing GAE and a cost-sensitive neural network, so we will discuss the parameters involved in each component separately in this section.

3.2.1.   Similarity kernel fusion parameter selection

Convergence is an important factor affecting the fusion of multi-source circRNA and disease similarity data. We concentrate on the value of the number of iterations t. We analyzed the number of iterations to converge for each type of circRNA and disease similarity data. Inspired by the study of Fan et al. ^[40], we chose to denote the relative errors of circRNA similarities and disease similarities during the iterative process as ${EC}_{t}$ and ${ED}_{t}$, respectively. The number of iterations ranged from 1 to 11 and the step size was 1; ${EC}_{t}$ and ${ED}_{t}$ were calculated after each iteration. The convergence process is depicted in Figure 3.

The results show that the convergence process of the similarity kernel fusion algorithm was fast, with the ${EC}_{t}$ values reaching 10⁻¹⁰ after six iterations and ${ED}_{t}$ values reaching 10⁻¹⁰ after ten iterations. The convergence process for two types of circRNA similarities both yielded relative errors of less than 0.01 at the second iteration. The relative errors of circRNA Gaussian interaction profile kernel similarity and circRNA functional similarity reached 10⁻¹⁰ during the sixth and seventh iterations, respectively. Two types of disease similarities obtained a relative error of less than 0.01 at the fifth iteration. The relative errors of disease Gaussian interaction profile kernel similarity and disease semantic similarity reached 10⁻¹⁰ during the 11th iteration. Fan et al. ^[40] concluded that the relative error of multi-source circRNA and disease similarity data of less than 10⁻¹⁰ could be identified as convergence. Therefore, we selected eleven as the number of iterations for circRNA similarity fusion and disease similarity fusion.

3.2.2.   Selection of the number of GCN layers

To explore the optimal GCN layer settings for the CDA-SKAG model in this study, we performed a five-fold cross-validation experiment based on the circRNA–disease dataset. Considering the amount of data, when the number of GCN layers exceeded three, it would trigger over-smoothing and impact the predictive performance of CDA-SKAG. Therefore, we set the GCN layers of the GAE to be 1, 2 and 3 layers, respectively, and then recorded the AUC, AUPR, F1 score, Acc, precision, recall and MCC under those different layer setting conditions. The experimental results are shown in Table 2.

As shown in Table 2, we obtained the highest results when the GCN layer setting was 1. The AUC, AUPR, F1 score, Acc, precision, recall and MCC values all gradually decreased as the number of GCN layers increased. Among them, the AUPR, F1 score and MCC decreased significantly. Over-smoothing appeared when the number of GCN layers exceeded the range of applicability of the dataset ^[41]. Over-smoothing manifests as the inability of the graph neural network to distinguish the features of different classes of nodes ^[42]. Too many GCN layers led to over-smoothing of the model, and the predictive model was unable to correctly distinguish between positive and negative samples, reducing the predictive performance of the model. Increasing the number of network layers caused the network to incorrectly identify the nodes' features and reduce the model's predictive capability. CDA-SKAG learned the embeddings, aiming to reconstruct the association matrix to predict the associations of circRNA–disease. Therefore, we selected 1 as the optimal number of layers for the GCN.

3.2.3.   Iterative number of attention-enhancing layers

To evaluate the impact of the number of iterations of the attention-enhancing layer, we set different iterations K and experimented on the circRNA–disease dataset. K = 1 means that the attention-enhancing layers only calculate the weight values of the node embedding feature without iterating again; K = 2, K = 3 and K = 4 means 1, 2 and 3 iterations, respectively, after calculating the weight of the node embedding feature. We set k to 1, 2, 3 and 4 in the experiment, and the results are shown in Table 3.

As we can see in Table 3, with the increase in the number of iterations, the metrics of AUC, F1 score, Acc, precision, recall and MCC increased at first, gradually decreasing after K > 2. The best results were obtained at K = 2. K = 1 indicates that it has calculated the weight value of the node embedding feature and has not reassigned the node embedding feature to optimize the feature combination; thus, the performance of the predictive model still has the possibility to be improved. When the number of iterations K was more than 2, a larger number of iterations generated noise and the node embedding feature was misrepresented during the iterations, impacting the effectiveness of the decoder in reconstructing the circRNA–disease association matrix. Therefore, we chose K = 2 as the optimal number of iterations of the attention-enhancing layer.

3.2.4.   Parameter selection of the cost-sensitive neural network

We experimented with a five-fold cross-validation of the positive sample weight to explore the optimal parameter selection for the cost-sensitive neural network. We fixed the weight value of negative samples at 1 and set the positive samples with weight values ranging from 1 to 12 respectively. We recorded the corresponding MCC as the evaluation metric. The results are shown in Figure 4.

As shown in Figure 4, the cost-sensitive neural network facilitates balancing the classification bias caused by the number of positive and negative samples. When the positive sample weight was taken as 2, the MCC value was significantly better than the result when the positive sample weight was taken as 1. It is shown that the cost-sensitive neural network can improve the classification accuracy of predictive models for unbalanced datasets and help the models to identify potential circRNA–disease associations more effectively. When the positive sample weight was taken as 5, the model achieved an optimal result of 85.25% for the evaluation metric MCC. As the positive sample weight value exceeded 5, it led to misclassification of some of the negative samples as positive samples and the false positive rate increased, causing the MCC trend to decrease. Therefore, we chose 5 as the positive sample weight value for the subsequent experiments.

3.3.   Ablation experiments

3.3.1.   Performance comparison for multi-source similarity fusion strategies

In this section, our purpose is to evaluate whether similarity kernel fusion algorithms can integrate circRNA and disease multi-source similarity more effectively. The similarity kernel fusion, information-filling and linear weighting algorithms were each coupled with an attention-enhancing GAE. We conducted a five-fold cross-validation on the circRNA-disease dataset and recorded the AUC, AUPR, F1 score, Acc, precision, recall and MCC values of different fusion strategies; the results are shown in Table 4.

As can be seen in Table 4, the similarity kernel fusion algorithm achieved the best performance, with higher results for the AUPR, F1 score, precision, recall and MCC than the information-filling and linear weighting algorithms. The similarity kernel fusion algorithm takes into account the influence of neighbor information on node similarity, fuses the non-linear information from different circRNA similarities and disease similarities and achieved a higher accuracy of sample classification compared to the above two linear fusion algorithms. In summary, we have introduced the similarity kernel fusion algorithm to fuse multi-source similarity data non-linearly and provide more accurate feature input for subsequent prediction tasks, helping the model to predict potential circRNA–disease associations more accurately.

3.3.2.   Model structural ablation experiment

To validate the effect of each module in CDA-SKAG for circRNA–disease association prediction, we designed model structural ablation experiments. In our experiments, we composed a baseline model by using a GAE combined with the cross-entropy function to predict circRNA–disease associations. We took the integrated similarity derived from the similarity kernel fusion algorithm as input to the baseline model. The GAE was replaced in the experiments with an attention-enhancing GAE (Att-GAE in Figure 5), and the cross-entropy loss function in the model was replaced with a cost-sensitive neural network to verify the contribution of each module. Each of the model compositions was subjected to a five-fold cross-validation on the circRNA–disease dataset; the results are shown in Figure 5.

The different colors in Figure 5 indicate the different model compositions. Orange represents the baseline model, consists of GAE and cross entropy. Green represents the model that replace GAE with Att-GAE. Comparing the performance indicators of the orange and green models, the figure shows that the F1 score, recall and MCC were improved, indicating that the attention-enhancing GAE has better feature extraction capabilities and can correctly identify more circRNA–disease associations. By comparing the performance indicators of the green and blue models, we can see that the AUPR, F1 score, recall and MCC increased significantly. The F1 score and MCC measured the classification accuracy of the model in the imbalanced dataset, indicating that the cost-sensitive neural network could offset the classification bias caused by class imbalance and improve the accuracy of the model in terms of its ability to identify positive and negative samples.

3.4.   Evaluation of CDA-SKAG predictive capability

To evaluate the predictive performance of the models, we compared CDA-SKAG with other existing circRNA–disease association prediction models. We chose six comparison models: DMCCDA ^[43], DMFCDA ^[44], GMNN2CD ^[45], CRPGCN ^[15], LLCDC ^[46] and GAMCLDA ^[20]. Among these models, DMCCDA and GAMCLDA are matrix-completion-based non-coding RNA–disease association prediction models; DMCCDA uses a dual matrix-completion algorithm and GAMCLDA uses a GAE to predict circRNA–disease associations. DMFCDA used a deep neural network to implement a matrix decomposition algorithm for learning the deep features of the circRNA and disease, and for predicting potential associations. LLCDC obtains the reconstructed similarity through locality-constrained linear coding, and it uses label propagation methods to predict the circRNA–disease association matrix. GMNN2CD uses information-filling algorithms as a multi-source similarity fusion method, incorporating graph Markov neural network-extracted features, to predict circRNA–disease associations. CRPGCN uses a linear weighting algorithm to fuse multiple sources of similarity data, and it is combined with a GCN to extract features to predict circRNA–disease associations. We conducted a five-fold cross-validation experiment on the circRNA–disease dataset to compare our model's predictive performance with that of the comparison model; the results are shown in Table 5.

As can be seen in Table 5 and Figure 6, CDA-SKAG achieved the highest AUC, AUPR, F1 score, Acc and MCC values. We notice that DMCCDA, DMFCDA and GAMCLDA obtained high recall values. However, the precision values were lower, indicating that these models identified a large number of unconfirmed associations as positive samples in the prediction results. Although more positive samples were identified, a large number of negative samples were misclassified. It suggests that these models failed to obtain the features of negative samples accurately, resulting in a failure to identify circRNA–disease associations accurately. LLCDC uses locality-constrained linear coding, which can retain local information and iteratively update the labels by using label propagation. CDA-SKAG achieved a higher AUPR, MCC and recall than LLCDC. CDA-SKAG not only captures the local information of the nodes, but it also obtains the non-linear features of the nodes themselves, which results in better feature extraction capability. Compared to the graph neural network-based models GMNN2CD and CRPGCN, our CDA-SKAG model obtained better results in terms of the AUC, AUPR, F1 score, Acc, recall and MCC values, whereas the precision metric of CDA-SKAG was equivalent to these two comparison models. It shows that our CDA-SKAG model can not only identify the majority of circRNA–disease associations, but it can also distinguish unrelated circRNA–disease pairs. Thus, CDA-SKAG can predict circRNA–disease associations more effectively.

3.5.   Case study

To demonstrate the effectiveness of CDA-SKAG in predicting novel associations between diseases and circRNAs, we conducted case studies on lung cancer and cervical cancer. In the case studies, we trained CDA-SKAG with known circRNA–disease associations obtained from the CircR2disease database as a dataset and ranked candidate circRNAs for the target diseases based on the prediction scores given by the models in this paper. We selected the top 15 candidate circRNAs and verified their correctness by using the authoritative database circRNA disease ^[47] or PubMed experimental literature. We have labeled the sources of validation of circRNA–disease associations in the "PMID" columns in Tables 6 and 7.

The 15 candidate circRNAs are listed in Table 6 as being associated with lung cancer. Two candidate results are included in the databases circRNA disease and CircR2Disease, and they show increased expression levels in lung cancer cells: hsa_circ_0043256 and circHIPK3. In addition, we recognized 10 circRNAs supported by the literature; refer to the list of predictions labeled "PMID" in Table 6. In summary, 12 of the 15 candidate circRNAs for lung cancer predicted by our model CDA-SKAG were confirmed to be associated with the disease.

The 15 candidate circRNAs are listed in Table 7 as being associated with cervical cancer. One candidate result is contained in the CircR2Disease database. The circRNA: hsa_circ_0000284 showed increased expression levels in cervical cancer cells. In addition, we found eight predicted circRNAs supported by the literature. Nine of the 15 candidate circRNAs predicted by CDA-SKAG for cervical cancer were confirmed to be associated with the disease. Thus, CDA-SKAG enables accurate prediction of novel associations between diseases and circRNAs based on the existing experimental results. It implies that CDA-SKAG can capture the complex structure of circRNA–disease association data and is capable of robustly inferring potential associations.

4.   Conclusions

In this paper, we have proposed CDA-SKAG for the prediction of potential circRNA–disease associations, and it is based on a similarity kernel fusion algorithm and an attention-enhancing GAE. CDA-SKAG integrates multi-source similarity data by using a similarity kernel fusion algorithm to construct the circRNA information space and disease information space. An attention-enhancing GAE is used to extract circRNA and disease features and predict potential circRNA–disease associations. A cost-sensitive neural network has been added to resolve the widespread class imbalance in circRNA–disease association datasets. By comparing the performance with existing circRNA–disease association predictive models, the results show that our CDA-SKAG model has comparable or even better predictive performance than existing association-based predictive models. Ablation experiments of the multi-source similarity fusion strategy and model architecture show that the similarity kernel fusion algorithm takes into account the non-linear information of circRNA and disease to obtain more reliable integrated similarity information; the combination of an attention-enhancing GAE and a cost-sensitive neural network can effectively process circRNA and disease features and help to predict circRNA–disease association more accurately. The current version of CDA-SKAG still has certain limitations. First, the model relies on known association data and multi-source similarity, while the variety of multi-source similarity data is limited. Second, present methods are limited by the selection of negative samples, which heavily impacts the prediction results' reliability. Finally, there is still room for improvement of the multi-source similarity fusion strategy. In the future, we can learn important studies to improve the association-based predictive model for the related fields of circRNA, such as lncRNA–miRNA interaction prediction, circRNA–miRNA association prediction and metabolite–disease association prediction ^{[48,49,50,51]}. The introduction of a variety of circRNA and disease data, such as miRNA–disease association data, circRNA–miRNA interaction data and the structural embedding feature of circRNA, will enrich the multi-source information of circRNAs and diseases ^[49,51]. With the continuous development of deep learning research and the increasing amount of data, we believe that deep learning will play a more significant role in predicting circRNA and disease association in the future.

Acknowledgments

This research was funded by the National Natural Science Foundation of China, grant number 62176177; and the Natural Science Foundation of Shanxi Province, grant number 202203021211121.

Conflict of interest

The authors declare that there is no conflict of interest.

Appendix