Analysis of data splitting on streamflow prediction using random forest

1.   Introduction

Water is an essential natural resource that sustains life, ecosystems, and society (^[1]). Due to the overexploitation of this natural resource, many parts of the world have been experiencing deteriorating situations such as rivers having little to no discharge ^[2,3,4,5], ecosystems being degraded ^[6], reduced water table levels ^[7], and decreasing lakes and wetland areas ^[8]. The hydrological time series is becoming increasingly essential for the effective distribution, management, and planning of water resources ^[9]. Furthermore, human activity and socioeconomic growth, in addition to climate change, have an impact on hydrological processes such as temperature, evaporation, and precipitation ^{[10,11,12,13]}. As a result, nonlinear and time-varying hydrological time series are a constant ^[14]. The intricate nonlinearity, high irregularity, and multi-scale irregularity of hydrological time series make forecasting an intimidating task. Hydrological time series prediction has been the subject of numerous studies ^[15], but a thorough knowledge of hydrological processes is still lacking. Particularly for complex time series, the existing forecasting methods still have low forecast accuracy. Highly precise streamflow predictions are necessary for hydrological application operations and planning on a range of time scales, including daily, weekly, and monthly. While long-term forecasting, such as weekly and monthly flow, is essential for planning hydropower generation, reservoir release scheduling, irrigation management, river sediment transport, and several others, real-time streamflow forecasting, or hourly and daily flow, is crucial for flood control and mitigation ^[16,17,18].

In the past few decades, data-driven machine-learning approaches have included support vector machines (SVMs), neural networks (NNs) ^[19,20,21], fuzzy logic ^[22,23,24], and hybrid ML techniques, which include a blend of deep learning algorithms, the nonlinear autoregressive network with exogenous inputs, multilayer perceptron, and random forest ^[25]. Additionally, grey wolf optimization (GWO)-integrated AI models ^[26] have drawn a lot of attention for streamflow forecasting applications. For instance, a study investigated and compared the effect of different dataset ratios in the prediction of groundwater using NN methods, observing that dataset size and model choice had significant effects on output ^[27]. Researchers working with ML algorithms have used training/testing ratios of 70/30, 80/20, and 90/10 for producing datasets ^[28,29,30]. ML-based models have been established and applied effectively and efficiently to solve a lot of real-world challenges ^{[31,32,33,34,35]}. The primary benefit of ML is its ability to subjectively analyze an infinite quantity of data and produce accurate results and evaluations. However, the quality of the data as well as how it is utilized determines the outcome ^[36]. Thus, in order to choose an appropriate data splitting for improved ML-based modeling, it is necessary to evaluate the impact of data splitting on the execution of soft computing models. This study investigates how data splitting affects the performance of random forest (RF) in predicting streamflow. The primary goal of the research was to develop hydrological models using monthly hydrological data and apply different training/testing ratios, namely 50/50, 60/40, 70/30, and 80/20. The objective was to analyze the impact of these ratios on the RF approach for forecasting river flow in the Kesinga sub-catchment of the Mahanadi basin.

2.   Materials and methods

2.1.   Modeling approach

2.1.1.   Random forest (RF)

RF is the ML model based on the ensemble method of learning ^[37,38,39]. RF is a highly preferred ensemble learning method in ML research and practice, having been introduced by ^[37], and successfully performing classification and regression tasks. The combination of decision trees is the basic idea underlying RF; it creates variety among the trees and minimizes overfitting by training each tree on an arbitrary subset of input characteristics and a different subset of the training data. To build an RF, a decision tree is created using a randomly selected subdivision of the training data by bootstrapping ^[40]. For each split, a selection of the input features is randomly considered, coined as the "random subspace" approach, to ensure each tree is different from the others. Adjusting the number of decision trees to be generated (Ntree) is a significant parameter to consider. Increasing the number of trees often improves the performance of RF. Since overfitting has little impact on RF, Ntree can be scaled relatively high. A subset of data can be used, known as out-of-bag (OOB) samples, for estimating the performance of the RF without needing a separate validation set. When performing RF regression, the individual trees are grown using the CART algorithm without pruning. Each tree uses different random subsets of features for splitting at every node and is grown to its maximum depth. The estimates of individual trees are combined to generate the final regression output, typically taking an average. A significant aspect of RF is the ability to determine variable importance. Variable importance is computed by evaluating how much a variable contributes to the improvement of prediction performance when it is randomly permuted. This approach can be used to evaluate the relevance of multiple features in the dataset and to select the most crucial variables for a particular task ^[41]. In conclusion, with their robustness, effectiveness, and interpretability, RFs are powerful and versatile ensemble learning models that have gained popularity. Figure 1 illustrates the working of RFs.

Figure 1.  Schematic representation of random forest.

DownLoad: Full-Size Img PowerPoint

2.2.   Methodology and dataset

To develop a model for the prediction of streamflow, a total of 169 data points were collected on a monthly basis for the years 1991–2004, from the Central Water Commission (CWC), Mahanadi and Eastern Rivers division, Bhubaneswar, Odisha. The total sample was then randomly chosen and segregated into two subsets, i.e., training and testing, with a 50/50, 60/40, 70/30, and 80/20 ratio. The models were generated using WEKA 3.9.5 software. The input characteristics were assessed to predict the aeration efficiency outcome using statistical measures like correlation coefficient (CC), mean absolute error (MAE), and root mean square error (RMSE). Figure 2 shows a schematic representation of the study site whereas, Figure 3 shows the flowchart for the current study.

Figure 2.  Study site: (a) Indian map showing the states; (b) Mahanadi Basin; (c) Kesinga sub-catchment.

DownLoad: Full-Size Img PowerPoint

Figure 3.  Flowchart of the methodology adopted in the current study.

DownLoad: Full-Size Img PowerPoint

In developing a model, choosing the input variable is the most important step. Samples of 12 distinct input data combinations were grouped, as shown in Table 2, in order to accomplish this. The number of variables that were chosen by looking at the monthly river flow is shown in the input M-1 to M-12. Models for streamflow prediction were developed using the same combinations of inputs. The details of the models are shown in Table 1.

Table 1.  Model structure with input parameter combinations.

Model	Input combinations
M-1	Zx = f (ax-1)
M-2	Zx = f (ax-1, ax-2)
M-3	Zx = f (ax-1, ax-2, ax-3)
M-4	Zx = f (ax-1, ax-2, ax-3, ax-4)
M-5	Zx = f (ax-1, ax-2, ax-3, ax-4, ax-5)
M-6	Zx = f (ax-1, ax-2, ax-3, ax-4, ax-5, ax-6)
M-7	Zx = f (ax-1, ax-2, ax-3, ax-4, ax-5, ax-6, ax-7)
M-8	Zx = f (ax-1, ax-2, ax-3, ax-4, ax-5, ax-6, ax-7, ax-8)
M-9	Zx = f (ax-1, ax-2, ax-3, ax-4, ax-5, ax-6, ax-7, ax-8, ax-9)
M-10	Zx = f (ax-1, ax-2, ax-3, ax-4, ax-5, ax-6, ax-7, ax-8, ax-9, ax-10)
M-11	Zx = f (ax-1, ax-2, ax-3, ax-4, ax-5, ax-6, ax-7, ax-8, ax-9, ax-10, ax-11)
M-12	Zx = f (ax-1, ax-2, ax-3, ax-4, ax-5, ax-6, ax-7, ax-8, ax-9, ax-10, ax-11, ax-12)

---

 | Show Table

DownLoad: CSV

2.3.   Performance evaluation parameters

The precision of the models utilized for E₂₀ at different jet geometries in an open channel was assessed using three statistical measurements: correlation coefficient (CC), mean absolute error (MAE), and root mean square error (RMSE). The calculations for CC, MAE, and RMSE are outlined in Eqs (1–3):

Where o = observed values

$\underset{\_}{o}$ = average of observed values

p = predicted values

$\underset{\_}{p}$ = average of predicted values

N = number of observations

3.   Results

3.1.   50/50 ratio

There were two user-defined parameters in RF regression-based models. First, there are k trees to be produced, and then there are m variables to be used in order to create a tree. It was discovered that 1 was the ideal value of k to obtain the optimal model, the M-11. Table 2 indicates that the M-11 RF model outperformed the other models. The CC, MAE, and RMSE performance values for the 50/50 separation method are displayed in the table. The table shows that M-11 has the lowest errors, the maximum CC value of 0.737, MAE = 94.468, and RMSE = 137.116. Using the M-11 RF model, Figure 4a, b shows the variation between the experimental and anticipated values of streamflow for the Kesinga basin.

Table 2.  Performance evaluation parameters using different data ratios.

Separation		Training			Testing
	Models	CC	MAE	RMSE	CC	MAE	RMSE
50/50	M-1	0.956	44.433	72.055	0.731	88.207	135.771
	M-2	0.974	35.728	56.577	0.679	95.683	147.251
	M-3	0.978	31.683	52.374	0.659	99.981	152.112
	M-4	0.976	34.474	55.117	0.639	108.467	158.318
	M-5	0.973	36.592	57.664	0.663	109.138	153.918
	M-6	0.976	36.581	56.497	0.658	110.903	154.807
	M-7	0.977	35.824	56.448	0.660	112.143	155.481
	M-8	0.977	35.319	56.607	0.661	113.066	155.373
	M-9	0.973	36.727	58.628	0.684	109.428	151.055
	M-10	0.976	35.643	56.626	0.725	100.850	141.984
	M-11	0.980	32.744	51.772	0.737	94.468	137.116
	M-12	0.984	28.970	46.125	0.708	90.424	141.166
60/40	M-1	0.963	34.482	60.529	0.653	87.942	161.539
	M-2	0.972	31.465	55.33	0.736	77.978	141.503
	M-3	0.975	32.451	53.434	0.798	73.571	125.408
	M-4	0.974	33.030	53.853	0.809	70.598	122.587
	M-5	0.976	33.808	53.628	0.799	74.699	125.404
	M-6	0.976	34.342	55.438	0.804	76.240	124.710
	M-7	0.976	35.895	56.491	0.805	76.619	124.995
	M-8	0.976	34.553	55.873	0.789	79.412	128.837
	M-9	0.977	34.535	56.214	0.791	79.737	128.685
	M-10	0.977	35.143	57.584	0.767	82.673	134.468
	M-11	0.978	32.779	55.071	0.822	72.491	119.448
	M-12	0.976	30.904	53.845	0.844	69.367	111.976
70/30	M-1	0.958	38.752	66.829	0.634	94.160	168.215
	M-2	0.975	30.038	53.587	0.745	76.024	144.438
	M-3	0.978	29.235	51.378	0.748	76.440	138.842
	M-4	0.977	30.009	51.736	0.744	76.912	139.626
	M-5	0.979	29.430	50.074	0.738	77.396	141.009
	M-6	0.981	29.861	49.188	0.739	79.247	140.047
	M-7	0.982	32.660	51.282	0.732	83.627	141.799
	M-8	0.9812	31.882	51.640	0.757	76.758	136.167
	M-9	0.982	30.907	50.207	0.743	79.559	139.651
	M-10	0.983	29.610	49.000	0.711	83.688	147.698
	M-11	0.984	27.606	45.975	0.688	92.821	151.125
	M-12	0.983	26.557	45.066	0.714	84.671	145.412
80-20	M-1	0.968	31.072	56.058	0.467	106.620	196.171
	M-2	0.976	27.919	50.385	0.606	88.491	172.731
	M-3	0.978	27.755	50.202	0.681	87.882	157.734
	M-4	0.980	27.790	49.072	0.688	85.108	154.887
	M-5	0.979	28.926	50.378	0.666	85.769	159.080
	M-6	0.982	28.887	49.219	0.675	85.142	156.360
	M-7	0.980	27.790	49.072	0.688	85.108	154.887
	M-8	0.981	30.202	49.489	0.617	90.404	166.448
	M-9	0.981	29.153	48.553	0.635	92.515	162.900
	M-10	0.981	27.142	48.077	0.623	94.394	165.182
	M-11	0.981	26.395	48.111	0.679	91.854	154.940
	M-12	0.979	26.924	48.047	0.708	95.575	150.163

---

 | Show Table

DownLoad: CSV

Figure 4.  M-11 RF model prediction for 50/50 data separation during (a) training and (b) testing stage.

DownLoad: Full-Size Img PowerPoint

3.2.   60/40 ratio

The values in Table 2 represent the performance evaluation of M-1 to M-12 RF model based on the 60/40 ratio data separation method, showing the CC, MAE, and RMSE values. The M-12 RF model outperformed all other models with a CC value of 0.976 and 0.844, a MAE value of 30.904 and 69.367, and a RMSE value of 53.845 and 111.976 during training and testing stages, respectively. Figure 5 shows the graphical representation of actual and predicted values of M-12 RF model during both training and testing stages.

Figure 5.  M-12 RF model prediction for 60/40 data separation during (a) training and (b) testing.

DownLoad: Full-Size Img PowerPoint

3.3.   70/30 ratio

The evaluation index values given in Table 2 show that the M-8 RF model is superior when data is segregated in a 70/30 ratio. The CC value achieved by the M-8 RF model was 0.9812 in the training stage and 0.757 during the testing stage. The MAE and RMSE values during training and testing stages are 31.882, 51.640, 76.75, and 136.167, respectively. The graphical representation of actual and predicted streamflow with 70/30 data is shown in Figure 6. The input parameter to obtain the best model with 70/30 ratio is shown in Table 3.

Figure 6.  M-12 RF model prediction for 70/30 data separation during (a) training and (b) testing stage.

DownLoad: Full-Size Img PowerPoint

Table 3.  Major factors using RF model.

Separation method	Best model	Number of seeds (k) to get the optimum model
50-50	M-11	k= 1
60-40	M-12	k= 3
70-30	M-8	k= 3
80-20	M-12	k= 3

---

 | Show Table

DownLoad: CSV

3.4.   80/20 ratio

From the evaluation index values shown in Table 2, it is observed that the highest CC during the testing stage was obtained by the M-12 RF model. During the training stage, the CC value obtained was 0.979; the MAE and RMSE values were 26.924 and 48.047, respectively. The CC, MAE, and RMSE values during the testing stage were 0.708, 95.575, and 150.163, respectively. The prediction of streamflow with 80/20 data separation is shown in Figure 7a, b.

Figure 7.  M-12 RF model prediction for 80/20 data separation during (a) training and (b) testing stage.

DownLoad: Full-Size Img PowerPoint

3.5.   Comparative results of data separation methods

In the aforementioned sections, the best RF model was observed for each data segregation method. The comparison among each data segregation method is also needed to provide insightful knowledge on which training and testing datasets need to be set to get the optimum result. For the intercomparison among data separation methods, the streamflow series hydrographs are shown in Figure 8. The actual and predicted streamflow with each data separation method is shown with colored dashed lines, which indicate consistent trends and static positions of each series. It is observed that the trend followed by the 60/40 data series is consistent with the actual data series. Figure 9 shows the box plot, which also confirms that the 60/40 data segregation method is the best. The lower and higher quartiles of the box are used to represent the 25th and 75th percentile values, while the box's median is used to represent the 50th percentile values. A vertical line matching the boxes indicates variation outside of the top and bottom quartiles. In Figure 9, the box plot for the prediction stage indicates that the width of the upper and bottom end of the boxes in the 60/40 data separation method is nearly identical to that of the actual values. Figure 10 also indicates that the 60/40 data separation method gives the highest CC during the testing stage, among others.

Figure 8.  Prediction of data separation using a hydrograph for RF.

DownLoad: Full-Size Img PowerPoint

Figure 9.  Box plot for actual and predicted values using a data separation method for the testing stage.

DownLoad: Full-Size Img PowerPoint

Figure 10.  Comparison among data separation me thods based on CC values during the testing stage.

DownLoad: Full-Size Img PowerPoint

4.   Discussion

Before soft computing models, mathematical models were employed to predict streamflow. The results produced by mathematical models were quite satisfactory. The main disadvantage of using conventional methods for time series prediction is that it requires a lot of time ^[42]. Thus, research has been exploring AI-driven models to reduce the time required for time series prediction. To develop and test the AI models, the dataset needs to be segregated into training and testing datasets. This paper explores four dataset split ratios, i.e., datasets separated into training and testing datasets in a 50:50, 60:40, 70:30, and 80:20 ratio. The extreme learning machine model's (ELM) exceptional flexibility in adapting to variations in training settings and data sizes is evident. The findings demonstrated that when the testing data size falls between 50% and 25% of the total number of data observations, the ELM model requires fewer input variables. But, in certain situations, the model needs extra input features, especially when the training data comprises 80%–90% of the total dataset ^[43]. Huang F, et al ^[44] investigated the computer-assisted diagnosis of ECG using a least-square support-vector machine (LS-SVM). The data was divided into three performance measures, i.e., 50/50, 70/30, and 80/20 training and testing. The classification precision obtained was 100% for all the training-to-testing ratios. Many researchers suggested using a training/testing set ratio of 70/30 or 80/20 to create datasets for landslide susceptibility issues ^[45,46,47]. Many other researchers have investigated dataset splitting in various other fields ^{[48,49,50,51]}). A study by Kulkarni S ^[52] found that when using ML models with an 80/20 training/testing dataset, the ANN-FF model was 100% accurate in classifying lung cancer. The performance was reduced to 96% with a 70/30 data split and further to 94% as the data was split into 60/40. But, as the data was split into 50/50, the accuracy rose to 98%. The SVM accomplished an accuracy of 100% for 50/50 training and testing datasets.

The findings of the current study imply that a training/testing ratio of 60/40 was ideal for both model testing and training. The current study's findings are consistent with the research conducted by AlOmar M. K, et al ^[43,53]. It was also observed that, with an increase in the percentage of training datasets, the MAE and RMSE error values also increased.

5.   Conclusions

The present study aimed to predict the streamflow of the Kesinga basin by using dataset splitting (training/testing), namely at 50/50, 60/40, 70/30, and 80/20 ratios. The RF model was used to predict the streamflow. Validation and comparison results showed that the 60/40 split provided the highest accuracy. The highest CC (0.844) was obtained by the M-12 RF model during the testing stage with a 60/40 dataset split. The MAE and RMSE values during the testing stage acquired by the same model were 69.367 and 111.976, respectively. The 60/40 ratio was followed by the 70/30 ratio, with CC, MAE, and RMSE values of 0.7571, 76.7581, and 136.167, respectively, during the testing stage.

Use of AI tools declaration

The authors declare they have not used Artificial Intelligence (AI) tools in the creation of this article.

Acknowledgments

The authors duly acknowledge with thanks the contribution of the authors from literature whose research articles have been cited in this study. This research did not receive any specific support from public, commercial, or nonprofit funding organizations. All the information was gathered from published works that are in the public domain and properly cited. The datapoints were collected on a monthly basis for the years 1991–2004, from the Central Water Commission (CWC), Mahanadi and Eastern Rivers division, Bhubaneswar, Odisha. Diksha Puri:Conceptualization, experimentation, methodology, formal analysis, models development, writing original draft Parveen Sihag: Reviewing and editing, revision recommendations M.S. Thakur: Supervision, approval of final draft., Mohammad Jameel: proof reading editing, Aaron Chadee: English check; Hazi Azamathulla: Supervision draft Editing.

Conflict of interest

The authors declare they did not have any conflict of interest in the creation of this article.