Advancing solar energy forecasting with modified ANN and light GBM learning algorithms

Muhammad Farhan Hanif; Muhammad Sabir Naveed; Mohamed Metwaly; Jicang Si; Xiangtao Liu; Jianchun Mi; Muhammad Farhan Hanif; Muhammad Sabir Naveed; Mohamed Metwaly; Jicang Si; Xiangtao Liu; Jianchun Mi

doi:10.3934/energy.2024017

AIMS Energy

2024, Volume 12, Issue 2: 350-386. doi: 10.3934/energy.2024017

Previous Article Next Article

Research article Special Issues

Advancing solar energy forecasting with modified ANN and light GBM learning algorithms

1.
Department of Energy & Resource Engineering, College of Engineering, Peking University, Beijing 100871, China
2.
Archaeology Department, College of Tourism and Archaeology, King Saud University, P.O. Box 2627, 12372 Riyadh, Saudi Arabia
3.
Department of Mechanical Engineering, FE&T, Bahauddin Zakariya University Multan-60000, Pakistan

Received: 02 January 2024 Revised: 08 February 2024 Accepted: 20 February 2024 Published: 01 March 2024

In the evolving field of solar energy, precise forecasting of Solar Irradiance (SI) stands as a pivotal challenge for the optimization of photovoltaic (PV) systems. Addressing the inadequacies in current forecasting techniques, we introduced advanced machine learning models, namely the Rectified Linear Unit Activation with Adaptive Moment Estimation Neural Network (RELAD-ANN) and the Linear Support Vector Machine with Individual Parameter Features (LSIPF). These models broke new ground by striking an unprecedented balance between computational efficiency and predictive accuracy, specifically engineered to overcome common pitfalls such as overfitting and data inconsistency. The RELAD-ANN model, with its multi-layer architecture, sets a new standard in detecting the nuanced dynamics between SI and meteorological variables. By integrating sophisticated regression methods like Support Vector Regression (SVR) and Lightweight Gradient Boosting Machines (Light GBM), our results illuminated the intricate relationship between SI and its influencing factors, marking a novel contribution to the domain of solar energy forecasting. With an R² of 0.935, MAE of 8.20, and MAPE of 3.48%, the model outshone other models, signifying its potential for accurate and reliable SI forecasting, when compared with existing models like Multi-Layer Perceptron, Long Short-Term Memory (LSTM), Multilayer-LSTM, Gated Recurrent Unit, and 1-dimensional Convolutional Neural Network, while the LSIPF model showed limitations in its predictive ability. Light GBM emerged as a robust approach in evaluating environmental influences on SI, outperforming the SVR model. Our findings contributed significantly to the optimization of solar energy systems and could be applied globally, offering a promising direction for renewable energy management and real-time forecasting.

Keywords:

Artificial Neural Network (ANN),
Support Vector Machine (SVM),
Lightweight Gradient Boosting Machines (Light GBM),
machine learning,
Solar Irradiance (SI),
solar forecasting

Citation: Muhammad Farhan Hanif, Muhammad Sabir Naveed, Mohamed Metwaly, Jicang Si, Xiangtao Liu, Jianchun Mi. Advancing solar energy forecasting with modified ANN and light GBM learning algorithms[J]. AIMS Energy, 2024, 12(2): 350-386. doi: 10.3934/energy.2024017

Related Papers:

[1]	Herbert F. Jelinek, Andrei V. Kelarev . A Survey of Data Mining Methods for Automated Diagnosis of Cardiac Autonomic Neuropathy Progression. AIMS Medical Science, 2016, 3(2): 217-233. doi: 10.3934/medsci.2016.2.217
[2]	Andrei V. Kelarev, Xun Yi, Hui Cui, Leanne Rylands, Herbert F. Jelinek . A survey of state-of-the-art methods for securing medical databases. AIMS Medical Science, 2018, 5(1): 1-22. doi: 10.3934/medsci.2018.1.1
[3]	Isaac Kofi Owusu, Emmanuel Acheamfour-Akowuah, Lois Amoah-Kumi, Yaw Amo Wiafe, Stephen Opoku, Enoch Odame Anto . The correlation between obesity and other cardiovascular disease risk factors among adult patients attending a specialist clinic in Kumasi. Ghana. AIMS Medical Science, 2023, 10(1): 24-36. doi: 10.3934/medsci.2023003
[4]	Frantisek Franek, W. F. Smyth, Xinfang Wang . The Role of The Prefix Array in Sequence Analysis: A Survey. AIMS Medical Science, 2017, 4(3): 261-273. doi: 10.3934/medsci.2017.3.261
[5]	Masoud Nazemiyeh, Mehrzad Hajalilou, Mohsen Rajabnia, Akbar Sharifi, Sabah Hasani . Diagnostic value of Endothelin 1 as a marker for diagnosis of pulmonary parenchyma involvement in patients with systemic sclerosis. AIMS Medical Science, 2020, 7(3): 234-242. doi: 10.3934/medsci.2020014
[6]	Kavin Mozhi James, Divya Ravikumar, Sindhura Myneni, Poonguzhali Sivagananam, Poongodi Chellapandian, Rejili Grace Joy Manickaraj, Yuvasree Sargunan, Sai Ravi Teja Kamineni, Vishnu Priya Veeraraghavan, Malathi Kullappan, Surapaneni Krishna Mohan . Knowledge, attitudes on falls and awareness of hospitalized patient's fall risk factors among the nurses working in Tertiary Care Hospitals. AIMS Medical Science, 2022, 9(2): 304-321. doi: 10.3934/medsci.2022013
[7]	Giuliano Crispatzu, Alexandra Schrader, Michael Nothnagel, Marco Herling, Carmen Diana Herling . A Critical Evaluation of Analytic Aspects of Gene Expression Profiling in Lymphoid Leukemias with Broad Applications to Cancer Genomics. AIMS Medical Science, 2016, 3(3): 248-271. doi: 10.3934/medsci.2016.3.248
[8]	Nicole Lavender, David W. Hein, Guy Brock, La Creis R. Kidd . Evaluation of Oxidative Stress Response Related Genetic Variants, Pro-oxidants, Antioxidants and Prostate Cancer. AIMS Medical Science, 2015, 2(4): 271-294. doi: 10.3934/medsci.2015.4.271
[9]	Manasseh B. Wireko, Jacobus Hendricks, Kweku Bedu-Addo, Marlise Van Staden, Emmanuel A. Ntim, Samuel F. Odoom, Isaac K. Owusu . Alcohol consumption and HIV disease prognosis among virally unsuppressed in Rural KwaZulu Natal, South Africa. AIMS Medical Science, 2023, 10(3): 223-236. doi: 10.3934/medsci.2023018
[10]	Katsiaryna V Gris, Kenzo Yamamoto, Marjan Gharagozloo, Shaimaa Mahmoud, Camille Simard, Pavel Gris, Denis Gris . Exhaustive behavioral profile assay to detect genotype differences between wild-type, inflammasome-deficient, and Nlrp12 knock-out mice. AIMS Medical Science, 2018, 5(3): 238-251. doi: 10.3934/medsci.2018.3.238

Abstract

1. Introduction

With the advent of the era of big data, deep learning technology has become a research hotspot in the field of artificial intelligence. It has shown great advantages in image recognition, speech recognition, natural language processing and other fields. The problem of sequence labeling is the most common problem in natural language. Shao et al. ^[1] assign semantic labels in input sequences, exploiting encoding patterns in the form of latent variables in conditional random fields to capture latent structure in observed data. Lin et al. ^[2] proposed an attentional segmentation recurrent neural network (ASRNN), which relies on a hierarchical attentional neural semi-Markov conditional random field (semi-CRF) model for sequence labeling tasks.

Convolutional neural networks (CNN) have been widely used in computer vision recognition tasks. Djenouri et al. ^[3] proposed a technique for particle clustering for object detection (CPOD), built on top of region-based methods, using outlier detection, clustering, particle swarm optimization (PSO), and deep convolutional networks to identify smart object data. Shao et al. ^[4] proposed an end-to-end multi-objective neuroevolution algorithm based on decomposition and dominance (MONEADD) for combinatorial optimization problems to improve the performance of the model in inference. From 2010 to 2017, the ImageNet Large Scale Visual Recognition Challenge has been held for seven years. The image classification accuracy of the champions has increased from 71.8% to 97.3%. The emergence of AlexNet in 2012 was a milestone in deep learning field. After that, the ImageNet dataset accuracy has been significantly improved by novel CNNs, like VGG ^[5], GoogleNet ^[6], ResNet ^[7,8], DenseNet ^[9], SE-Net ^[10], and automatic neutral architecture search ^[11,12,13].

However, it is necessary to consider high accuracy, platform resources, and the efficiency of systems in real-world applications, e.g., automatic drive systems, intelligent robot systems, and mobile device applications. Moreover, most of the best-performing CNNs need to run on a high-performance graphics processing unit (GPU). So, real-world tasks have driven the development of more lightweight CNNs, to allow CNN to be used in more low-performance devices ^[14,15], like Xception ^[16], MobileNet ^[17], MobileNet V2 ^[18,19], ShuffleNet ^[20], ShuffleNet V2 ^[21] and CondenseNet ^[22]. Group convolution and depth-wise separable convolution ^[23] are crucial in these works.

As the best paper at the CVPR 2017 conference, DenseNet beat the best performing ResNet on ImageNet without group convolution or depth-wise separable convolution. Subsequently, the SE-Net achieved the best results in the history of ImageNet in ILSVRC2017, but there are still too many parameters in SE-Net. Following these works, Huang et al. ^[9] have proposed Learned Group Convolutions to improve DenseNet connection and convolution methods. Inspired by these jobs, we study using Squeeze-and-Excitation block (SE-block) to improve the lightweight CNN. Furthermore, we explore how to design the structure of the convolutional layer to enhance the network's performance.

We propose a more efficient network, CED-Net, which combines bottleneck layer with learned group convolution and SE block. Learned group convolution can crop the network channel during the training phase. And the SE block can recalibrate the feature channel to enhance the channel beneficial to the network. Through experiments, we demonstrate that CED-Net is superior to other lightweight network in terms of accuracy, the number of parameters, and FLOPs.

2. Related works

2.1. Model compression and efficient network architectures

In the past few years, designing CNNs by adjusting an optimal depth to balance accuracy and performance was a very active field. Most recent work has been many progresses in algorithm optimization exploration, including pruning redundant connections ^{[24,25,26,27]}, using low-accuracy or quantized weights ^[28,29], or designing efficient network architectures.

Early researchers proved pruning redundant and quantization are effective methods because deep neural networks often have a substantial number of redundant weights that can be pruned or quantized without sacrificing (and sometimes even improving) accuracy. For CNNs, different pruning techniques may lead to varying levels of granularity ^[30]. Fine-grained pruning, e.g., independent weight pruning ^[31], generally achieves a high degree of sparsity. Coarse grained pruning methods such as filter-level pruning earn a lower degree of sparsity, but the resulting networks are much more regular, facilitating efficient implementations.

Recently researchers have explored the structures of the efficient network that can be applied on mobile devices such as MobileNet V2, ShuffleNet V2, and NasNet. In these networks, depth-wise separable convolutions play a vital role, which can reduce a large number of network parameters without significantly reducing the accuracy. However, according to the Howard et al. ^[17,18], a large amount of depth-wise separable convolutions will decrease the computational speed of the network. Therefore, CED-Net uses a more efficient group convolution and densely connected architecture to reduce the number of parameters of the network. Furthermore, because many deep-learning libraries efficiently implement group convolutions, they save a lot of computational time in theory and practice.

In addition, the bottleneck layer proposed in ResNet can effectively reduce parameters for multilayer network. Our experiments show that CED-Net can achieve higher accuracy and fewer parameters than CondenseNet of the same structure when layers are deeper.

2.2. Learned group convolution and squeeze-and-excitation

Huang et al. ^[9], as the best paper for CVPR2017, proposed a densely connection network that is better than the previous champion ResNet on the ImageNet. After that, CondenseNet achieved the same accuracy with only half of the number of parameters of DenseNet. In CondenseNet, learned group convolution plays a key role; it can train the network with sparsity inducing regularization for a fixed number of iterations. Subsequently, it prunes away unimportant filters with low magnitude weights. Because many deep-learning libraries efficiently implement group convolutions, they save a lot of computational time in theory and practice.

Moreover, the Squeeze-and-Excitation structure that shines on ILSVRC2017 has been experimented on by most famous networks. Squeeze and Excitation are two very critical operations. First, it is used to model the interdependencies between feature channels explicitly. It is a new "channel recalibration" strategy. Specifically, by automatically learning the importance of each feature channel, SE-Net enhances the proper channel and suppresses useless channels. Most of the current mainstream networks are constructed based on superimposed basic blocks. It can be seen that the SE module can be embedded in almost all network structures, so CED-Net achieves more efficient performance by embedding the SE module.

3. Channel enhancement dense network

In this section, we first introduce the structure and function of the bottleneck layer. Next, we explore how SE Block as a channel enhancement block can improve the performance of CED-Net. Finally, we describe the network details of CED-Net for CIFAR dataset.

As shown in , H, W, ${C}_{in}$ are the height, width, and the number of channels of the input image, respectively, and g is the growth coefficient of the channel. CED-Net consists of multiple dense blocks for feature extraction. The dense block is shown in Figure 2(c). It consists of two 1 × 1 LG-Conv (Learned Group Convolution) layers and one 3 × 3 G-Conv (Group Convolution) layer. Each 1 × 1 LG-Conv layer uses a permute operation for channel shuffling to reduce accuracy. BN-ReLU nonlinearly activates the input and output in the dense block. And use the AvgPool layer for down sampling.

Figure 1. A 5-layer dense block with channel enhancement and bottleneck layer.

DownLoad: Full-Size Img PowerPoint

Figure 2. Different networks' bottleneck layer or dense block. (a) ResNet. (b) CondenseNet. (c) CED-Net.

DownLoad: Full-Size Img PowerPoint

3.1. Bottleneck layer with learned group convolution

The bottleneck layer is proposed in ResNet, and the detailed structure is shown in Figure 2(a). The three-layer bottleneck structure consists of 1 × 1, 3 × 3, and 1 × 1 convolutional layers, where two 1 × 1 convolutions are used to reduce and increase (restore) dimensions. The 3 × 3 convolutional layer can be seen as a bottleneck for a smaller input/output dimension. We replace the 1 × 1 standard convolution with the learned group convolution, and the 3 × 3 standard convolution is replaced with the group convolution. Unlike ResNet, the CED-Net replaces element-wise addition with channel concatenation. Because it can use the semantic information of different scale feature maps to achieve better performance by increasing the channel, the element addition operation does not take up too much memory during network transmission. Still, it may introduce extra noise that will lose some feature map information.

Figure 2(b) shows the structure used in CondenseNet. The Permute layer, enabling shuffling between channels, is designed to reduce the adverse effects of the introduction of 1 × 1 LG-Conv. But there are still many parameters in a deep network with the bottleneck layer. Figure 2(c) shows part of the structure used by CED-Net. This structure has fewer parameters than that in Figure 2(b). Expressly, the condense factor and bottleneck factor in CED-Net are set to 4 and reduced by half compared to CondenseNet. This is to reduce the parameters caused by adding a 1 × 1 LG-Conv layer.

One dense layer used in CED-Net is of quadratic time complexity (Θ(25G²/4+4CG)) concerning the number (C) of input channels and the number (G) of output channels. Compared with ordinary 3 × 3 convolution (Θ(9CG)), as a result of C is much greater than G with the deepening of network layers, CED-Net reduces the time complexity by half.

Figure 3 shows how channels change the process of the bottleneck layer based on learning group convolution. The parameters and calculation amount are 1/4 of the standard bottleneck layer. Based on the image classification comparing experiments on the CIFAR dataset, we can conclude that our structure can increase the classification accuracy by 0.4% when the number of parameters and the amount of calculation is almost the same as CondenseNet (see Section 4). When network layers are deeper (depth is 272), the number of parameters and the amount of calculation of CED-Net are smaller than the CondenseNet of the same depth. Still, the classification accuracy is higher than that of CondenseNet.

Figure 3. Bottleneck layer with Learned Group Convolutions.

DownLoad: Full-Size Img PowerPoint

3.2. Channel enhancement with Squeeze and Excitation

In CED-Net, since the network is a densely connected structure, the input data of each convolution layer has a large amount of channel information. And the output after convolution is the sum of all previous channel information. This has led to the entanglement of information and spatial relevance. Furthermore, in lightweight networks, group convolution can significantly reduce the amount of computation by ensuring that each convolution operation is only on the corresponding input channel group. However, if multiple sets of convolutions are stacked together, there is a side effect: A channel output is only derived from a few numbers of input channels. This would reduce the information flow between channel groups and express information.

Therefore, we use the channel permute (see Figure 2(c)) and the Squeeze-and-Excitation block to make the information between the groups more circulated to allow the network to focus on more helpful information. As shown in Figure 4, Squeeze-and-Excitation blocks can improve the representation of the network by increasing the interdependence between convolution feature channels. The detailed process is divided into two steps: Squeeze and Excitation.

Figure 4. Channel Enhancement with Squeeze-and-Excitation.

DownLoad: Full-Size Img PowerPoint

Squeeze. CNNs all have the problem that due to the nature of convolutional calculations, each convolution filter can only focus on specific spatial information. To alleviate this problem, the Squeeze, as a global description operation, encodes the global spatial information into the channel descriptor and calculates the mean of each channel through global average pooling.

${z}_{c} = {F}_{sq}\left({u}_{c}\right) = \frac{1}{W\times H}\sum _{i = 1}^{W}\sum _{j = 1}^{H}{u}_{c}(i, j)$

(1)

As shown in Eq (1), where ${\mathrm{Z}}_{\mathrm{c}}$ is the output of the squeeze layer, W, H are the width and height of the input feature map of the current layer. ${u}_{c}$ is the input feature map, and ${F}_{sq}\left(\mathrm{*}\right)$ can represent the global information of the entire feature map. The global average pooling used in this paper squeezes the feature map into a value to indicate the importance of the corresponding channel.

Excitation. To take advantage of the information obtained by the squeeze operation, the excitation operation needs to meet two criteria to achieve full capture of channel dependencies. First, it must be able to learn nonlinear interactions between channels. And second, it must learn a non-mutually exclusive relationship. Specifically, the gate mechanism is parameterized by concatenating two fully connected (FC) layers above and below the nonlinear (ReLU) and then activated with the sigmoid function.

$s = {F}_{ex}\left(z, W\right) = \sigma \left(g\left(z, W\right)\right) = \sigma \left({W}_{2}\theta \left({W}_{1}z\right)\right)$

(2)

where $\mathrm{\theta }$ is the ReLU function. ${\mathrm{W}}_{1}\in {\mathrm{R}}^{\frac{\mathrm{C}}{\mathrm{r}}\times \mathrm{C}}$ , ${\mathrm{W}}_{2}\in {\mathrm{R}}^{\mathrm{C}\times \frac{\mathrm{C}}{\mathrm{r}}}$ are the weights of the dimensionality reduction layer and the dimensionality increase layer, respectively. Where r is the dimensionality reduction rate, and C is the number of channels. To limit the complexity of the model and increase the generalization, a "bottleneck" is formed by a two-layer FC layer around a nonlinear map, where r sets 16. Finally, after obtaining the so-called gate, by multiplying the channel gates by the corresponding feature maps, you can control the flow of information for each feature map.

We embed the Squeeze-and-Excitation block into the 3 × 3 G-Conv layer because the number of input/output feature channels in the first 3 × 3 G-Conv is the same and smaller. The Squeeze-and-Excitation block can effectively enhance the effective channel after feature extraction without extra parameters. According to the research results of Hu et al., this method can balance the accuracy of the model and the number of parameters.

Algorithm 1 Image classification based on CED-Net
Input: In = datasets ( ${\boldsymbol{x}}_\mathbf{1}, {\boldsymbol{y}}_\mathbf{1}$ ), ( ${\boldsymbol{x}}_\mathbf{2}, {\boldsymbol{y}}_\mathbf{2}$ ), …, ( ${\boldsymbol{x}}_{\boldsymbol{m}}, {\boldsymbol{y}}_{\boldsymbol{m}}$ )
Output: Op = Classification accuracy: ( ${\boldsymbol{y}}_{1}, {\boldsymbol{y}}_\mathbf{2}, \dots, {\boldsymbol{y}}_{\boldsymbol{n}}$ )
Set: CED-Net feature extraction: ${\boldsymbol{G}}_{\boldsymbol{k}}(·)$ , k $\in$ (0, n)
for x = 1 : m do
$\boldsymbol{S}\boldsymbol{o}\boldsymbol{f}\boldsymbol{t}\boldsymbol{m}\boldsymbol{a}\boldsymbol{x}\left({\boldsymbol{G}}_{\boldsymbol{k}}\left({\boldsymbol{x}}_{\boldsymbol{i}}\right)\right) = \frac{{\boldsymbol{e}}^{{\boldsymbol{g}}_{\boldsymbol{i}}}}{{\sum }_{\boldsymbol{k}}^{\boldsymbol{n}}{\boldsymbol{e}}^{{\boldsymbol{g}}_{\boldsymbol{k}}}}$
i $\in$ [1, m], where ${\boldsymbol{g}}_{\boldsymbol{i}}$ is one class value in ${\boldsymbol{G}}_{\boldsymbol{k}}(·)$ .
Return Op
end for

3.3. Network design

CED-Net can guarantee good performance while maintaining lightweight models because of the effective combination of bottleneck layer structure and channel enhancement blocks. An important difference between CED-Net and other network architectures is that CED-Net has a very narrow layer. The relatively small channel growth rate is sufficient to obtain the most advanced results on the test dataset. This can increase the proportion of features from the later layers relative to features from the previous layers. So, we set the channel growth rate of a dense connection layer to 4. And we found that if the number of early layers is set too deep, it will significantly increase the FLOPs of the network.

Architectural details. The model used in our experiments has three dense blocks. Before the data enters the first dense block, the input image would go through a 3 × 3 standard convolution which output channels are 16 and stride size is 2. In the dense layer, the number of channel enhancement blocks should be set according to the growth rate, the input channels, and the output channels, see Eq (3).

$n = \frac{Cout-Cin}{g}$

(3)

where g is the growth rate, ${\mathrm{C}}_{\mathrm{i}\mathrm{n}}$ is the input channels, n is the number of channel enhancement blocks, and ${\mathrm{C}}_{\mathrm{o}\mathrm{u}\mathrm{t}}$ is the output channels. For example, in the experiment, we set the growth rate to 8, 16, and 32, and the channels of dense layer output is 256, 756, and 1696 respectively, so the number of channel enhancement blocks in the dense layer are all 30.

For each convolutional layer with a kernel size of 3 × 3, each side of the input is zeros-padded to keep the feature size fixed. In general, we add the batch normalization layer and the ReLU function after the last dense layer and then use the global average pooling to compress the feature map into one dimension as the input of the Softmax layer. The exact network configuration is shown in Tables 1 and 2.

Table 1. Network structure of CED-Net on CIFAR.

Layers	Output Size	Output Channels	Repeat	Stride
3 × 3 Convolution	32 × 32	16	1	1
Dense bottleneck block	32 × 32	256 (g = 8)	30	1
Avg pooling	16 × 16	256 (g = 8)	1	2
Dense bottleneck block	16 × 16	736 (g = 16)	30	1
Avg pooling	8 × 8	736 (g = 16)	1	2
Dense bottleneck block	8 × 8	1696 (g = 32)	30	1
Global avg pooling	1 × 1	1696	1	8
Fully connected	1 × 1	10	1

| Show Table

DownLoad: CSV

Table 2. Network structure of CED-Net on ImageNet.

Layers	Output Size	Output Channels	Repeat	Stride
3 × 3 Convolution	112 × 112	64	1	2
Dense bottleneck block	112 × 112	96 (g = 8)	4	1
Avg pooling	56 × 56	96 (g = 8)	1	2
Dense bottleneck block	56 × 56	192 (g = 16)	6	1
Avg pooling	28 × 28	192 (g = 16)	1	2
Dense bottleneck block	28 × 28	448 (g = 32)	8	1
Avg pooling	14 × 14	448 (g = 32)	1	2
Dense bottleneck block	14 × 14	1088 (g = 64)	10	1
Avg pooling	7 × 7	1088 (g = 64)	1	2
Dense bottleneck block	7 × 7	2112 (g = 128)	8	1
Global avg pooling	1 × 1	2112	1	7
Fully connected	1 × 1	1000	1

| Show Table

DownLoad: CSV

The training process of CED-Net is shown in Algorithm 1. (x_i, y_i) in the input represent the images and label of the ith batch respectively. For each batch, we use softmax to obtain the output Yi of CED-Net. Finally, the image features G_k of n categories are obtained.

4. Experiments

This section conducted experiments on the CIFAR10, CIFAR-100, and the ImageNet (ILSVRC 2012) datasets. First, we compared them with other advanced convolutional neural networks, such as VGG16, ResNet-101, and DenseNet. Then, we conducted ablation experiments to CED-Net, mainly comparing three networks, the primary network of CED-Net-128, the optimization network with only the bottleneck layer, and the network with only the channel enhancement block. Through these experiments, we verify the effectiveness of our improved method. Next, we will introduce the data set and the evaluation indicators of the experiment.

4.1. Dataset

The CIFAR-10 and CIFAR-100 datasets consist of colored natural images with 32 × 32 pixels. CIFAR-10 consists of images drawn from 10 classes and CIFAR-100 from 100 classes. The training and test sets contain 50, 000 and 10, 000 images, respectively, and we picked up 5000 training images as a validation set. We adopt a standard data augmentation scheme (mirroring/shifting) and image zero-padded with 4 pixels per side, and then randomly cropped to generate a 32 × 32 image. The image is flipped horizontally at a probability of 0.5 and normalized by subtracting the channel average and dividing by the channel standard deviation.

The ImageNet datasets consist of 224 × 224 pixels colored natural images with 1000 classes. The training and validation sets contain 1, 280, 000 and 50, 000 images, respectively. We adopt the data-augmentation scheme at training time and perform a rescaling to 256 × 256 followed by a 224 × 224 center crop at test time before feeding the input image into the networks.

4.2. Evaluation criteria

We evaluate CED-Net on three criteria:

4.2.1. Classification accuracy

Accuracy is the most common metric. It is the number of samples that are paired divided by the number of all samples. Generally speaking, the higher the accuracy is, the better the classifier will be:

$accuracy = \left(TP+TN\right)/\left(P+N\right)$

(4)

where P (positive) is the number of positive examples in the sample, and N (negative) is the number of negative examples. TP (true positives) is the number of samples that are positive examples that are correctly classified. TN (true negatives) is the number of samples that are actually negative that are correctly classified.

4.2.2. Model parameter

For a single convolutional kernel we have:

$parameters = {k}^{2}\times Cin\times Cout$

(5)

where 𝑘 is the convolution filter's size, 𝐶in is the input channels, and 𝐶out is the output channels;

4.2.3. FLOPs (floating-point operations)

To measure the amount of calculation of the model, we compute the number of FLOPs of each layer. For convolutional kernels, we have:

$FLOPs = 2HW({k}^{2}Cin+ 1)Cout$

(6)

where 𝐻, 𝑊 are height and width. For fully connected layers, we compute FLOPs as:

$FLOPs = (2I- 1)O$

(7)

where 𝐼 is the input dimensionality and 𝑂 is the output dimensionality.

4.2.4. Other criteria

To further prove the stability of CED-Net, we added the interpretation and comparison of precision, recall and F-measure in the ablation experiment:

$precision = TP/\left(TP+FP\right)$

(8)

$recall = TP/\left(TP+FN\right)$

(9)

$F-measure = 2*precision*recall/\left(precision+recall\right)$

(10)

4.3. The result of image classification

4.3.1. Training details

We train all models with stochastic gradient descent (SGD) using similar optimization hyper-parameters ^{[23,24,25,26,27,28,29,30]}. And we set the Nesterov momentum weight to 0.9 without damping and use a weight decay of 0.0001. All models are trained with mini-batch size 128 for 200 epochs on the training datasets. We use the cosine annealing learning rate curve, starting from 0.1 and gradually reducing to 0.

4.3.2. Results on CIFAR

In this part, we train CED-Net and other advanced convolutional neural networks on the CIFAR-10 and CIFAR100 datasets. We compared these models under the above three evaluation criteria. See Table 3 for a detailed list.

Table 3. The classification accuracy on CIFAR-10 and CIFAR-100.

Model	Params	FLOPs	CIFAR-10	CIFAR-100
VGG-16	14.73 M	314 M	92.64	72.23
ResNet-101	42.51 M	2515 M	93.75	77.78
ResNeXt-29	9.13 M	1413 M	94.82	78.83
MobileNet V2	2.30 M	92 M	94.43	68.08
DenseNet-121	6.96 M	893 M	94.04	77.01
CondenseNet-86	0.52 M	65 M	94.48	76.36
CondenseNet-182	4.20 M	513 M	95.87	80.13
CED-Net-128 CED-Net-272	0.69 M 5.32 M	75 M 649 M	94.89 96.31	77.35 80.72

| Show Table

DownLoad: CSV

In Table 3, we show the results of comparing 128-layer CED-Net and 272-layer CED-Net with other state-of-the-art CNN architectures. All models were trained in 200 epochs in the experiment. The results show that after introducing the bottleneck layer structure and channel enhancement blocks to CED-Net, the CondenseNet increases the accuracy by 0.4–0.5% with minimal parameters and FLOPs cost compared with the same number of stacked blocks n datasets. Moreover, compared to the more advanced MobileNet V2, CED-Net is more accurate without using depth-wise separable convolutions. And the parameter amount is 1/4 of it, and the FLOPs are also more minor.

4.3.3. Results on ImageNet

In this part, we train CED-Net and other advanced convolutional neural networks on the ImageNet datasets. We compared these models under the above four evaluation criteria. See Table 4 for a detailed list.

Table 4. The classification accuracy on ImageNet.

Model	Params	FLOPs	Top-1	Top-5
VGG-16	138.36 M	15.48 G	71.93	90.67
ResNet-101	44.55 M	7.83 G	80.13	95.4
MobileNet V2	3.5 M	0.3 G	71.8	91
DenseNet-121	7.98 M	2.87 G	74.98	92.29
CondenseNet	4.8 M	0.53 G	73.8	91.7
SE-Net	115 M	20.78 G	81.32	95.53
CED-Net-115	9.3 M	1.13 G	78.65	93.7

| Show Table

DownLoad: CSV

In Table 4, we show the results of comparing 115-layer CED-Net with other CNN architectures. The results show that the accuracy of Top-1 and Top-5 is improved by 4.85 and 2%, respectively, compared with the same depth of CondenseNet. At the same time, the dense bottleneck block used in the CED net is more complex. Compared with DenseNet, CED-Net increases the number of parameters by 16.5% but reduces the amount of calculation by 39.4%; the accuracies of Top-1 and top-5 are improved by 3.67 and 1.41%, respectively. Compared with SE-Net, CED-Net reduces the Top-1 accuracy by 2.67%, but the parameter quantity is only 8.1% of SE-Net.

Some misclassification images are shown in Figure 5. There may be unavoidable interference information in these pictures; Also, it may be that the network model constructed in this paper does not learn a sufficient number of diverse features and cannot correctly identify each picture with different features.

Figure 5. ImageNet misclassified pictures.

DownLoad: Full-Size Img PowerPoint

4.3.4. Redundant LG-Conv

In the dense bottleneck block shown in Figure 2, we use the learned group revolution before and after the 3 × 3 group convolution, which means that there are two consecutive learned group convolutions between the two 3 × 3 group convolutions. The two index layers used have redundancy, but we think it is necessary. These redundancies can improve the learned group revolution's generalization performance and help subsequent feature extraction. But this design increases the amount of calculation and parameters of the intermediate convolution.

4.3.5. Ablation experiments

In this part, we performed a CED-Net ablation experiment. We trained four models on the CIFAR-10 dataset, CEDNet-128a with no bottleneck layer and channel enhancement block, CED-Net-128b with convolutional layer structure changed to bottleneck layer, CED-Net-128c with channel enhancement block based on CondenseNet-86 and CED-Net-128 that we proposed in this paper.

In Table 5, CondenseNet-86 is our basic model. It can be seen that when we turn the structure of CondenseNet into a bottleneck layer, the parameters and FLOPs of the network are only slightly improved, and the accuracy can be increased by about 0.3%. When we added the channel enhancement block to CondenseNet-86, we saw not much increase in FLOPs. But the parameters are raised, and the accuracy can be improved by about 0.3%. In our CED-Net-128, the accuracy rate has been significantly improved, and the channel enhancement block mainly causes the increase in parameters. The bottleneck layer structure causes an increase in FLOPs. In addition, the Accuracy, Precision, Recall and F-measure of each model are very close, which prove that the four models have extracted stable features.

Table 5. The result of ablation experiments of CIFAR-10.

Model	Params	FLOPs	Accuracy	Precision	Recall	F-measure
CondenseNet^a	0.52M	65.82M	94.48	94.50	94.48	94.49
CED-Net-128^b	0.59M	75.04M	94.75	94.75	94.75	94.75
CED-Net-128^c	0.66M	67.04M	94.74	94.76	94.74	94.75
CED-Net-128	0.69M	75.41M	94.89	94.89	94.89	94.88
Note: ^aThe basic model of CED-Net same as CondenseNet-86 without bottleneck layer and channel enhancement block; ^bThe basic model of CED-Net only add a bottleneck layer; ^cThe basic model of CED-Net only add channel enhancement block.

| Show Table

DownLoad: CSV

5. Conclusions

This paper introduces CED-Net: a more efficient densely concatenated convolutional neural network based on feature enhancement block and bottleneck layer structure, which increases accuracy by learning group convolution and feature reuse. To make the reasoning effective, the pruned network can be converted to a network with conventional group convolution, which is effectively implemented in most deep learning libraries. In our experiments, CED-Net outperformed its underlying network CondenseNet and other advanced convolutional neural networks such as Mobilenet V2 and ResNeXt in terms of computational efficiency at the same accuracy level. Moreover, CED-Net has a much simpler structure with higher accuracy. We anticipate further research in CED-Net to combine this framework to the Neural Architecture Search (NAS), so as to design more lightweight Convolutional Neural Network models. We hope our work will draw more attention toward a broader view of using lightweight architecture for deep learning.

Acknowledgments

This work was supported by the National Natural Science Foundation of China (No. 61976217), the Opening Foundation of Key Laboratory of Opto-technology and Intelligent Control, Ministry of Education (KFKT2020-3), the Fundamental Research Funds of Central Universities (No. 2019XKQ YMS87), Science and Technology Planning Project of Xuzhou (No. KC21193).

Conflict of interest

The authors declare there is no conflict of interest.

References

[1]	Guan Y, Lu H, Jiang Y, et al. (2021) Changes in global climate heterogeneity under the 21st century global warming. Ecol Indic 130: 108075. https://doi.org/10.1016/j.ecolind.2021.108075 doi: 10.1016/j.ecolind.2021.108075
[2]	Sohani A, Shahverdian MH, Sayyaadi H, et al. (2021) Energy and exergy analyses on seasonal comparative evaluation of water flow cooling for improving the performance of monocrystalline PV module in hot-arid climate. Sustainability 13: 6084. https://doi.org/10.3390/su13116084 doi: 10.3390/su13116084
[3]	Sahebi HK, Hoseinzadeh S, Ghadamian H, et al. (2021) Techno-economic analysis and new design of a photovoltaic power plant by a direct radiation amplification system. Sustainability 13: 11493. https://doi.org/10.3390/su132011493 doi: 10.3390/su132011493
[4]	Hoseinzadeh S, Ghasemi MH, Heyns S (2020) Application of hybrid systems in solution of low power generation at hot seasons for micro hydro systems. Renewable Energy 160: 323–332. https://doi.org/10.1016/j.renene.2020.06.149 doi: 10.1016/j.renene.2020.06.149
[5]	Makkiabadi M, Hoseinzadeh S, Mohammadi M, et al. (2020) Energy feasibility of hybrid PV/wind systems with electricity generation assessment under Iran environment. Appl Sol Energy 56: 517–525. https://doi.org/10.3103/s0003701x20060079 doi: 10.3103/s0003701x20060079
[6]	Hannan MA, Al-Shetwi AQ, Ker PJ, et al. (2021) Impact of renewable energy utilization and artificial intelligence in achieving sustainable development goals. Energy Rep 7: 5359–5373. https://doi.org/10.1016/j.egyr.2021.08.172 doi: 10.1016/j.egyr.2021.08.172
[7]	Rafique MM, Rehman S (2017) National energy scenario of Pakistan—Current status, future alternatives, and institutional infrastructure: An overview. Renewable Sustainable Energy Rev 69: 156–167. https://doi.org/10.1016/j.rser.2016.11.057 doi: 10.1016/j.rser.2016.11.057
[8]	Pikus M, Wąs J (2023) Using deep neural network methods for forecasting energy productivity based on comparison of simulation and DNN results for central Poland—Swietokrzyskie Voivodeship. Energies 16: 6632. https://doi.org/10.3390/en16186632 doi: 10.3390/en16186632
[9]	Rafique MM, Bahaidarah HMS, Anwar MK (2019) Enabling private sector investment in off-grid electrification for cleaner production: Optimum designing and achievable rate of unit electricity. J Clean Prod 206: 508–523. https://doi.org/10.1016/j.jclepro.2018.09.123 doi: 10.1016/j.jclepro.2018.09.123
[10]	Sørensen ML, Nystrup P, Bjerregård MB, et al. (2023) Recent developments in multivariate wind and solar power forecasting. Wiley Interdiscip Rev Energy Environ, 12. https://doi.org/10.1002/wene.465 doi: 10.1002/wene.465
[11]	Wang H, Liu Y, Zhou B, et al. (2020) Taxonomy research of artificial intelligence for deterministic solar power forecasting. Energy Convers Manage 214: 112909. https://doi.org/10.1016/j.enconman.2020.112909 doi: 10.1016/j.enconman.2020.112909
[12]	Sobri S, Koohi-Kamali S, Rahim NA (2018) Solar photovoltaic generation forecasting methods: A review. Energy Convers Manage 156: 459–497. https://doi.org/10.1016/j.enconman.2017.11.019 doi: 10.1016/j.enconman.2017.11.019
[13]	Mokarram M, Mokarram MJ, Gitizadeh M, et al. (2020) A novel optimal placing of solar farms utilizing multi-criteria decision-making (MCDA) and feature selection. J Clean Prod 261: 121098. https://doi.org/10.1016/j.jclepro.2020.121098 doi: 10.1016/j.jclepro.2020.121098
[14]	Cesar LB, Silva RAE, Callejo MÁM, et al. (2022) Review on Spatio-temporal solar forecasting methods driven by in Situ measurements or their combination with satellite and numerical weather prediction (NWP) estimates. Energies 15: 4341. https://doi.org/10.3390/EN15124341 doi: 10.3390/EN15124341
[15]	Miller SD, Rogers MA, Haynes JM, et al. (2018) Short-term solar irradiance forecasting via satellite/model coupling. Sol Energy 168: 102–117. https://doi.org/10.1016/j.solener.2017.11.049 doi: 10.1016/j.solener.2017.11.049
[16]	Hao Y, Tian C (2019) A novel two-stage forecasting model based on error factor and ensemble method for multi-step wind power forecasting. Appl Energy 238: 368–383. https://doi.org/10.1016/j.apenergy.2019.01.063 doi: 10.1016/j.apenergy.2019.01.063
[17]	Murata A, Ohtake H, Oozeki T (2018) Modeling of uncertainty of solar irradiance forecasts on numerical weather predictions with the estimation of multiple confidence intervals. Renewable Energy 117: 193–201. https://doi.org/10.1016/j.renene.2017.10.043 doi: 10.1016/j.renene.2017.10.043
[18]	Munkhammar J, van der Meer D, Widén J (2019) Probabilistic forecasting of high-resolution clear-sky index time-series using a Markov-chain mixture distribution model. Sol Energy 184: 688–695. https://doi.org/10.1016/j.solener.2019.04.014 doi: 10.1016/j.solener.2019.04.014
[19]	Halabi LM, Mekhilef S, Hossain M (2018) Performance evaluation of hybrid adaptive neuro-fuzzy inference system models for predicting monthly global solar radiation. Appl Energy 213: 247–261. https://doi.org/10.1016/j.apenergy.2018.01.035 doi: 10.1016/j.apenergy.2018.01.035
[20]	Dong J, Olama MM, Kuruganti T, et al. (2020) Novel stochastic methods to predict short-term solar radiation and photovoltaic power. Renewable Energy 145: 333–346. https://doi.org/10.1016/j.renene.2019.05.073 doi: 10.1016/j.renene.2019.05.073
[21]	Ahmad T, Zhang D, Huang C (2021) Methodological framework for short-and medium-term energy, solar and wind power forecasting with stochastic-based machine learning approach to monetary and energy policy applications. Energy 231: 120911. https://doi.org/10.1016/j.energy.2021.120911 doi: 10.1016/j.energy.2021.120911
[22]	Ağbulut Ü, Gürel AE, Biçen Y (2021) Prediction of daily global solar radiation using different machine learning algorithms: Evaluation and comparison. Renewable Sustainable Energy Rev 135: 110114. https://doi.org/10.1016/j.rser.2020.110114 doi: 10.1016/j.rser.2020.110114
[23]	Jumin E, Basaruddin FB, Yusoff YBM, et al. (2021) Solar radiation prediction using boosted decision tree regression model: A case study in Malaysia. Environ Sci Pollut Res 28: 26571–26583. https://doi.org/10.1007/s11356-021-12435-6 doi: 10.1007/s11356-021-12435-6
[24]	Benali L, Notton G, Fouilloy A, et al. (2019) Solar radiation forecasting using artificial neural network and random forest methods: Application to normal beam, horizontal diffuse and global components. Renewable Energy 132: 871–884. https://doi.org/10.1016/j.renene.2018.08.044 doi: 10.1016/j.renene.2018.08.044
[25]	Zendehboudi A, Baseer MA, Saidur R (2018) Application of support vector machine models for forecasting solar and wind energy resources: A review. J Clean Prod 199: 272–285. https://doi.org/10.1016/j.jclepro.2018.07.164 doi: 10.1016/j.jclepro.2018.07.164
[26]	André PS, Dias LMS, Correia SFH, et al. (2024) Artificial neural networks for predicting optical conversion efficiency in luminescent solar concentrators. Sol Energy 268: 112290. https://doi.org/10.1016/j.solener.2023.112290 doi: 10.1016/j.solener.2023.112290
[27]	Girimurugan R, Selvaraju P, Jeevanandam P, et al. (2023) Application of deep learning to the prediction of solar irradiance through missing data. Int J Photoenergy 2023: 4717110. https://doi.org/10.1155/2023/4717110 doi: 10.1155/2023/4717110
[28]	Noman AM, Khan H, Sher HA, et al. (2023) Scaled conjugate gradient artificial neural network-based ripple current correlation MPPT algorithms for PV system. Int J Photoenergy 2023: 8891052. https://doi.org/10.1155/2023/8891052 doi: 10.1155/2023/8891052
[29]	Ricci L, Papurello D (2023) A prediction model for energy production in a solar concentrator using artificial neural networks. Int J Energy Res 2023: 9196506. https://doi.org/10.1155/2023/9196506 doi: 10.1155/2023/9196506
[30]	Konstantinou M, Peratikou S, Charalambides AG (2021) Solar photovoltaic forecasting of power output using LSTM networks. Atmosphere 12: 124. https://doi.org/10.3390/atmos12010124 doi: 10.3390/atmos12010124
[31]	Pan C, Tan J, Feng D (2021) Prediction intervals estimation of solar generation based on gated recurrent unit and kernel density estimation. Neurocomputing 453: 552–562. https://doi.org/10.1016/j.neucom.2020.10.027 doi: 10.1016/j.neucom.2020.10.027
[32]	Feng C, Zhang J, Zhang W, et al. (2022) Convolutional neural networks for intra-hour solar forecasting based on sky image sequences. Appl Energy 310: 118438. https://doi.org/10.1016/j.apenergy.2021.118438 doi: 10.1016/j.apenergy.2021.118438
[33]	Colak HE, Memisoglu T, Gercek Y (2020) Optimal site selection for solar photovoltaic (PV) power plants using GIS and AHP: A case study of Malatya Province, Turkey. Renewable Energy 149: 565–576. https://doi.org/10.1016/j.renene.2019.12.078 doi: 10.1016/j.renene.2019.12.078
[34]	Mousapour Mamoudan M, Ostadi A, Pourkhodabakhsh N, et al. (2023) Hybrid neural network-based metaheuristics for prediction of financial markets: A case study on global gold market. J Comput Des Eng 10: 1110–1125. https://doi.org/10.1093/jcde/qwad039 doi: 10.1093/jcde/qwad039
[35]	Gholizadeh H, Fathollahi-Fard AM, Fazlollahtabar H, et al. (2022) Fuzzy data-driven scenario-based robust data envelopment analysis for prediction and optimization of an electrical discharge machine's parameters. Expert Syst Appl 193: 116419. https://doi.org/10.1016/j.eswa.2021.116419 doi: 10.1016/j.eswa.2021.116419
[36]	Ghazikhani A, Babaeian I, Gheibi M, et al. (2022) A smart post-processing system for forecasting the climate precipitation based on machine learning computations. Sustainability 14: 6624. https://doi.org/10.3390/su14116624 doi: 10.3390/su14116624
[37]	Han Z, Zhao J, Leung H, et al. (2021) A review of deep learning models for time series prediction. IEEE Sens J 21: 7833–7848. https://doi.org/10.1109/jsen.2019.2923982 doi: 10.1109/jsen.2019.2923982
[38]	Ghimire S, Deo RC, Raj N, et al. (2019) Deep solar radiation forecasting with convolutional neural network and long short-term memory network algorithms. Appl Energy 253: 113541. https://doi.org/10.1016/j.apenergy.2019.113541 doi: 10.1016/j.apenergy.2019.113541
[39]	Zang H, Liu L, Sun L, et al. (2020) Short-term global horizontal irradiance forecasting based on a hybrid CNN-LSTM model with spatiotemporal correlations. Renewable Energy 160: 26–41. https://doi.org/10.1016/j.renene.2020.05.150 doi: 10.1016/j.renene.2020.05.150
[40]	Rathore N, Rathore P, Basak A, et al. (2021) Multi Scale Graph Wavenet for wind speed forecasting. 2021 IEEE International Conference on Big Data (Big Data), Orlando, FL, USA, 4047–4053. https://doi.org/10.1109/bigdata52589.2021.9671624
[41]	Shaikh AK, Nazir A, Khalique N, et al. (2023) A new approach to seasonal energy consumption forecasting using temporal convolutional networks. Results Eng 19: 101296. https://doi.org/10.1016/j.rineng.2023.101296 doi: 10.1016/j.rineng.2023.101296
[42]	Qu J, Qian Z, Pei Y (2021) Day-ahead hourly photovoltaic power forecasting using attention-based CNN-LSTM neural network embedded with multiple relevant and target variables prediction pattern. Energy 232: 120996. https://doi.org/10.1016/j.energy.2021.120996 doi: 10.1016/j.energy.2021.120996
[43]	Zhan C, Zhang X, Yuan J, et al. (2024) A hybrid approach for low-carbon transportation system analysis: integrating CRITIC-DEMATEL and deep learning features. Int J Environ Sci Technol 21: 791–804. https://doi.org/10.1007/s13762-023-04995-6 doi: 10.1007/s13762-023-04995-6
[44]	Kumari P, Toshniwal D (2021) Deep learning models for solar irradiance forecasting: A comprehensive review. J Clean Prod 318: 128566. https://doi.org/10.1016/j.jclepro.2021.128566 doi: 10.1016/j.jclepro.2021.128566
[45]	Akram MW, Li G, Jin Y, et al. (2019) CNN based automatic detection of photovoltaic cell defects in electroluminescence images. Energy 189: 116319. https://doi.org/10.1016/j.energy.2019.116319 doi: 10.1016/j.energy.2019.116319
[46]	Halton C (2023) Predictive analytics: Definition, model types, and uses, 2021. Investopedia Available from: https://www.investopedia.com/terms/p/predictive-analytics.asp#: ~: text = the most common predictive models, deep learning methods and technologies.
[47]	Manju S, Sandeep M (2019) Prediction and performance assessment of global solar radiation in Indian cities: A comparison of satellite and surface measured data. J Clean Prod 230: 116–128. https://doi.org/10.1016/j.jclepro.2019.05.108 doi: 10.1016/j.jclepro.2019.05.108
[48]	Ahmad S, Parvez M, Khan TA, et al. (2022) A hybrid approach using AHP-TOPSIS methods for ranking of soft computing techniques based on their attributes for prediction of solar radiation. Environ Challenges 9: 100634. https://doi.org/10.1016/j.envc.2022.100634 doi: 10.1016/j.envc.2022.100634
[49]	Ağbulut Ü, Gürel AE, Biçen Y (2021) Prediction of daily global solar radiation using different machine learning algorithms: Evaluation and comparison. Renewable Sustainable Energy Rev 135: 110114. https://doi.org/10.1016/j.rser.2020.110114 doi: 10.1016/j.rser.2020.110114
[50]	Islam S, Roy NK (2023) Renewable's integration into power systems through intelligent techniques: Implementation procedures, key features, and performance evaluation. Energy Rep 9: 6063–6087. https://doi.org/10.1016/j.egyr.2023.05.063 doi: 10.1016/j.egyr.2023.05.063
[51]	Farooqui SZ (2014) Prospects of renewables penetration in the energy mix of Pakistan. Renewable Sustainable Energy Rev 29: 693–700. https://doi.org/10.1016/j.rser.2013.08.083 doi: 10.1016/j.rser.2013.08.083
[52]	Government of Pakistan FD (2022) Pakistan Economic Survey 2021-22. Available from: https://www.finance.gov.pk/survey_2022.html.
[53]	Đukanović M, Kašćelan L, Vuković S, et al. (2023) A machine learning approach for time series forecasting with application to debt risk of the Montenegrin electricity industry. Energy Rep 9: 362–369. https://doi.org/10.1016/j.egyr.2023.05.240 doi: 10.1016/j.egyr.2023.05.240
[54]	Irfan M, Zhao ZY, Mukeshimana MC, et al. (2019) Wind energy development in South Asia: Status, potential and policies. 2019 2nd International Conference on Computing, Mathematics and Engineering Technologies (iCoMET), Sukkur, Pakistan, 1–6. https://doi.org/10.1109/icomet.2019.8673484
[55]	Energy system of Asia Pacific. International Energy Agency. Available from: https://www.iea.org/regions/asia-pacific.
[56]	Climate change. International Energy Agency. Available from: https://www.iea.org/.
[57]	Rafique MM, Rehman S (2017) National energy scenario of Pakistan—Current status, future alternatives, and institutional infrastructure: An overview. Renewable Sustainable Energy Rev. 69: 156–167. https://doi.org/10.1016/j.rser.2016.11.057 doi: 10.1016/j.rser.2016.11.057
[58]	Awan U, Knight I (2020) Domestic sector energy demand and prediction models for Punjab Pakistan. J Building Eng 32: 101790. https://doi.org/10.1016/j.jobe.2020.101790 doi: 10.1016/j.jobe.2020.101790
[59]	Muhammad F, Waleed Raza M, Khan S, et al. (2017) Different solar potential co-ordinates of Pakistan. Innovative Energy Res 6: 1–8. https://doi.org/10.4172/2576-1463.1000173 doi: 10.4172/2576-1463.1000173
[60]	Farooq M, Shakoor A (2013) Severe energy crises and solar thermal energy as a viable option for Pakistan. J Renewable Sustainable Energy 5: 013104. https://doi.org/10.1063/1.4772637 doi: 10.1063/1.4772637
[61]	Shabbir N, Usman M, Jawad M, et al. (2020) Economic analysis and impact on national grid by domestic photovoltaic system installations in Pakistan. Renewable Energy 153: 509–521. https://doi.org/10.1016/j.renene.2020.01.114 doi: 10.1016/j.renene.2020.01.114
[62]	Global solar atlas. Available from: https://globalsolaratlas.info/map.
[63]	CDPC, Department PM Climate Records Quetta. Available from: https://cdpc.pmd.gov.pk/.
[64]	JRC Photovoltaic Geographical Information System (PVGIS)—European Commission. Available from: https://re.jrc.ec.europa.eu/pvg_tools/en/#PVP/.
[65]	Earthdata. NASA. Available from: https://www.earthdata.nasa.gov/.
[66]	Welcome to Colaboratory—Google Colaboratory. Available from: https://colab.research.google.com/.
[67]	Emmanuel T, Maupong T, Mpoeleng D, et al. (2021) A survey on missing data in machine learning. J Big Data 8: 1–37. https://doi.org/10.1186/S40537-021-00516-9 doi: 10.1186/S40537-021-00516-9
[68]	Ackerman S, Farchi E, Raz O, et al. (2020) Detection of data drift and outliers affecting machine learning model performance over time. arXiv In: JSM Proceedings, Nonparametric Statistics Section, 20202. Philadelphia, PA: American Statistical Association, 144–160. https://doi.org/10.48550/arXiv.2012.09258
[69]	Khadka N (2019) General machine learning practices using Python. Available from: https://www.theseus.fi/bitstream/handle/10024/226305/Khadka_Nibesh.pdf?sequence = 2.
[70]	Pereira Barata A, Takes FW, Van Den Herik HJ, et al. (2019) Imputation methods outperform missing-indicator for data missing completely at random. 2019 International Conference on Data Mining Workshops (ICDMW), Beijing, China, 407–414. https://doi.org/10.1109/ICDMW.2019.00066 doi: 10.1109/ICDMW.2019.00066
[71]	Wu P, Zhang Q, Wang G, et al. (2023) Dynamic feature selection combining standard deviation and interaction information. Int J Mach Learn Cyber 14: 1407–1426. https://doi.org/10.1007/S13042-022-01706-4 doi: 10.1007/S13042-022-01706-4
[72]	Begum S, Meraj S, Shetty BS (2023) Successful data mining: With dimension reduction. Proceedings of the International Conference on Applications of Machine Intelligence and Data Analytics, 11–22. https://doi.org/10.2991/978-94-6463-136-4_3 doi: 10.2991/978-94-6463-136-4_3
[73]	Li B, Wu F, Lim S-N, et al. (2021) On feature normalization and data augmentation. IEEE/CVF Conference on Computer Vision and Pattern, 12383–12392. https://doi.org/10.48550/arXiv.2002.11102 doi: 10.48550/arXiv.2002.11102
[74]	Ramirez-Vergara J, Bosman LB, Leon-Salas WD, et al. (2021) Ambient temperature and solar irradiance forecasting prediction horizon sensitivity analysis. Machine Learning Appl 6: 100128. https://doi.org/10.1016/j.mlwa.2021.100128 doi: 10.1016/j.mlwa.2021.100128
[75]	Verbois H, Huva R, Rusydi A, et al. (2018) Solar irradiance forecasting in the tropics using numerical weather prediction and statistical learning. Sol Energy 162: 265–277. https://doi.org/10.1016/j.solener.2018.01.007 doi: 10.1016/j.solener.2018.01.007
[76]	Ssekulima EB, Anwar MB, Al Hinai A, et al. (2016) Wind speed and solar irradiance forecasting techniques for enhanced renewable energy integration with the grid: A review. IET Renewable Power Generation 10: 885–989. https://doi.org/10.1049/iet-rpg.2015.0477 doi: 10.1049/iet-rpg.2015.0477
[77]	Kumar N, Sinha UK, Sharma SP, et al. (2017) Prediction of daily global solar radiation using Neural Networks with improved gain factors and RBF Networks. Int J Renewable Energy Res 7: 1235–1244. https://doi.org/10.20508/ijrer.v7i3.5988.g7156 doi: 10.20508/ijrer.v7i3.5988.g7156
[78]	Siva Krishna Rao KDV, Premalatha M, Naveen C (2018) Models for forecasting monthly mean daily global solar radiation from in-situ measurements: Application in Tropical Climate, India. Urban Clim 24: 921–939. https://doi.org/10.1016/j.uclim.2017.11.004 doi: 10.1016/j.uclim.2017.11.004
[79]	Yıldırım HB, Çelik Ö, Teke A, et al. (2018) Estimating daily Global solar radiation with graphical user interface in Eastern Mediterranean region of Turkey. Renewable Sustainable Energy Rev 82: 1528–1537. https://doi.org/10.1016/j.rser.2017.06.030 doi: 10.1016/j.rser.2017.06.030
[80]	Mohaideen Abdul Kadhar K, Anand G (2021) Basics of Python programming. Data Sci Raspberry Pi, 13–47. https://doi.org/10.1007/978-1-4842-6825-4_2 doi: 10.1007/978-1-4842-6825-4_2
[81]	Gholizadeh S (2022) Top popular Python libraries in research. J Robot Auto Res 3: 142–145. http://dx.doi.org/10.33140/jrar.03.02.02 doi: 10.33140/jrar.03.02.02
[82]	Stančin I, Jović A (2019) An overview and comparison of free Python libraries for data mining and big data analysis. 42nd International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), 977–982. https://doi.org/10.23919/mipro.2019.8757088 doi: 10.23919/mipro.2019.8757088
[83]	Voigtlaender F (2023) The universal approximation theorem for complex-valued neural networks. Appl Comput Harmon Anal 64: 33–61. https://doi.org/10.1016/j.acha.2022.12.002 doi: 10.1016/j.acha.2022.12.002
[84]	Winkler DA, Le TC (2017) Performance of deep and shallow neural networks, the universal approximation theorem, activity cliffs, and QSAR. Mol Inf, 36. https://doi.org/10.1002/minf.201600118 doi: 10.1002/minf.201600118
[85]	Lu Y, Lu J (2020) A universal approximation theorem of Deep Neural Networks for expressing probability distributions. arXiv. https://doi.org/10.48550/arXiv.2004.08867 doi: 10.48550/arXiv.2004.08867
[86]	Dubey SR, Singh SK, Chaudhuri BB (2022) Activation functions in deep learning: A comprehensive survey and benchmark. Neurocomputing 503: 92–108. https://doi.org/10.1016/j.neucom.2022.06.111 doi: 10.1016/j.neucom.2022.06.111
[87]	Tato A, Nkambou R (2018) Improving Adam optimizer. ICLR 2018 Workshop. Available from: https://openreview.net/forum?id = HJfpZq1DM.
[88]	Toh SC, Lai SH, Mirzaei M, et al. (2023) Sequential data processing for IMERG satellite rainfall comparison and improvement using LSTM and ADAM optimizer. Appl Sci 13: 7237. https://doi.org/10.3390/app13127237 doi: 10.3390/app13127237
[89]	Amose J, Manimegalai P, Narmatha C, et al. (2022) Comparative performance analysis of Kernel functions in Support Vector Machines in the diagnosis of pneumonia using lung sounds. Proceedings of 2022 2nd International Conference on Computing and Information Technology, ICCIT 2022, 320–324. https://doi.org/10.1109/iccit52419.2022.9711608 doi: 10.1109/iccit52419.2022.9711608
[90]	Karyawati AE, Wijaya KDY, Supriana IW, et al. (2023) A comparison of different Kernel functions of SVM classification method for spam detection. JITK 8: 91–97. https://doi.org/10.33480/jitk.v8i2.2463 doi: 10.33480/jitk.v8i2.2463
[91]	Munir MA, Khattak A, Imran K, et al. (2019) Solar PV generation forecast model based on the most effective weather parameters. 1st International Conference on Electrical, Communication and Computer Engineering, ICECCE 2019, 24–25. https://doi.org/10.1109/icecce47252.2019.8940664 doi: 10.1109/icecce47252.2019.8940664
[92]	Wang F, Mi Z, Su S, et al. (2012) Short-term solar irradiance forecasting model based on artificial neural network using statistical feature parameters. Energies 5: 1355–1370. https://doi.org/10.3390/en5051355 doi: 10.3390/en5051355
[93]	Kashyap Y, Bansal A, Sao AK (2015) Solar radiation forecasting with multiple parameters neural networks. Renewable Sustainable Energy Rev 49: 825–835. https://doi.org/10.1016/j.rser.2015.04.077 doi: 10.1016/j.rser.2015.04.077
[94]	Sayad S. Support Vector Machine-Regression (SVR). An introduction to data science. Available from: http://www.saedsayad.com/support_vector_machine_reg.htm.
[95]	Lu Y, Roychowdhury V (2008) Parallel randomized sampling for support vector machine (SVM) and support vector regression (SVR). Knowl Inf Syst 14: 233–247. https://doi.org/10.1007/s10115-007-0082-6 doi: 10.1007/s10115-007-0082-6
[96]	Kleynhans T, Montanaro M, Gerace A, et al. (2017) Predicting top-of-atmosphere thermal radiance using MERRA-2 atmospheric data with deep learning. Remote Sens 9: 1133. https://doi.org/10.3390/rs9111133 doi: 10.3390/rs9111133
[97]	Obviously AI: data science without code (2022) Obviously AI Inc. Available from: https://app.obviously.ai/predict.
[98]	Data Science, what is Light GBM? Available from: https://datascience.eu/machine-learning/1-what-is-light-gbm/.
[99]	Mandot P (2017) What is LightGBM, how to implement it? How to fine tune the parameters? Medium. Available from: https://medium.com/@pushkarmandot/https-medium-com-pushkarmandot-what-is-lightgbm-how-to-implement-it-how-to-fine-tune-the-parameters-60347819b7fc.
[100]	Gueymard CA (2014) A review of validation methodologies and statistical performance indicators for modeled solar radiation data: Towards a better bankability of solar projects. Renewable Sustainable Energy Rev 39: 1024–1034. https://doi.org/10.1016/j.rser.2014.07.117 doi: 10.1016/j.rser.2014.07.117
[101]	De Paiva GM, Pimentel SP, Alvarenga BP, et al. (2020) Multiple site intraday solar irradiance forecasting by machine learning algorithms: MGGP and MLP neural networks. Energies 13: 3005. https://doi.org/10.3390/en13113005 doi: 10.3390/en13113005
[102]	Yildirim A, Bilgili M, Ozbek A (2023) One-hour-ahead solar radiation forecasting by MLP, LSTM, and ANFIS approaches. Meteorology Atmospheric Physics, 135. https://doi.org/10.1007/s00703-022-00946-x doi: 10.1007/s00703-022-00946-x
[103]	Huang X, Li Q, Tai Y, et al. (2021) Hybrid deep neural model for hourly solar irradiance forecasting. Renewable Energy 171: 1041–1060. https://doi.org/10.1016/j.renene.2021.02.161 doi: 10.1016/j.renene.2021.02.161
[104]	Mellit A, Pavan AM, Lughi V (2021) Deep learning neural networks for short-term photovoltaic power forecasting. Renewable Energy 172: 276–288. https://doi.org/10.1016/j.renene.2021.02.166 doi: 10.1016/j.renene.2021.02.166
[105]	Bhatt A, Ongsakul W, Nimal Madhu M, et al. (2022) Sliding window approach with first-order differencing for very short-term solar irradiance forecasting using deep learning models. Sustainable Energy Technol Assess, 50. https://doi.org/10.1016/j.seta.2021.101864 doi: 10.1016/j.seta.2021.101864
[106]	Wang J, Zhong H, Lai X, et al. (2019) Exploring key weather factors from analytical modeling toward improved solar power forecasting. IEEE Trans Smart Grid 10: 1417–1427. https://doi.org/10.1109/tsg.2017.2766022 doi: 10.1109/tsg.2017.2766022
[107]	Basak SC, Vracko MG (2020) Parsimony principle and its proper use/application in computer-assisted drug design and QSAR. Curr Comput Aided Drug Des 16: 1–5. https://doi.org/10.2174/157340991601200106122854 doi: 10.2174/157340991601200106122854
[108]	Almekhlafi MAA (2018) Justification of the advisability of using solar energy for the example of the Yemen republic. National University of Civil Defence of Ukraine, 41–50. Available from: http://repositsc.nuczu.edu.ua/handle/123456789/7224.
[109]	Naseri M, Hussaini MS, Iqbal MW, et al. (2021) Spatial modeling of solar photovoltaic power plant in Kabul, Afghanistan. J Mt Sci 18: 3291–3305. https://doi.org/10.1007/S11629-021-7035-5 doi: 10.1007/S11629-021-7035-5
[110]	Elizabeth Michael N, Hasan S, Al-Durra A, et al. (2022) Short-term solar irradiance forecasting based on a novel Bayesian optimized deep long short-term memory neural network. Appl Energy 324: 119727. https://doi.org/10.1016/j.apenergy.2022.119727 doi: 10.1016/j.apenergy.2022.119727
[111]	Safaraliev MK, Odinaev IN, Ahyoev JS, et al. (2020) Energy potential estimation of the region's solar radiation using a solar tracker. Appl Sol Energy 56: 270–275. https://doi.org/10.3103/s0003701x20040118 doi: 10.3103/s0003701x20040118
[112]	Rodríguez-Benítez FJ, Arbizu-Barrena C, Huertas-Tato J, et al. (2020) A short-term solar radiation forecasting system for the Iberian Peninsula. Part 1: Models description and performance assessment. Sol Energy 195: 396–412. https://doi.org/10.1016/j.solener.2019.11.028 doi: 10.1016/j.solener.2019.11.028

This article has been cited by:

1.	Andrei V. Kelarev, Xun Yi, Hui Cui, Leanne Rylands, Herbert F. Jelinek, A survey of state-of-the-art methods for securing medical databases, 2018, 5, 2375-1576, 1, 10.3934/medsci.2018.1.1
2.	Hend Amraoui, Faouzi Mhamdi, Mourad Elloumi, 2019, Chapter 43, 978-3-030-35230-1, 591, 10.1007/978-3-030-35231-8_43
3.	Hend Amraoui, Faouzi Mhamdi, Mourad Elloumi, 2019, Association Rule Mining Using Discrete Jaya Algorithm, 978-1-7281-4484-9, 872, 10.1109/HPCS48598.2019.9188123

Reader Comments

Your name:*

Email:*
© 2024 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)