Evaluating the potential of renewable diesel production from algae cultured on wastewater: techno-economic analysis and life cycle assessment

Ankita Juneja; Ganti S. Murthy; Ankita Juneja; Ganti S. Murthy

doi:10.3934/energy.2017.2.239

AIMS Energy

2017, Volume 5, Issue 2: 239-257. doi: 10.3934/energy.2017.2.239

Previous Article Next Article

Research article Topical Sections

Evaluating the potential of renewable diesel production from algae cultured on wastewater: techno-economic analysis and life cycle assessment

Ankita Juneja ^{1,2
,
,},
Ganti S. Murthy ¹

1.
Biological and Ecological Engineering, Oregon State University, Corvallis OR 97331, USA
2.
Agricultural and Biological Engineering, University of Illinois Urbana Champaign, Urbana IL 61801, USA

Received: 07 December 2016 Accepted: 27 February 2017 Published: 13 March 2017

Algae, a renewable energy source, has an added advantage of consuming nutrients from wastewater and consequently aiding in wastewater treatment. The algae thus produced can be processed using alternative paths for conversion to fuels. However, due to high moisture content of algae, wet algae processing methods are being encouraged to avoid the dewatering cost and energy. Hydrothermal liquefaction is one such technology that converts the algae into high heating value bio-oil under high temperature and pressure. This bio-oil can be further upgraded to renewable diesel (RD) which can be used in diesel powered vehicles without any modifications. The objective of this study is to evaluate the economic viability and to estimate the energy use and greenhouse gas (GHG) emissions during life cycle of RD production from algae grown in wastewater using hydrothermal liquefaction. Economic analysis of RD production on commercial scale was performed using engineering process model of RD production plant with processing capacity of 60 Mgal wastewater/day, simulated in SuperPro designer. RD yields for algae were estimated as 10.18 MML/year with unit price of production as $1.75/RD. The GHG emissions during life cycle of RD production were found to be 6.2 times less than those produced for conventional diesel. Sensitivity analysis indicated a potential to reduce ethanol production cost either by using high lipid algae or increasing the plant size. The integrated economic and ecological assessment analyses are helpful in determining long-term sustainability of a product and can be used to drive energy policies in an environmentally sustainable direction.

Keywords:

Citation: Ankita Juneja, Ganti S. Murthy. Evaluating the potential of renewable diesel production from algae cultured on wastewater: techno-economic analysis and life cycle assessment[J]. AIMS Energy, 2017, 5(2): 239-257. doi: 10.3934/energy.2017.2.239

Related Papers:

[1]	Herbert F. Jelinek, Andrei V. Kelarev . A Survey of Data Mining Methods for Automated Diagnosis of Cardiac Autonomic Neuropathy Progression. AIMS Medical Science, 2016, 3(2): 217-233. doi: 10.3934/medsci.2016.2.217
[2]	Andrei V. Kelarev, Xun Yi, Hui Cui, Leanne Rylands, Herbert F. Jelinek . A survey of state-of-the-art methods for securing medical databases. AIMS Medical Science, 2018, 5(1): 1-22. doi: 10.3934/medsci.2018.1.1
[3]	Isaac Kofi Owusu, Emmanuel Acheamfour-Akowuah, Lois Amoah-Kumi, Yaw Amo Wiafe, Stephen Opoku, Enoch Odame Anto . The correlation between obesity and other cardiovascular disease risk factors among adult patients attending a specialist clinic in Kumasi. Ghana. AIMS Medical Science, 2023, 10(1): 24-36. doi: 10.3934/medsci.2023003
[4]	Frantisek Franek, W. F. Smyth, Xinfang Wang . The Role of The Prefix Array in Sequence Analysis: A Survey. AIMS Medical Science, 2017, 4(3): 261-273. doi: 10.3934/medsci.2017.3.261
[5]	Masoud Nazemiyeh, Mehrzad Hajalilou, Mohsen Rajabnia, Akbar Sharifi, Sabah Hasani . Diagnostic value of Endothelin 1 as a marker for diagnosis of pulmonary parenchyma involvement in patients with systemic sclerosis. AIMS Medical Science, 2020, 7(3): 234-242. doi: 10.3934/medsci.2020014
[6]	Kavin Mozhi James, Divya Ravikumar, Sindhura Myneni, Poonguzhali Sivagananam, Poongodi Chellapandian, Rejili Grace Joy Manickaraj, Yuvasree Sargunan, Sai Ravi Teja Kamineni, Vishnu Priya Veeraraghavan, Malathi Kullappan, Surapaneni Krishna Mohan . Knowledge, attitudes on falls and awareness of hospitalized patient's fall risk factors among the nurses working in Tertiary Care Hospitals. AIMS Medical Science, 2022, 9(2): 304-321. doi: 10.3934/medsci.2022013
[7]	Giuliano Crispatzu, Alexandra Schrader, Michael Nothnagel, Marco Herling, Carmen Diana Herling . A Critical Evaluation of Analytic Aspects of Gene Expression Profiling in Lymphoid Leukemias with Broad Applications to Cancer Genomics. AIMS Medical Science, 2016, 3(3): 248-271. doi: 10.3934/medsci.2016.3.248
[8]	Nicole Lavender, David W. Hein, Guy Brock, La Creis R. Kidd . Evaluation of Oxidative Stress Response Related Genetic Variants, Pro-oxidants, Antioxidants and Prostate Cancer. AIMS Medical Science, 2015, 2(4): 271-294. doi: 10.3934/medsci.2015.4.271
[9]	Manasseh B. Wireko, Jacobus Hendricks, Kweku Bedu-Addo, Marlise Van Staden, Emmanuel A. Ntim, Samuel F. Odoom, Isaac K. Owusu . Alcohol consumption and HIV disease prognosis among virally unsuppressed in Rural KwaZulu Natal, South Africa. AIMS Medical Science, 2023, 10(3): 223-236. doi: 10.3934/medsci.2023018
[10]	Katsiaryna V Gris, Kenzo Yamamoto, Marjan Gharagozloo, Shaimaa Mahmoud, Camille Simard, Pavel Gris, Denis Gris . Exhaustive behavioral profile assay to detect genotype differences between wild-type, inflammasome-deficient, and Nlrp12 knock-out mice. AIMS Medical Science, 2018, 5(3): 238-251. doi: 10.3934/medsci.2018.3.238

Abstract

1. Introduction

With the advent of the era of big data, deep learning technology has become a research hotspot in the field of artificial intelligence. It has shown great advantages in image recognition, speech recognition, natural language processing and other fields. The problem of sequence labeling is the most common problem in natural language. Shao et al. ^[1] assign semantic labels in input sequences, exploiting encoding patterns in the form of latent variables in conditional random fields to capture latent structure in observed data. Lin et al. ^[2] proposed an attentional segmentation recurrent neural network (ASRNN), which relies on a hierarchical attentional neural semi-Markov conditional random field (semi-CRF) model for sequence labeling tasks.

Convolutional neural networks (CNN) have been widely used in computer vision recognition tasks. Djenouri et al. ^[3] proposed a technique for particle clustering for object detection (CPOD), built on top of region-based methods, using outlier detection, clustering, particle swarm optimization (PSO), and deep convolutional networks to identify smart object data. Shao et al. ^[4] proposed an end-to-end multi-objective neuroevolution algorithm based on decomposition and dominance (MONEADD) for combinatorial optimization problems to improve the performance of the model in inference. From 2010 to 2017, the ImageNet Large Scale Visual Recognition Challenge has been held for seven years. The image classification accuracy of the champions has increased from 71.8% to 97.3%. The emergence of AlexNet in 2012 was a milestone in deep learning field. After that, the ImageNet dataset accuracy has been significantly improved by novel CNNs, like VGG ^[5], GoogleNet ^[6], ResNet ^[7,8], DenseNet ^[9], SE-Net ^[10], and automatic neutral architecture search ^[11,12,13].

However, it is necessary to consider high accuracy, platform resources, and the efficiency of systems in real-world applications, e.g., automatic drive systems, intelligent robot systems, and mobile device applications. Moreover, most of the best-performing CNNs need to run on a high-performance graphics processing unit (GPU). So, real-world tasks have driven the development of more lightweight CNNs, to allow CNN to be used in more low-performance devices ^[14,15], like Xception ^[16], MobileNet ^[17], MobileNet V2 ^[18,19], ShuffleNet ^[20], ShuffleNet V2 ^[21] and CondenseNet ^[22]. Group convolution and depth-wise separable convolution ^[23] are crucial in these works.

As the best paper at the CVPR 2017 conference, DenseNet beat the best performing ResNet on ImageNet without group convolution or depth-wise separable convolution. Subsequently, the SE-Net achieved the best results in the history of ImageNet in ILSVRC2017, but there are still too many parameters in SE-Net. Following these works, Huang et al. ^[9] have proposed Learned Group Convolutions to improve DenseNet connection and convolution methods. Inspired by these jobs, we study using Squeeze-and-Excitation block (SE-block) to improve the lightweight CNN. Furthermore, we explore how to design the structure of the convolutional layer to enhance the network's performance.

We propose a more efficient network, CED-Net, which combines bottleneck layer with learned group convolution and SE block. Learned group convolution can crop the network channel during the training phase. And the SE block can recalibrate the feature channel to enhance the channel beneficial to the network. Through experiments, we demonstrate that CED-Net is superior to other lightweight network in terms of accuracy, the number of parameters, and FLOPs.

2. Related works

2.1. Model compression and efficient network architectures

In the past few years, designing CNNs by adjusting an optimal depth to balance accuracy and performance was a very active field. Most recent work has been many progresses in algorithm optimization exploration, including pruning redundant connections ^{[24,25,26,27]}, using low-accuracy or quantized weights ^[28,29], or designing efficient network architectures.

Early researchers proved pruning redundant and quantization are effective methods because deep neural networks often have a substantial number of redundant weights that can be pruned or quantized without sacrificing (and sometimes even improving) accuracy. For CNNs, different pruning techniques may lead to varying levels of granularity ^[30]. Fine-grained pruning, e.g., independent weight pruning ^[31], generally achieves a high degree of sparsity. Coarse grained pruning methods such as filter-level pruning earn a lower degree of sparsity, but the resulting networks are much more regular, facilitating efficient implementations.

Recently researchers have explored the structures of the efficient network that can be applied on mobile devices such as MobileNet V2, ShuffleNet V2, and NasNet. In these networks, depth-wise separable convolutions play a vital role, which can reduce a large number of network parameters without significantly reducing the accuracy. However, according to the Howard et al. ^[17,18], a large amount of depth-wise separable convolutions will decrease the computational speed of the network. Therefore, CED-Net uses a more efficient group convolution and densely connected architecture to reduce the number of parameters of the network. Furthermore, because many deep-learning libraries efficiently implement group convolutions, they save a lot of computational time in theory and practice.

In addition, the bottleneck layer proposed in ResNet can effectively reduce parameters for multilayer network. Our experiments show that CED-Net can achieve higher accuracy and fewer parameters than CondenseNet of the same structure when layers are deeper.

2.2. Learned group convolution and squeeze-and-excitation

Huang et al. ^[9], as the best paper for CVPR2017, proposed a densely connection network that is better than the previous champion ResNet on the ImageNet. After that, CondenseNet achieved the same accuracy with only half of the number of parameters of DenseNet. In CondenseNet, learned group convolution plays a key role; it can train the network with sparsity inducing regularization for a fixed number of iterations. Subsequently, it prunes away unimportant filters with low magnitude weights. Because many deep-learning libraries efficiently implement group convolutions, they save a lot of computational time in theory and practice.

Moreover, the Squeeze-and-Excitation structure that shines on ILSVRC2017 has been experimented on by most famous networks. Squeeze and Excitation are two very critical operations. First, it is used to model the interdependencies between feature channels explicitly. It is a new "channel recalibration" strategy. Specifically, by automatically learning the importance of each feature channel, SE-Net enhances the proper channel and suppresses useless channels. Most of the current mainstream networks are constructed based on superimposed basic blocks. It can be seen that the SE module can be embedded in almost all network structures, so CED-Net achieves more efficient performance by embedding the SE module.

3. Channel enhancement dense network

In this section, we first introduce the structure and function of the bottleneck layer. Next, we explore how SE Block as a channel enhancement block can improve the performance of CED-Net. Finally, we describe the network details of CED-Net for CIFAR dataset.

As shown in , H, W, ${C}_{in}$ are the height, width, and the number of channels of the input image, respectively, and g is the growth coefficient of the channel. CED-Net consists of multiple dense blocks for feature extraction. The dense block is shown in Figure 2(c). It consists of two 1 × 1 LG-Conv (Learned Group Convolution) layers and one 3 × 3 G-Conv (Group Convolution) layer. Each 1 × 1 LG-Conv layer uses a permute operation for channel shuffling to reduce accuracy. BN-ReLU nonlinearly activates the input and output in the dense block. And use the AvgPool layer for down sampling.

Figure 1. A 5-layer dense block with channel enhancement and bottleneck layer.

DownLoad: Full-Size Img PowerPoint

Figure 2. Different networks' bottleneck layer or dense block. (a) ResNet. (b) CondenseNet. (c) CED-Net.

DownLoad: Full-Size Img PowerPoint

3.1. Bottleneck layer with learned group convolution

The bottleneck layer is proposed in ResNet, and the detailed structure is shown in Figure 2(a). The three-layer bottleneck structure consists of 1 × 1, 3 × 3, and 1 × 1 convolutional layers, where two 1 × 1 convolutions are used to reduce and increase (restore) dimensions. The 3 × 3 convolutional layer can be seen as a bottleneck for a smaller input/output dimension. We replace the 1 × 1 standard convolution with the learned group convolution, and the 3 × 3 standard convolution is replaced with the group convolution. Unlike ResNet, the CED-Net replaces element-wise addition with channel concatenation. Because it can use the semantic information of different scale feature maps to achieve better performance by increasing the channel, the element addition operation does not take up too much memory during network transmission. Still, it may introduce extra noise that will lose some feature map information.

Figure 2(b) shows the structure used in CondenseNet. The Permute layer, enabling shuffling between channels, is designed to reduce the adverse effects of the introduction of 1 × 1 LG-Conv. But there are still many parameters in a deep network with the bottleneck layer. Figure 2(c) shows part of the structure used by CED-Net. This structure has fewer parameters than that in Figure 2(b). Expressly, the condense factor and bottleneck factor in CED-Net are set to 4 and reduced by half compared to CondenseNet. This is to reduce the parameters caused by adding a 1 × 1 LG-Conv layer.

One dense layer used in CED-Net is of quadratic time complexity (Θ(25G²/4+4CG)) concerning the number (C) of input channels and the number (G) of output channels. Compared with ordinary 3 × 3 convolution (Θ(9CG)), as a result of C is much greater than G with the deepening of network layers, CED-Net reduces the time complexity by half.

Figure 3 shows how channels change the process of the bottleneck layer based on learning group convolution. The parameters and calculation amount are 1/4 of the standard bottleneck layer. Based on the image classification comparing experiments on the CIFAR dataset, we can conclude that our structure can increase the classification accuracy by 0.4% when the number of parameters and the amount of calculation is almost the same as CondenseNet (see Section 4). When network layers are deeper (depth is 272), the number of parameters and the amount of calculation of CED-Net are smaller than the CondenseNet of the same depth. Still, the classification accuracy is higher than that of CondenseNet.

Figure 3. Bottleneck layer with Learned Group Convolutions.

DownLoad: Full-Size Img PowerPoint

3.2. Channel enhancement with Squeeze and Excitation

In CED-Net, since the network is a densely connected structure, the input data of each convolution layer has a large amount of channel information. And the output after convolution is the sum of all previous channel information. This has led to the entanglement of information and spatial relevance. Furthermore, in lightweight networks, group convolution can significantly reduce the amount of computation by ensuring that each convolution operation is only on the corresponding input channel group. However, if multiple sets of convolutions are stacked together, there is a side effect: A channel output is only derived from a few numbers of input channels. This would reduce the information flow between channel groups and express information.

Therefore, we use the channel permute (see Figure 2(c)) and the Squeeze-and-Excitation block to make the information between the groups more circulated to allow the network to focus on more helpful information. As shown in Figure 4, Squeeze-and-Excitation blocks can improve the representation of the network by increasing the interdependence between convolution feature channels. The detailed process is divided into two steps: Squeeze and Excitation.

Figure 4. Channel Enhancement with Squeeze-and-Excitation.

DownLoad: Full-Size Img PowerPoint

Squeeze. CNNs all have the problem that due to the nature of convolutional calculations, each convolution filter can only focus on specific spatial information. To alleviate this problem, the Squeeze, as a global description operation, encodes the global spatial information into the channel descriptor and calculates the mean of each channel through global average pooling.

${z}_{c} = {F}_{sq}\left({u}_{c}\right) = \frac{1}{W\times H}\sum _{i = 1}^{W}\sum _{j = 1}^{H}{u}_{c}(i, j)$

(1)

As shown in Eq (1), where ${\mathrm{Z}}_{\mathrm{c}}$ is the output of the squeeze layer, W, H are the width and height of the input feature map of the current layer. ${u}_{c}$ is the input feature map, and ${F}_{sq}\left(\mathrm{*}\right)$ can represent the global information of the entire feature map. The global average pooling used in this paper squeezes the feature map into a value to indicate the importance of the corresponding channel.

Excitation. To take advantage of the information obtained by the squeeze operation, the excitation operation needs to meet two criteria to achieve full capture of channel dependencies. First, it must be able to learn nonlinear interactions between channels. And second, it must learn a non-mutually exclusive relationship. Specifically, the gate mechanism is parameterized by concatenating two fully connected (FC) layers above and below the nonlinear (ReLU) and then activated with the sigmoid function.

$s = {F}_{ex}\left(z, W\right) = \sigma \left(g\left(z, W\right)\right) = \sigma \left({W}_{2}\theta \left({W}_{1}z\right)\right)$

(2)

where $\mathrm{\theta }$ is the ReLU function. ${\mathrm{W}}_{1}\in {\mathrm{R}}^{\frac{\mathrm{C}}{\mathrm{r}}\times \mathrm{C}}$ , ${\mathrm{W}}_{2}\in {\mathrm{R}}^{\mathrm{C}\times \frac{\mathrm{C}}{\mathrm{r}}}$ are the weights of the dimensionality reduction layer and the dimensionality increase layer, respectively. Where r is the dimensionality reduction rate, and C is the number of channels. To limit the complexity of the model and increase the generalization, a "bottleneck" is formed by a two-layer FC layer around a nonlinear map, where r sets 16. Finally, after obtaining the so-called gate, by multiplying the channel gates by the corresponding feature maps, you can control the flow of information for each feature map.

We embed the Squeeze-and-Excitation block into the 3 × 3 G-Conv layer because the number of input/output feature channels in the first 3 × 3 G-Conv is the same and smaller. The Squeeze-and-Excitation block can effectively enhance the effective channel after feature extraction without extra parameters. According to the research results of Hu et al., this method can balance the accuracy of the model and the number of parameters.

Algorithm 1 Image classification based on CED-Net
Input: In = datasets ( ${\boldsymbol{x}}_\mathbf{1}, {\boldsymbol{y}}_\mathbf{1}$ ), ( ${\boldsymbol{x}}_\mathbf{2}, {\boldsymbol{y}}_\mathbf{2}$ ), …, ( ${\boldsymbol{x}}_{\boldsymbol{m}}, {\boldsymbol{y}}_{\boldsymbol{m}}$ )
Output: Op = Classification accuracy: ( ${\boldsymbol{y}}_{1}, {\boldsymbol{y}}_\mathbf{2}, \dots, {\boldsymbol{y}}_{\boldsymbol{n}}$ )
Set: CED-Net feature extraction: ${\boldsymbol{G}}_{\boldsymbol{k}}(·)$ , k $\in$ (0, n)
for x = 1 : m do
$\boldsymbol{S}\boldsymbol{o}\boldsymbol{f}\boldsymbol{t}\boldsymbol{m}\boldsymbol{a}\boldsymbol{x}\left({\boldsymbol{G}}_{\boldsymbol{k}}\left({\boldsymbol{x}}_{\boldsymbol{i}}\right)\right) = \frac{{\boldsymbol{e}}^{{\boldsymbol{g}}_{\boldsymbol{i}}}}{{\sum }_{\boldsymbol{k}}^{\boldsymbol{n}}{\boldsymbol{e}}^{{\boldsymbol{g}}_{\boldsymbol{k}}}}$
i $\in$ [1, m], where ${\boldsymbol{g}}_{\boldsymbol{i}}$ is one class value in ${\boldsymbol{G}}_{\boldsymbol{k}}(·)$ .
Return Op
end for

3.3. Network design

CED-Net can guarantee good performance while maintaining lightweight models because of the effective combination of bottleneck layer structure and channel enhancement blocks. An important difference between CED-Net and other network architectures is that CED-Net has a very narrow layer. The relatively small channel growth rate is sufficient to obtain the most advanced results on the test dataset. This can increase the proportion of features from the later layers relative to features from the previous layers. So, we set the channel growth rate of a dense connection layer to 4. And we found that if the number of early layers is set too deep, it will significantly increase the FLOPs of the network.

Architectural details. The model used in our experiments has three dense blocks. Before the data enters the first dense block, the input image would go through a 3 × 3 standard convolution which output channels are 16 and stride size is 2. In the dense layer, the number of channel enhancement blocks should be set according to the growth rate, the input channels, and the output channels, see Eq (3).

$n = \frac{Cout-Cin}{g}$

(3)

where g is the growth rate, ${\mathrm{C}}_{\mathrm{i}\mathrm{n}}$ is the input channels, n is the number of channel enhancement blocks, and ${\mathrm{C}}_{\mathrm{o}\mathrm{u}\mathrm{t}}$ is the output channels. For example, in the experiment, we set the growth rate to 8, 16, and 32, and the channels of dense layer output is 256, 756, and 1696 respectively, so the number of channel enhancement blocks in the dense layer are all 30.

For each convolutional layer with a kernel size of 3 × 3, each side of the input is zeros-padded to keep the feature size fixed. In general, we add the batch normalization layer and the ReLU function after the last dense layer and then use the global average pooling to compress the feature map into one dimension as the input of the Softmax layer. The exact network configuration is shown in Tables 1 and 2.

Table 1. Network structure of CED-Net on CIFAR.

Layers	Output Size	Output Channels	Repeat	Stride
3 × 3 Convolution	32 × 32	16	1	1
Dense bottleneck block	32 × 32	256 (g = 8)	30	1
Avg pooling	16 × 16	256 (g = 8)	1	2
Dense bottleneck block	16 × 16	736 (g = 16)	30	1
Avg pooling	8 × 8	736 (g = 16)	1	2
Dense bottleneck block	8 × 8	1696 (g = 32)	30	1
Global avg pooling	1 × 1	1696	1	8
Fully connected	1 × 1	10	1

| Show Table

DownLoad: CSV

Table 2. Network structure of CED-Net on ImageNet.

Layers	Output Size	Output Channels	Repeat	Stride
3 × 3 Convolution	112 × 112	64	1	2
Dense bottleneck block	112 × 112	96 (g = 8)	4	1
Avg pooling	56 × 56	96 (g = 8)	1	2
Dense bottleneck block	56 × 56	192 (g = 16)	6	1
Avg pooling	28 × 28	192 (g = 16)	1	2
Dense bottleneck block	28 × 28	448 (g = 32)	8	1
Avg pooling	14 × 14	448 (g = 32)	1	2
Dense bottleneck block	14 × 14	1088 (g = 64)	10	1
Avg pooling	7 × 7	1088 (g = 64)	1	2
Dense bottleneck block	7 × 7	2112 (g = 128)	8	1
Global avg pooling	1 × 1	2112	1	7
Fully connected	1 × 1	1000	1

| Show Table

DownLoad: CSV

The training process of CED-Net is shown in Algorithm 1. (x_i, y_i) in the input represent the images and label of the ith batch respectively. For each batch, we use softmax to obtain the output Yi of CED-Net. Finally, the image features G_k of n categories are obtained.

4. Experiments

This section conducted experiments on the CIFAR10, CIFAR-100, and the ImageNet (ILSVRC 2012) datasets. First, we compared them with other advanced convolutional neural networks, such as VGG16, ResNet-101, and DenseNet. Then, we conducted ablation experiments to CED-Net, mainly comparing three networks, the primary network of CED-Net-128, the optimization network with only the bottleneck layer, and the network with only the channel enhancement block. Through these experiments, we verify the effectiveness of our improved method. Next, we will introduce the data set and the evaluation indicators of the experiment.

4.1. Dataset

The CIFAR-10 and CIFAR-100 datasets consist of colored natural images with 32 × 32 pixels. CIFAR-10 consists of images drawn from 10 classes and CIFAR-100 from 100 classes. The training and test sets contain 50, 000 and 10, 000 images, respectively, and we picked up 5000 training images as a validation set. We adopt a standard data augmentation scheme (mirroring/shifting) and image zero-padded with 4 pixels per side, and then randomly cropped to generate a 32 × 32 image. The image is flipped horizontally at a probability of 0.5 and normalized by subtracting the channel average and dividing by the channel standard deviation.

The ImageNet datasets consist of 224 × 224 pixels colored natural images with 1000 classes. The training and validation sets contain 1, 280, 000 and 50, 000 images, respectively. We adopt the data-augmentation scheme at training time and perform a rescaling to 256 × 256 followed by a 224 × 224 center crop at test time before feeding the input image into the networks.

4.2. Evaluation criteria

We evaluate CED-Net on three criteria:

4.2.1. Classification accuracy

Accuracy is the most common metric. It is the number of samples that are paired divided by the number of all samples. Generally speaking, the higher the accuracy is, the better the classifier will be:

$accuracy = \left(TP+TN\right)/\left(P+N\right)$

(4)

where P (positive) is the number of positive examples in the sample, and N (negative) is the number of negative examples. TP (true positives) is the number of samples that are positive examples that are correctly classified. TN (true negatives) is the number of samples that are actually negative that are correctly classified.

4.2.2. Model parameter

For a single convolutional kernel we have:

$parameters = {k}^{2}\times Cin\times Cout$

(5)

where 𝑘 is the convolution filter's size, 𝐶in is the input channels, and 𝐶out is the output channels;

4.2.3. FLOPs (floating-point operations)

To measure the amount of calculation of the model, we compute the number of FLOPs of each layer. For convolutional kernels, we have:

$FLOPs = 2HW({k}^{2}Cin+ 1)Cout$

(6)

where 𝐻, 𝑊 are height and width. For fully connected layers, we compute FLOPs as:

$FLOPs = (2I- 1)O$

(7)

where 𝐼 is the input dimensionality and 𝑂 is the output dimensionality.

4.2.4. Other criteria

To further prove the stability of CED-Net, we added the interpretation and comparison of precision, recall and F-measure in the ablation experiment:

$precision = TP/\left(TP+FP\right)$

(8)

$recall = TP/\left(TP+FN\right)$

(9)

$F-measure = 2*precision*recall/\left(precision+recall\right)$

(10)

4.3. The result of image classification

4.3.1. Training details

We train all models with stochastic gradient descent (SGD) using similar optimization hyper-parameters ^{[23,24,25,26,27,28,29,30]}. And we set the Nesterov momentum weight to 0.9 without damping and use a weight decay of 0.0001. All models are trained with mini-batch size 128 for 200 epochs on the training datasets. We use the cosine annealing learning rate curve, starting from 0.1 and gradually reducing to 0.

4.3.2. Results on CIFAR

In this part, we train CED-Net and other advanced convolutional neural networks on the CIFAR-10 and CIFAR100 datasets. We compared these models under the above three evaluation criteria. See Table 3 for a detailed list.

Table 3. The classification accuracy on CIFAR-10 and CIFAR-100.

Model	Params	FLOPs	CIFAR-10	CIFAR-100
VGG-16	14.73 M	314 M	92.64	72.23
ResNet-101	42.51 M	2515 M	93.75	77.78
ResNeXt-29	9.13 M	1413 M	94.82	78.83
MobileNet V2	2.30 M	92 M	94.43	68.08
DenseNet-121	6.96 M	893 M	94.04	77.01
CondenseNet-86	0.52 M	65 M	94.48	76.36
CondenseNet-182	4.20 M	513 M	95.87	80.13
CED-Net-128 CED-Net-272	0.69 M 5.32 M	75 M 649 M	94.89 96.31	77.35 80.72

| Show Table

DownLoad: CSV

In Table 3, we show the results of comparing 128-layer CED-Net and 272-layer CED-Net with other state-of-the-art CNN architectures. All models were trained in 200 epochs in the experiment. The results show that after introducing the bottleneck layer structure and channel enhancement blocks to CED-Net, the CondenseNet increases the accuracy by 0.4–0.5% with minimal parameters and FLOPs cost compared with the same number of stacked blocks n datasets. Moreover, compared to the more advanced MobileNet V2, CED-Net is more accurate without using depth-wise separable convolutions. And the parameter amount is 1/4 of it, and the FLOPs are also more minor.

4.3.3. Results on ImageNet

In this part, we train CED-Net and other advanced convolutional neural networks on the ImageNet datasets. We compared these models under the above four evaluation criteria. See Table 4 for a detailed list.

Table 4. The classification accuracy on ImageNet.

Model	Params	FLOPs	Top-1	Top-5
VGG-16	138.36 M	15.48 G	71.93	90.67
ResNet-101	44.55 M	7.83 G	80.13	95.4
MobileNet V2	3.5 M	0.3 G	71.8	91
DenseNet-121	7.98 M	2.87 G	74.98	92.29
CondenseNet	4.8 M	0.53 G	73.8	91.7
SE-Net	115 M	20.78 G	81.32	95.53
CED-Net-115	9.3 M	1.13 G	78.65	93.7

| Show Table

DownLoad: CSV

In Table 4, we show the results of comparing 115-layer CED-Net with other CNN architectures. The results show that the accuracy of Top-1 and Top-5 is improved by 4.85 and 2%, respectively, compared with the same depth of CondenseNet. At the same time, the dense bottleneck block used in the CED net is more complex. Compared with DenseNet, CED-Net increases the number of parameters by 16.5% but reduces the amount of calculation by 39.4%; the accuracies of Top-1 and top-5 are improved by 3.67 and 1.41%, respectively. Compared with SE-Net, CED-Net reduces the Top-1 accuracy by 2.67%, but the parameter quantity is only 8.1% of SE-Net.

Some misclassification images are shown in Figure 5. There may be unavoidable interference information in these pictures; Also, it may be that the network model constructed in this paper does not learn a sufficient number of diverse features and cannot correctly identify each picture with different features.

Figure 5. ImageNet misclassified pictures.

DownLoad: Full-Size Img PowerPoint

4.3.4. Redundant LG-Conv

In the dense bottleneck block shown in Figure 2, we use the learned group revolution before and after the 3 × 3 group convolution, which means that there are two consecutive learned group convolutions between the two 3 × 3 group convolutions. The two index layers used have redundancy, but we think it is necessary. These redundancies can improve the learned group revolution's generalization performance and help subsequent feature extraction. But this design increases the amount of calculation and parameters of the intermediate convolution.

4.3.5. Ablation experiments

In this part, we performed a CED-Net ablation experiment. We trained four models on the CIFAR-10 dataset, CEDNet-128a with no bottleneck layer and channel enhancement block, CED-Net-128b with convolutional layer structure changed to bottleneck layer, CED-Net-128c with channel enhancement block based on CondenseNet-86 and CED-Net-128 that we proposed in this paper.

In Table 5, CondenseNet-86 is our basic model. It can be seen that when we turn the structure of CondenseNet into a bottleneck layer, the parameters and FLOPs of the network are only slightly improved, and the accuracy can be increased by about 0.3%. When we added the channel enhancement block to CondenseNet-86, we saw not much increase in FLOPs. But the parameters are raised, and the accuracy can be improved by about 0.3%. In our CED-Net-128, the accuracy rate has been significantly improved, and the channel enhancement block mainly causes the increase in parameters. The bottleneck layer structure causes an increase in FLOPs. In addition, the Accuracy, Precision, Recall and F-measure of each model are very close, which prove that the four models have extracted stable features.

Table 5. The result of ablation experiments of CIFAR-10.

Model	Params	FLOPs	Accuracy	Precision	Recall	F-measure
CondenseNet^a	0.52M	65.82M	94.48	94.50	94.48	94.49
CED-Net-128^b	0.59M	75.04M	94.75	94.75	94.75	94.75
CED-Net-128^c	0.66M	67.04M	94.74	94.76	94.74	94.75
CED-Net-128	0.69M	75.41M	94.89	94.89	94.89	94.88
Note: ^aThe basic model of CED-Net same as CondenseNet-86 without bottleneck layer and channel enhancement block; ^bThe basic model of CED-Net only add a bottleneck layer; ^cThe basic model of CED-Net only add channel enhancement block.

| Show Table

DownLoad: CSV

5. Conclusions

This paper introduces CED-Net: a more efficient densely concatenated convolutional neural network based on feature enhancement block and bottleneck layer structure, which increases accuracy by learning group convolution and feature reuse. To make the reasoning effective, the pruned network can be converted to a network with conventional group convolution, which is effectively implemented in most deep learning libraries. In our experiments, CED-Net outperformed its underlying network CondenseNet and other advanced convolutional neural networks such as Mobilenet V2 and ResNeXt in terms of computational efficiency at the same accuracy level. Moreover, CED-Net has a much simpler structure with higher accuracy. We anticipate further research in CED-Net to combine this framework to the Neural Architecture Search (NAS), so as to design more lightweight Convolutional Neural Network models. We hope our work will draw more attention toward a broader view of using lightweight architecture for deep learning.

Acknowledgments

This work was supported by the National Natural Science Foundation of China (No. 61976217), the Opening Foundation of Key Laboratory of Opto-technology and Intelligent Control, Ministry of Education (KFKT2020-3), the Fundamental Research Funds of Central Universities (No. 2019XKQ YMS87), Science and Technology Planning Project of Xuzhou (No. KC21193).

Conflict of interest

The authors declare there is no conflict of interest.

References

[1]	Ndong R, Montrejaud VM, Saint GO, et al. (2009) Life cycle assessment of biofuels from Jatropha curcas in West Africa: a field study. GCB Bioenerg 1: 197–210. doi: 10.1111/j.1757-1707.2009.01014.x
[2]	Chisti Y (2007) Biodiesel from microalgae. Biotechnol adv 25: 294–306. doi: 10.1016/j.biotechadv.2007.02.001
[3]	Becker EW (1994) Microalgae: biotechnology and microbiology, Cambridge University Press, 10: 165.
[4]	Adam F, Abert VM, Peltier G, et al. (2012) "Solvent-free" ultrasound-assisted extraction of lipids from fresh microalgae cells: a green, clean and scalable process. Bioresource Technol 114: 457–465. doi: 10.1016/j.biortech.2012.02.096
[5]	Tanzi CD, Vian MA, Chemat F (2013) New procedure for extraction of algal lipids from wet biomass: a green clean and scalable process. Bioresource Technol 134: 271–275. doi: 10.1016/j.biortech.2013.01.168
[6]	Cheng J, Yu T, Li T, et al. (2013) Using wet microalgae for direct biodiesel production via microwave irradiation. Bioresource Technol 131: 531–535. doi: 10.1016/j.biortech.2013.01.045
[7]	Halim R, Gladman B, Danquah MK, et al. (2011) Oil extraction from microalgae for biodiesel production. Bioresource Technol 102: 178–185. doi: 10.1016/j.biortech.2010.06.136
[8]	Zhu Y, Biddy MJ, Jones SB, et al. (2014) Techno-economic analysis of liquid fuel production from woody biomass via hydrothermal liquefaction (HTL) and upgrading. Appl Energ 129: 384–394. doi: 10.1016/j.apenergy.2014.03.053
[9]	Akhtar J, Amin NAS (2011) A review on process conditions for optimum bio-oil yield in hydrothermal liquefaction of biomass. Renew Sust Energ Rev 15: 1615–1624. doi: 10.1016/j.rser.2010.11.054
[10]	Brown TM, Duan P, Savage PE (2010) Hydrothermal liquefaction and gasification of Nannochloropsis sp. Energ Fuel 24: 3639–3646.
[11]	Peterson AA, Vogel F, Lachance RP, et al. (2008) Thermochemical biofuel production in hydrothermal media: a review of sub-and supercritical water technologies. Energ Environ Sci 1: 32–65. doi: 10.1039/b810100k
[12]	Delrue F, Li-Beisson Y, Setier PA, et al. (2013) Comparison of various microalgae liquid biofuel production pathways based on energetic, economic and environmental criteria. Bioresource Technol 136: 205–212.
[13]	Thomas E, David WR, Timothy RG (2010) California renewable diesel multimedia evaluation, The University of California, Davis, Berkeley.
[14]	Amer L, Adhikari B, Pellegrino J (2011) Technoeconomic analysis of five microalgae-to-biofuels processes of varying complexity. Bioresource Technol 2011: 9350–9359.
[15]	Davis R, Aden A, Pienkos PT (2011) Techno-economic analysis of autotrophic microalgae for fuel production. Appl Energ 88: 3524–3531. doi: 10.1016/j.apenergy.2011.04.018
[16]	Ahmad F, Khan AU, Yasar A (2013) The potential of Chlorella vulgaris for wastewater treatment and biodiesel production. Pakistan J Bot 45: 461–465.
[17]	Tebbani S, Filali R, Lopes F, et al. (2014) CO₂ biofixation by Microalgae: automation process, John Wiley & Sons.
[18]	Biller P, Ross A (2011) Potential yields and properties of oil from the hydrothermal liquefaction of microalgae with different biochemical content. Bioresource Technol 102: 215–225. doi: 10.1016/j.biortech.2010.06.028
[19]	Wett B, Buchauer K, Fimml C (2007) In energy self-sufficiency as a feasible concept for wastewater treatment systems, IWA Leading Edge Technology Conference, Singapore, Asian Water, 21–24.
[20]	Borowitzka M (2005) Culturing microalgae in outdoor ponds, In: algal culturing techniques, Andersen, Ed. Academic Press, NY, USA, 205–217.
[21]	Jones S, Davis R, Zhu Y, et al. (2014) Process design and economics for the conversion of algal biomass to hydrocarbons: whole algae hydrothermal liquefaction and upgrading, Department of Energy Bioenergy Technologies Office, US.
[22]	Lundquist TJ, Woertz IC, Quinn N, et al. (2011) A realistic technology and engineering assessment of algae biofuel production. Energ Biosci I, 1–153.
[23]	Jonker J, Faaij A (2013) Techno-economic assessment of micro-algae as feedstock for renewable bio-energy production. Appl Energ 102: 461–475.
[24]	Sheehan J, Dunahay T, Benemann J, et al. (1998) A look back at the US department of energy's aquatic species program: biodiesel from algae, National Renewable Energy Laboratory Golden, CO, 328.
[25]	Ellis TG (2004) Chemistry of wastewater, Encyclopedia of Life Support System (EOLSS).
[26]	Chen Y, Liu J, Ju YH (1998) Flotation removal of algae from water. Colloid Surface B 12: 49–55. doi: 10.1016/S0927-7765(98)00059-9
[27]	Hansel PA (2014) Efficient flocculation of microalgae for biomass production using cationic starch. Algal Res 5: 133–139. doi: 10.1016/j.algal.2014.07.002
[28]	Jazrawi C, Biller P, Ross AB (2013) Pilot plant testing of continuous hydrothermal liquefaction of microalgae. Algal Res 2: 268–277. doi: 10.1016/j.algal.2013.04.006
[29]	Jena U, Das K, Kastner J (2011) Effect of operating conditions of thermochemical liquefaction on biocrude production from Spirulina platensis. Bioresource Technol 102: 6221–6229. doi: 10.1016/j.biortech.2011.02.057
[30]	Berglin EJ, Enderlin CW, Schmidt AJ (2012) Review and assessment of commercial vendors/options for feeding and pumping biomass slurries for hydrothermal liquefaction, Pacific Northwest National Laboratory, Office of Scientific & Technical Information Technical Reports.
[31]	Garcia AL, Vos MP, Torri C, et al. (2013) Recycling nutrients in algae biorefinery. Chem Sus Chem 6: 1330–1333. doi: 10.1002/cssc.201200988
[32]	Faeth JL, Valdez PJ, Savage PE (2013) Fast hydrothermal liquefaction of Nannochloropsis sp. to produce biocrude. Energ Fuel 27: 1391–1398.
[33]	Holliday RL, King JW, List GR (1997) Hydrolysis of vegetable oils in sub-and supercritical water. Ind Eng Chem Res 36: 932–935.
[34]	Jena U, Vaidyanathan N, Chinnasamy S (2011) Evaluation of microalgae cultivation using recovered aqueous co-product from thermochemical liquefaction of algal biomass. Bioresource Technol 102: 3380–3387. doi: 10.1016/j.biortech.2010.09.111
[35]	Elliott DC, Biller P, Ross AB (2014) Hydrothermal liquefaction of biomass: developments from batch to continuous process. Bioresource Technol 178: 147–156.
[36]	Elliott DC, Neuenschwander GG, Hart TR (2009) Catalytic hydrothermal gasification of lignin-rich biorefinery residues and algae, Pacific Northwest National Laboratory Pnnl.
[37]	Elliott DC (2008) Catalytic hydrothermal gasification of biomass. Biofuel Bioprod Bioref 2: 254–265. doi: 10.1002/bbb.74
[38]	Cruz FE, Oliveira JS (2008) Petroleum refinery hydrogen production unit: exergy and production cost evaluation. Int J Thermodyn 11: 187–193.
[39]	Kim S, Dale BE (2002) Allocation procedure in ethanol production system from corn grain I system expansion. Int J Life Cy Assess 7: 237–243. doi: 10.1007/BF02978879
[40]	Juneja A, Kumar D, Murthy GS (2013) Economic feasibility and environmental life cycle assessment of ethanol production from lignocellulosic feedstock in Pacific Northwest US. J Renew Sust Energ 5: 023142. doi: 10.1063/1.4803747
[41]	Frank ED, Elgowainy A, Han J, et al. (2013) Life cycle comparison of hydrothermal liquefaction and lipid extraction pathways to renewable diesel from algae. Mitig Adapt Strat Gl 18: 137–158. doi: 10.1007/s11027-012-9395-1
[42]	King J, Holliday R, List G (1999) Hydrolysis of soybean oil. in a subcritical water flow reactor. Green Chem 1: 261–264.
[43]	Juneja A, Ceballos RM, Murthy GS (2013) Effects of environmental factors and nutrient availability on the biochemical composition of algae for biofuels production: a review. Energies 6: 4607–4638.
[44]	Sun A, Davis R, Starbuck M, et al. (2011) Comparative cost analysis of algal oil production for biofuels. Energy 36: 5169–5179. doi: 10.1016/j.energy.2011.06.020
[45]	Benemann JR, Oswald WJ (1996) Systems and economic analysis of microalgae ponds for conversion of CO₂ to biomass, Final report, California Univ., Berkeley, CA, Dept. of Civil Engineering.
[46]	Xiang X (2013) Techno-economic analysis of algal lipid fuels, Dissertation, Oregon State University.
[47]	Richardson JW, Johnson MD, Outlaw JL (2012) Economic comparison of open pond raceways to photo bio-reactors for profitable production of algae for transportation fuels in the Southwest. Algal Res 1: 93–100. doi: 10.1016/j.algal.2012.04.001
[48]	Palou RI, Wang MQ (2010) Updated estimation of energy efficiencies of US petroleum refineries, Argonne National Laboratory, US.
[49]	Frank E, Han J, Palou RI, et al. (2011) Life-cycle analysis of algal lipid fuels with the greet model, Center for Transportation Research, Energy Systems Division, Argonne National Laboratory, Oak Ridge.
[50]	Jorquera O, Kiperstok A, Sales EA (2010) Comparative energy life-cycle analyses of microalgal biomass production in open ponds and photobioreactors. Bioresource Technol 101: 1406–1413.
[51]	Weissman JC, Goebel R (1987) Design and analysis of microalgal open pond systems for the purpose of producing fuels: a subcontract report, Solar Energy Research Inst., Golden, CO., USA.
[52]	Xu X, Song C, Wincek R, et al. (2003) Separation of CO₂ from power plant flue gas using a novel CO₂ "molecular basket" adsorbent. Fuel Chem Div Prepr 48: 162–163.

This article has been cited by:

1.	Andrei V. Kelarev, Xun Yi, Hui Cui, Leanne Rylands, Herbert F. Jelinek, A survey of state-of-the-art methods for securing medical databases, 2018, 5, 2375-1576, 1, 10.3934/medsci.2018.1.1
2.	Hend Amraoui, Faouzi Mhamdi, Mourad Elloumi, 2019, Chapter 43, 978-3-030-35230-1, 591, 10.1007/978-3-030-35231-8_43
3.	Hend Amraoui, Faouzi Mhamdi, Mourad Elloumi, 2019, Association Rule Mining Using Discrete Jaya Algorithm, 978-1-7281-4484-9, 872, 10.1109/HPCS48598.2019.9188123

Reader Comments

Your name:*

Email:*
© 2017 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)