Figure 1.
This is a figure. Schemes follow the same formatting. The characteristics of the surface defects of strip steel. (a) Low contrast quality. (b) Different textural and geometric structures. (c) Diversity of defects.
Citation: Siddhartha Kundu. Mathematical model of a short translatable G-quadruplex and an assessment of its relevance to misfolding-induced proteostasis[J]. Mathematical Biosciences and Engineering, 2020, 17(3): 2470-2493. doi: 10.3934/mbe.2020135
| [1] | Zhe Zou, Minjie Zheng . Design and stabilization analysis of luxury cruise with dynamic positioning systems based on sampled-data control. Mathematical Biosciences and Engineering, 2023, 20(8): 14026-14045. doi: 10.3934/mbe.2023626 |
| [2] | Bin Hang, Beibei Su, Weiwei Deng . Adaptive sliding mode fault-tolerant attitude control for flexible satellites based on T-S fuzzy disturbance modeling. Mathematical Biosciences and Engineering, 2023, 20(7): 12700-12717. doi: 10.3934/mbe.2023566 |
| [3] | Shuaiwen Sun, Zhijing Xu . Large kernel convolution YOLO for ship detection in surveillance video. Mathematical Biosciences and Engineering, 2023, 20(8): 15018-15043. doi: 10.3934/mbe.2023673 |
| [4] | Xueyan Wang . A fuzzy neural network-based automatic fault diagnosis method for permanent magnet synchronous generators. Mathematical Biosciences and Engineering, 2023, 20(5): 8933-8953. doi: 10.3934/mbe.2023392 |
| [5] | Wei Zou, Yanxia Shen, Lei Wang . Design of robust fuzzy iterative learning control for nonlinear batch processes. Mathematical Biosciences and Engineering, 2023, 20(11): 20274-20294. doi: 10.3934/mbe.2023897 |
| [6] | Lixiang Zhang, Yian Zhu, Jie Ren, Wei Lu, Ye Yao . A method for detecting abnormal behavior of ships based on multi-dimensional density distance and an abnormal isolation mechanism. Mathematical Biosciences and Engineering, 2023, 20(8): 13921-13946. doi: 10.3934/mbe.2023620 |
| [7] | Yue Song, Yi Zhang, Song Yang, Na Li . Investigation on stability and controller design for singular bio-economic systems with stochastic fluctuations. Mathematical Biosciences and Engineering, 2021, 18(3): 2991-3005. doi: 10.3934/mbe.2021150 |
| [8] | Qianhong Zhang, Fubiao Lin, Xiaoying Zhong . On discrete time Beverton-Holt population model with fuzzy environment. Mathematical Biosciences and Engineering, 2019, 16(3): 1471-1488. doi: 10.3934/mbe.2019071 |
| [9] | Chaoyue Wang, Zhiyao Ma, Shaocheng Tong . Adaptive fuzzy output-feedback event-triggered control for fractional-order nonlinear system. Mathematical Biosciences and Engineering, 2022, 19(12): 12334-12352. doi: 10.3934/mbe.2022575 |
| [10] | Jinxin Du, Lei Liu . Adaptive fuzzy fixed time formation control of state constrained nonlinear multi-agent systems against FDI attacks. Mathematical Biosciences and Engineering, 2024, 21(3): 4724-4741. doi: 10.3934/mbe.2024207 |
Strip steel is widely used in industrial production, including automobile, electromechanical, aerospace, ship and so on. Fundamentally speaking, there are inherent problems in the quality of strip steel, which will not only affect the beauty and comfort of products, but also these areas are usually the starting point of physical damage or chemical corrosion, which also has an adverse impact on the quality and service performance. The main defects of strip steel surface products are poor appearance, quality and use safety. Therefore, effective, rapid and accurate detection of surface defects is the primary problem in the iron and steel industry. At present, the whole manufacturing industry pays more and more attention to the surface defect detection technology, so as to find and effectively control the product quality in time, infer the causes of defects according to the detection results, and improve the production process, so as to reduce or eliminate the occurrence of defects.
The early detection methods of surface defects are mainly based on the manual inspection techniques, which have low efficiency and high cost. Recently, automatic defect inspection (ADI) technology methods based on machine learning have developed rapidly. ADI method not only has higher detection efficiency and accuracy, but also significantly reduces the human and financial resources. Despite this, it is still a very challenging task for ADI to identify the intrinsic and diverse defects in steel. The varieties of strip steel surface defects are shown in Figure 1. The surface defects of strip steel mainly have the following three characteristics, which make it difficult for surface detect detection.
1) Low contrast quality. Surface defect images are usually captured by CDD cameras. However, the environment for the image acquisition of surface defects are affected by light and dust, which resulting in low contrast between background and defects, as shown in Figure 1(a). This case increases the difficulty in defect detection.
2) Different textural and geometric structures. Generally, the defect images collected from different materials exhibit diverse textures. The Figure 1(b) depicts differences in texture features in same type of defects in different materials, where the boundary of defects is fuzzy and irregular. These factors also increase the difficulty in surface defect detection.
3) Diversity of defects. Surface defects always include many categories like inclusion, patches and scratches in which, some features are obvious while others are ambiguous. Further, the defects of the same category invariably show significant differences in appearance, while some defects of different categories have great similarities in appearance, as shown in Figure 1(c). These factors further improve the difficulty in detection process.
To address the above challenges, local binary pattern was applied for surface defects detection [1,2]. Djukic et al. [3] distinguished real defects from random noise pixels by dynamic threshold processing. An entity sparsity pursuit approach was also proposed for surface defects inspection [4]. Neogi et al. [5] suggested a global adaptive percentile thresholding of gradient images, which segment the defect regions and retain the characteristics of the defect without considering the size of the defect. In [6], a Gabor filter combination is proposed to detect the tiny holes on steel slabs. Li et al. [7] proposed an unsupervised approach based on a small number of flawless samples to detect and locate defects in random colour texture. On the other hand, Cohen et al. [8] connected the Markov with Gaussian distribution, and proposed Gaussian Markov Random Field to model the texture image of a non-defective fabric texture. However, all these methods are designed to identify defect detection by designing some artificial features, which lack generality.
Recently, especially CNN based methods are outstanding in the field of machine vision. These methods can automatically extract target features, find the internal feature relationship and law in the sample through iterative optimization, adaptively learn image features and complete object detection tasks, and solve the shortcomings of low efficiency and low detection accuracy of manual design features. A semi-supervised approach based on CNN was used to classify the strip steel surface defect [9]. Since the industrial defect images are difficult to collect, Natarajan et al. [10] adopted transfer learning to extract multi-level features and then input these features into SVM classifiers to avoid the over fitting caused by the small samples. However, the accuracy of these methods needs further improvement.
In this work, a pixel-wise detection framework based on CNN for strip steel surface defect detection is proposed to obtain multi-scale context information from high-level features by different sizes of convolution kernels. A cross integration is adopted to realize the effective utilization of these context information and to decode the information, which realizes the feature information complementation. The output of the framework is accurate pixel-wise classification and location. The main contributions of this study are:
● A pixel-wise detection framework based on CNN for strip steel surface defect detection is introduced. The output of the detection framework is the pixel-wise binary saliency maps of defect regions, which can effectively evaluate the quality of strip steel products.
● A contextual weighting module is proposed, which uses convolutional kernels with different size to obtain multi-scale context feature information from the convolution layers to achieve overall perception of the defect.
● In the decoder module, the cross integration is used to integrate the context information and previous decoded information into the current decoding block to realize feature information complementation.
● The proposed method is tested on the NEU-strip steel surface defect dataset, and the experimental results prove the effectiveness of the proposed method.
In this section, two kinds of detection methods for surface defect will be introduced, including: i) traditional approaches; ii) deep learning-based approaches.
The traditional methods for surface defect detection mainly include three categories: the statistical-based approaches, the filter-based approaches and the model-based approaches.
These methods use random phenomenon to analysis the distribution of random variables from the perspective of statistics, so as to realize the description of the image texture. Neogi et al. [5] proposed a global adaptive percentile thresholding of gradient images, which segment the defect regions and retain the characteristics of the defect without considering the size of the defect. Win et al. [11] proposed two thresholding methods namely, contrast-adjusted Otsu's method and contrast-adjusted median-based Otsu's method for automated defect detection system. Ricci et al. [12] used canny operator to detect the defect edges. Hu et al. [13] used Fourier shape descriptors for description of outline features in steel surface defects. Zhao et al. [14] proposed a two-level labelling technique based on super pixels. This method clustered pixels into super pixels and then the super pixels into sub-regions. Wang et al. [15] extracted and fused features of co-occurrence matrix and the histogram of oriented gradient to describe the local and the global texture characteristics, respectively. Chu et al. [16] proposed a smoothed local binary patterns by applying weight on the local neighbourhood. Fekri-Ershad et al. [17] applied a new noise-resistant and multi-resolution version of the LBP to extract jointly the colour and texture features jointly. Song et al. [1] proposed an adjacent evaluation completed local binary patterns against noise for defect inspection. Zhang et al. [18] used gray level co-occurrence matrix (GLCM) and HU invariant moments for feature extraction, and then applied adaptive genetic algorithm for feature selection.
The principle of this method was to transform the original image in frequency domain, and then use the corresponding filter to consider the image and to remove the features with low noise and correlation, so that the algorithm can extract more valuable information. Ai et al. [19] adopted kernel locality preserving projections and curvelet transform extract feature for the surface longitudinal cracks detection of the slabs. In [6,20], a Gabor filter combination is proposed to detect the tiny holed on steel slabs. Other method [On the other hand, Choi et al. [21] adopted two Gabor filters to detect the seam cracks on the steel plates, which have high detection performance and can effectively reduce noise. Wu et al. [22] used modular maximum of inter scale correlation of wavelet coefficient to determine the positions of the defects, and then used the prior knowledge about the characteristics of the surface defect defects for their classification. Öztürk et al. [23] proposed novel BiasFeed cellular neural network model for glass defect inspection. Li et al. [24] proposed a second-order derivative and morphology operations, the row-by-row adaptive thresholding, and 2-D wavelet transform to process the images showing different defects of the castings. Liu et al. [25] applied a non-subsampled shearlet transform and the kernel locality preserving projection to the surface defect detection. Akdemir et al. [26] adopted wavelet transforms to glass surface defects detection.
These methods are based on the construction model of the image, and uses the statistics of model parameters as texture features. Different textures are expressed as different values of model parameters under some assumptions. In [7], an unsupervised approach based on a small number of flawless samples was used to detect and locate the defects in random color texture. Cohen et al. [27] connected Markov with Gaussian distribution, and proposed Gaussian Markov Random Field to model the texture image of a non-defective fabric texture. Song et al. [28] proposed a saliency propagation algorithm based on multiple constraints and improved texture features (MCITF) for surface defect detection.
Recently, deep learning based on CNN approaches have achieved outperformed in the field of machine vision tasks. Many scholars have solved the problem of industrial defect detection by deep learning. In [9], a semi-supervised approach based on CNN was used to classify the strip steel surface defect. Since the industrial defect images are difficult to collect, Natarajan et al. [10] adopted transfer learning to extract the multi-level features and then input these features into SVM classifiers to avoid the over fitting caused by small samples. Masci et al. [29] proposed a Multi-scale pyramidal pooling network for generic steel defect classification. He et al. [30] proposed a multi-group convolutional neural network (MG-CNN) to inspect the defects of the steel surface. In [31], an end-to-end detection framework was proposed, which integrated multi-level features to complete the detection of the strip steel surface defect. The output of the network located the defect areas through some dense bounding boxes and gave the category name to these defects. Kou et al. [32] developed an end-to-end defect detection model based on YOLO-V3 for the surface defect detection on strip steel. In [33], a pre-trained deep learning network is used to extract multi-scale features from raw image patches to achieve image classification and defect segmentation. In [34], a multi-scale feature-clustering-based fully convolutional was proposed for the texture surface defect detection. Neven et al. [35] proposed a multi-branch U-Net for steel surface defect type and severity segmentation. Zhou et al. [36] proposed edge-aware multi-level interactive network for salient object detection of strip steel surface defects. Song et al. [37] adopted encoder-decoder residual network for salient object detection of strip steel surface defects. Dong et al. [38] proposed a pyramid feature fusion and global context attention network for automated surface defect segmentation. Although these methods achieved outstanding performance in the defects detection, they still need to be improved especially, in the feature extraction and utilization. Unlike previous studies, this paper proposes a pixel-wise detection framework based on CNN for strip steel surface defect detection.
The surface defect inception is formulated in this work as a pixel-wise segmentation task. Given a defect image, the proposed framework outputs a binary map, the defect area is represented by "1", while the non-defect area is represented by "0". The architecture of the framework mainly includes three parts: an encoder, the contextual weighting module and a decoder as shown in Figure 2.
Given a defect image, the framework first extracts the multi-level features from fine, shallow layers (enc1) to coarse, deep layers (enc5) by a pre-trained VGG-16 [39] network which is called an encoder module. The encoder module is composed of convolution layers and max pooling layers. In order to retain the spatial information of each pixel, the fully connection layers of VGG-16 network is removed. Subsequently, a contextual weighting module is adopted to obtain multi-scale contextual information from the high-level features to keep the shape and size in variance of the final features. In the encoder, the features extracted from enc3, enc4 and enc5 are considered as high-level features. In the decoder, the output of each con-textual weighting network is fused to the input of the same decoder in a feedback fashion. The final output of the decoder is a defect binary saliency map.
The encoder is used to extract multi-level features of the defect images, which is built on the pre-trained VGG-16 network. The encoder module mainly consists of 5 convolution layers and 4 max pooling layers. The details of the encoder module, i.e. blocks encx where, x = 1, …, 5 are listed in Table 1. In the encoder, the convolutional layer performs sliding on the input local areas through a series of convolutional kernels to obtain the features of the input image, followed by ReLU and BN. Let T={(Xn,Zn),n=1,...,N} represents training data, where Xn={xni,i=1,⋅⋅⋅,|Xn|} denotes the input image and Zn={zni,i=1,⋅⋅⋅,|Zn|} is the corresponding ground truth for Xn. The convolution of Xn is as follows:
| C=σ(Wxni+b) | (1) |
| Stage | Template | ||
| enc1 | conv 3 × 3, stride = 1, D = 64, BN + ReLU | max_pooling 2 × 2, stride = 2 | ×2 |
| enc2 | conv 3 × 3, stride = 1, D = 128, BN + ReLU | max_pooling 2 × 2, stride = 2 | ×2 |
| enc3 | conv 3 × 3, stride = 1, D = 256, BN + ReLU | max_pooling 2 × 2, stride = 2 | ×2 |
| enc4 | conv 3 × 3, stride = 1, D = 512, BN + ReLU | max_pooling 2 × 2, stride = 2 | ×2 |
| enc5 | conv 3 × 3, stride = 1, D = 512, BN + ReLU | max_pooling 2 × 2, stride = 2 | ×2 |
DownLoad:
CSV
where, W denotes weights, b refers bias, and σ represents the ReLU activation. By sliding the convolution kernels to obtain the feature sets. The pooling layers adopt 2 × 2 pool filter to down-scale the input feature maps, which is to change the spatial dimension and reduce the amount of calculation. The output of pooling layer is given below:
| ˆC=pool(C) | (2) |
where pool denotes the max pooling with 2×2 pool filter and stride 2. The encoder finally generates five resolution feature maps F = {f1, f2, …, f5}, and f1 denotes the enc1 features and so on.
The fusion of convolutional features obtained from different stages is a common mechanism in most detection methods, because these features not only contain low-level visual information, but also include high-level abstract information. The earlier methods [40,41] combine these features directly from bottom to top. However, this simple combination may induce some bad features in the images to be integrated into the final prediction. To address this issue, a contextual weighting module, inspired by [42], is proposed in Figure 3. The CWM applies different convolution kernels to extract multi-scale contextual information from high-level features, which provides entire description for interpretation of the whole scene especially, multi-scale and multi shape objects. In the CWM, the features f3, f4 and f5 are used as high-level features. CWM used four stacked convolutional kernels (1×1, 3×3, 5×5, 7×7) to obtain multi-scale contextual information from the high-level features, and each kernel generates a feature map with the size of high-level features. For high-level feature f3, the output multi-scale contextual information can be denoted by F3:
| M13=BN(σ(W1×1f3+b)) | (3) |
| M33=BN(σ(W3×3f3+b)) | (4) |
| M53=BN(σ(W5×5f3+b)) | (5) |
| M73=BN(σ(W7×7f3+b)) | (6) |
Where BN denotes Batch Normalization, σ is nonlinear activation function ReLU. Wi×i denote the i × i convolutional kernel. The size of each generated features Mi3 (i = 1, 3, 5, 7) is the same as that of f3, and the number of channels is 32.
Then these feature maps are fused by concatenation. After that, 1×1 convolutional kernels are used to resize the channel of concatenated features to reduce the computation of the contextual weighting. The output saliency map G3 is formulated as:
| G3=BN(σ(W1×1CAT(M13+M33+M53+M73)+b)) | (7) |
Where BN denotes Batch Normalization, σ is nonlinear activation function ReLU, CAT denotes concatenation. W1×1 is 1×1 convolutional kernel with 128 channels. The number of channels of G3 is 128.
For high-level feature f4 and f5, the model generates G4 and G5 in the same way as G3.
In this section, a novel decoder module is proposed, which includes 4 blocks (dec2, dec3, dec4, dec5), as shown in Figure 2. The dec3 and dec4 are fusion decoders, which are composed of the former one or two decoders and the output from contextual weighting module connected with enc3 and enc4, respectively. To enable effective fusion of these features, which must ensure that they have the same dimensionality. Firstly, a series of 3×3×D convolution kernels are applied to reduce channel dimension of these fused feature maps, where D is 32. Then a bilinear interpolation is applied to upsample low-resolution features to the target spatial resolution of the features that will be fused. Subsequently, these feature maps are fused by element-wise concatenation, as shown in Figure 4. The output decx is defined as follows:
| decx={up×2(decx+1)x=2CAT(up×4(ϕ(ch(decx+2;θ))),up×2(ϕ(ch(decx+1;θ))),ϕ(ch(Gx;θ)))x=3CAT(up×2(ϕ(ch(decx+1;θ))),ϕ(ch(Gx;θ)))x=4 | (8) |
The final prediction Yp is formulated as:
| Yp=up×2(W1×1∗dec2+b) | (9) |
where, CAT refers concatenation, up denotes upsample, ϕ represents the ReLU activation and ch is 3×3 convolution.
Loss function is the most basic and key factor in machine learning, which is used to measure the quality of model prediction. In this paper, three losses are applied to optimize the model. The final loss is defined as:
| L=lBCE+lIoU+lSSIM | (10) |
where lBCE, lIoU and lSSIM represent the BCE loss, IoU loss and SSIM loss, respectively.
The BCE [43] loss is applied to compute the similarity between the prediction and ground truth, which is defined as:
| lBCE=−∑TlogP−∑(1−T)log(1−P) | (11) |
where T ∈[0, 1] denotes the ground truth, and P∈[0, 1] is the predicted probability.
The IOU [44] loss is used to measure the repeatability between the prediction and the ground truth, which is defined as:
| lIoU=1−∑TP∑[P+T−PT] | (12) |
where T ∈[0, 1] denotes the ground truth, and P∈[0, 1] is the predicted probability.
The SSIM [45] are originally applied to measure the structural similarity of two images. Let p={pi=1,⋅⋅⋅,N2} and t={ti=1,⋅⋅⋅,N2} represent the pixel values of two corresponding patches (size: N × N) cropped from the prediction P and ground truth T, respectively. The SSIM is computed as:
| lIoU=1−(2vpvt+C1)(2σpt+C2)(v2p+v2t+C1)(σ2p+σ2t+C2) | (13) |
where vp and vt are the mean of p and t, respectively. σ2p and σ2t are the variance of p and t, respectively. σpt is their covariance. C1 and C2 are small constants that are applied to avoid dividing by zero.
This section mainly consists of six experimental parts: the details of implementation, the dataset and the evaluation metrics, the performance of the proposed method and other previous methods, followed by the ablation study and analysis of failure cases.
The proposed method is implemented based on TensorFlow [46] framework. The weights of new convolution layers in the framework are initialized with standard deviation 0.01 and biases are initialized to 0. The weights of backbone network are initialized using pre-trained ImageNet [47] network. The momentum and weight decay are set to 0.9 and 0.0005, respectively. The initial learning rate is set to 5e-5, which decreased by 10 after 10 epochs. The framework is trained for 300 epochs in total.
In the experiment of this study, three kinds of surface defect of strip steel [1] are selected, including Scratches, Patches, and Inclusion, as shown in Figure 5. All categories of defects are considered as detection targets. In the dataset, the training set includes 3630 defect samples, and the test set includes 792 defect samples. All the samples are resized to 256×256 during in the process of training network.
To evaluate the proposed framework, four metrics are used along with other previous state-of-the-art approaches, namely precision-recall (PR) curves, F-measure score and mean absolute error (MAE). The PR curve demonstrates the average recall and precision and of saliency maps at different thresholds, formulated as follows:
| Recall=TPTP+FN | (14) |
| Precision=TPTP+FP | (15) |
where FN, FP and TP indicate correctly the number judge of false negative pixels, false positive and true positive, respectively. F-measure, refers Fβ and is computed by weighted harmonic mean of recall and precision under nonnegative weight β, which defined as:
| Fβ=(1+β2)Precision×Recallβ2×Precision+Recall | (16) |
the β2 = 0.3 is used in other methods.
MAE [48] is used to calculate the mean absolute error between the ground truth and the prediction. First, the prediction and the ground truth are binarized. Then, the MAE score is computed by:
| MAE=1W×HW∑x=1H∑y=1|P(x,y)−S(x,y)| | (17) |
where P and S refer the prediction and the ground truth, respectively, while H and W are the height and width of images, respectively.
In this subsection, the proposed method is compared with 10 previous state-of-the-art methods, including BSCA [49], FT [50], MIL [51], RC [52], SMD [53], FCN [40], UNet [41], DN [54], DHSNet [55] and DSS [56], all the compared are pixel-wised method. For the sake of comparison, the same evaluation metrics and code are used to evaluate the output prediction maps.
Qualitative comparison results are shown in Figure 5, the proposed method can accurately detect defects and highlight them evenly in various challenging cases, i) low contrast quality between the defect region and background (e.g., row 1 and 3); ii) different textural and geometric structures (e.g., row 4 and 5); iii) diversity of defects (e.g., row 6, 7 and 8). For low contrast quality: some methods are missing or detecting a rough defect area which cannot express the defects vividly. For different textural and geometric structures, the defect region detected by methods are with little noise and not obvious. For diversity of defects, some methods cannot detect all categories of defects. The detection effect of deep learning-based methods is better than that of the traditional methods. However, for some minor defects, FCN, UNet, DN, DSS and DHSNet are either missing or incomplete detection areas. Instead, the proposed approach not only can distinguish the defect area and background effectively under low contrast, but also locate and detect the defects in different positions, scales and shapes accurately.
The advantages of the proposed method are shown in Figure 6. The method achieves outstanding performance among all the compared methods on strip steel surface defect dataset in terms of all evaluation metrics. It further improves the P-R curve and F-measure, and reduces MAE significantly. As listed in Table 2, the proposed method outperforms the competitive methods in Fβ and MAE. Compared with the traditional methods, Fβ is improved 36.12%, and decreased by 12.08% in MAE. Compared with deep learning method, the Fβ is improved by 0.38%, and decreased by 0.02% in MAE. The comparison of the above qualitative and quantitative analysis further proves the effectiveness of this method.
| Method | BSCA[49] | FT[50] | MIL[51] | RC[52] | SMD[53] | FCN[40] | UNet[41] | DN[54] | DHSNet[55] | DSS[56] | My |
| MAE↓ | 0.2462 | 0157. 7 | 0.1764 | 0.1228 | 0.1913 | 0.0604 | 0.0643 | 0.0481 | 0.0399 | 0.0371 | 0.0369 |
| Fβ | 0.3404 | 0.4462 | 0.4873 | 0.3719 | 0.4665 | 0.6788 | 0.6783 | 0.7272 | 0.8447 | 0.8051 | 0.8485 |
DownLoad:
CSV
In this work, the ablation study on the proposed CWN to verify the effectiveness of CWN. First, the CWN is removed, and directly combined the feature maps output from the encoder with the decoded feature maps through dense short connections, and output the optimal model after training the overall network. The ablation study further add the CWN module into the proposed method, and output the trained model after the same training. Finally, the two trained models are tested separately and output saliency prediction maps. As listed in Table 3, the contextual weighting module get declines by 0.21% in MAE and improve the performance by 0.48% in Fβ. These results prove the effectiveness of contextual weighting module to in the framework. In addition, the ablation study on the loss function to verify the effectiveness of the loss function. As listed in Table 3, the loss function get declines by 1.21% in MAE and improve the performance by 14.60% in Fβ.
| Method | MAE | Fβ |
| CMN−+ lBCE | 0.0511 | 0.6977 |
| CMN++lBCE | 0.0490 | 0.7025 |
| CMN++L | 0.0369 | 0.8485 |
DownLoad:
CSV
The results of this study show that the proposed method is outstanding over the previous state-of-the-art methods on the strip steel surface defect dataset. However, some defect images still pose challenges to these methods. The images (c) and (d) of Figure 7 show that the detection of some defect images are lack of integrity. The images (a), (b), (e) and (f) show that some defects are missed. Figure 7 shows the reasons leading to failure detection are attributed to some defects are too small to be detected; some defects show low contrast, so it is difficult to judge whether they really are defects, and in some cases, the characteristics of some defect areas apparently change. In the future, I plan to focus on solving these problems.
In this paper, a pixel-wise inspection framework based on CNN for the surface defect inspection of strip steel is proposed. Firstly, the encoder of the framework is built on the pre-trained VGG-16 network, which is used to extract multi-level features. Next, the contextual weighting module uses convolutional kernels with different size to obtain multi-scale context feature information from the convolution layers, which achieve overall perception of defect. Finally, in the decoder module, the cross integration is used to integrate the context information and previously decoded information into the current decoding block, which realizes the feature information complementation. The experiments of this study demonstrate that the proposed method is outstanding over the previous state-of-the-art methods in detection of strip steel defect dataset. To sum up, the proposed method can detect defects accurately, which makes the network strong robust and effective in defect detection. In the future, I will further optimize the algorithm model.
This work is supported by the National Key R & D Program of China with No.2018YFB1800402.
The authors declared that they have no conflicts of interest in this work.
| [1] | E. Y. Lam, D. Beraldi, D. Tannahill, S. Balasubramanian, G-quadruplex structures are stable and detectable in human genomic DNA, Nat. Commun., 4 (2013), 1796. |
| [2] | P. Agarwala, S. Pandey, S. Maiti, The tale of RNA G-quadruplex, Org. Biomol. Chem., 13 (2015), 5570-5585. |
| [3] | A. Y. Zhang, S. Balasubramanian, The kinetics and folding pathways of intramolecular G-quadruplex nucleic acids, J. Am. Chem. Soc., 134 (2012), 19297-19308. |
| [4] | D. Rhodes, H. J. Lipps, G-quadruplexes and their regulatory roles in biology, Nucleic Acids Res., 43 (2015), 8627-8637. |
| [5] | S. Millevoi, H. Moine, S. Vagner, G-quadruplexes in RNA biology, Wiley Interdiscip. Rev. RNA, 3 (2012), 495-507. |
| [6] | K. Hoogsteen, The crystal and molecular structure of a hydrogen-bonded complex between 1-methylthymine and 9-methyladenine, Acta Crystallograph., 16 (1963), 907-916. |
| [7] | J. Amato, A. Pagano, S. Cosconati, G. Amendola, I. Fotticchia, N. Iaccarino, et al., Discovery of the first dual G-triplex/G-quadruplex stabilizing compound: A new opportunity in the targeting of G-rich DNA structures?, Biochim. Biophys. Acta Gen. Subj., 1861 (2017), 1271-1280. |
| [8] | M. Cheng, Y. Cheng, J. Hao, G. Jia, J. Zhou, J. L. Mergny, et al., Loop permutation affects the topology and stability of G-quadruplexes, Nucleic Acids Res., 46 (2018), 9264-9275. |
| [9] | S. Pandey, P. Agarwala, S. Maiti, Effect of loops and G-quartets on the stability of RNA G-quadruplexes, J. Phys. Chem. B, 117 (2013), 6896-6905. |
| [10] | F. Hao, Y. Ma, Y. Guan, Effects of central loop length and metal ions on the thermal stability of G-quadruplexes, Molecules, 24 (2019). |
| [11] |
B. A. Tucker, J. S. Hudson, L. Ding, E. Lewis, R. D. Sheardy, E. Kharlampieva, et al., Stability of the Na(+) form of the human telomeric G-quadruplex: Role of adenines in stabilizing G-quadruplex structure, ACS Omega, 3 (2018), 844-855. doi: 10.1021/acsomega.7b01649
|
| [12] | A. K. Todd, M. Johnston, S. Neidle, Highly prevalent putative quadruplex sequence motifs in human DNA, Nucleic Acids Res., 33 (2005), 2901-2907. |
| [13] | L. Q. Gu, Y. Wang, Biomedical diagnosis perspective of epigenetic detections using alpha-hemolysin nanopore, AIMS Mater. Sci., 2 (2015), 448-472. |
| [14] | V. T. Mukundan, A. T. Phan, Bulges in G-quadruplexes: broadening the definition of G-quadruplex-forming sequences, J. Am. Chem. Soc., 135 (2013), 5017-5028. |
| [15] | A. Guedin, J. Gros, P. Alberti, J. L. Mergny, How long is too long? Effects of loop size on G-quadruplex stability, Nucleic Acids Res., 38 (2010), 7858-7868. |
| [16] | J. M. Garant, M. J. Luce, M. S. Scott, J. P. Perreault, G4RNA: An RNA G-quadruplex database, Database (Oxford), 2015 (2015), bav059. |
| [17] | A. Bedrat, L. Lacroix, J. L. Mergny, Re-evaluation of G-quadruplex propensity with G4Hunter, Nucleic Acids Res., 44 (2016), 1746-1759. |
| [18] | J. M. Garant, J. P. Perreault, M. S. Scott, Motif independent identification of potential RNA G-quadruplexes by G4RNA screener, Bioinformatics, 33 (2017), 3532-3537. |
| [19] | K. Wethmar, A. Barbosa-Silva, M. A. Andrade-Navarro, A. Leutz, uORFdb—a comprehensive literature database on eukaryotic uORF biology, Nucleic Acids Res., 42 (2014), D60-67. |
| [20] |
J. Ma, C. C. Ward, I. Jungreis, S. A. Slavoff, A. G. Schwaid, J. Neveu, et al., Discovery of human sORF-encoded polypeptides (SEPs) in cell lines and tissue, J. Proteome Res., 13 (2014), 1757-1765. doi: 10.1021/pr401280w
|
| [21] | S. A. Slavoff, A. J. Mitchell, A. G. Schwaid, M. N. Cabili, J. Ma, J. Z. Levin, et al., Peptidomic discovery of short open reading frame-encoded peptides in human cells, Nat. Chem. Biol., 9 (2013), 59-64. |
| [22] | M. C. Frith, A. R. Forrest, E. Nourbakhsh, K. C. Pang, C. Kai, J. Kawai, et al., The abundance of short proteins in the mammalian proteome, PLoS Genet., 2 (2006), e52. |
| [23] | C. Weldon, J. G. Dacanay, V. Gokhale, P. V. L. Boddupally, I. Behm-Ansmant, G. A. Burley, et al., Specific G-quadruplex ligands modulate the alternative splicing of Bcl-X, Nucleic Acids Res., 46 (2018), 886-896. |
| [24] | C. K. Kwok, S. Balasubramanian, Targeted detection of G-quadruplexes in cellular RNAs, Angew. Chem. Int. Ed. Engl., 54 (2015), 6751-6754. |
| [25] | G. Mirihana Arachchilage, M. J. Morris, S. Basu, A library screening approach identifies naturally occurring RNA sequences for a G-quadruplex binding ligand, Chem. Commun. (Camb.), 50 (2014), 1250-1252. |
| [26] | R. C. Olsthoorn, G-quadruplexes within prion mRNA: The missing link in prion disease?, Nucleic Acids Res., 42 (2014), 9327-9333. |
| [27] | T. Endoh, Y. Kawasaki, N. Sugimoto, Stability of RNA quadruplex in open reading frame determines proteolysis of human estrogen receptor alpha, Nucleic Acids Res., 41 (2013), 6222-6231. |
| [28] | J. F. Fisette, D. R. Montagna, M. R. Mihailescu, M. S. Wolfe, A G-rich element forms a G-quadruplex and regulates BACE1 mRNA alternative splicing, J. Neurochem., 121 (2012), 763-773. |
| [29] | D. Gomez, A. Guedin, J. L. Mergny, B. Salles, J. F. Riou, M. P. Teulade-Fichou, et al., A G-quadruplex structure within the 5'-UTR of TRF2 mRNA represses translation in human cells, Nucleic Acids Res., 38 (2010), 7187-7198. |
| [30] | P. Agarwala, S. Pandey, K. Mapa, S. Maiti, The G-quadruplex augments translation in the 5' untranslated region of transforming growth factor β2, Biochemistry, 52 (2013), 1528-1538. |
| [31] | A. Arora, B. Suess, An RNA G-quadruplex in the 3' UTR of the proto-oncogene PIM1 represses translation, RNA Biol., 8 (2011), 802-805. |
| [32] | M. J. Morris, S. Basu, An unusually stable G-quadruplex within the 5'-UTR of the MT3 matrix metalloproteinase mRNA represses translation in eukaryotic cells, Biochemistry, 48 (2009), 5313-5319. |
| [33] | S. Kumari, A. Bugaut, J. L. Huppert, S. Balasubramanian, An RNA G-quadruplex in the 5' UTR of the NRAS proto-oncogene modulates translation, Nat. Chem. Biol., 3 (2007), 218-221. |
| [34] | R. van der Lee, M. Buljan, B. Lang, R. J. Weatheritt, G. W. Daughdrill, A. K. Dunker, et al., Classification of intrinsically disordered regions and proteins, Chem. Rev., 114 (2014), 6589-6631. |
| [35] | N. E. Davey, G. Trave, T. J. Gibson, How viruses hijack cell regulation, Trends Biochem. Sci., 36 (2011), 159-169. |
| [36] | M. Jucker, L. C. Walker, Self-propagation of pathogenic protein aggregates in neurodegenerative diseases, Nature, 501 (2013), 45-51. |
| [37] | M. Goedert, M. G. Spillantini, K. Del Tredici, H. Braak, 100 years of Lewy pathology, Nat. Rev. Neurol., 9 (2013), 13-24. |
| [38] | D. Piovesan, F. Tabaro, I. Micetic, M. Necci, F. Quaglia, C. J. Oldfield, et al., DisProt 7.0: A major update of the database of disordered proteins, Nucleic Acids Res., 45 (2017), D219-D227. |
| [39] | S. Hutchinson, A. Furger, D. Halliday, D. P. Judge, A. Jefferson, H. C. Dietz, et al., Allelic variation in normal human FBN1 expression in a family with Marfan syndrome: A potential modifier of phenotype?, Hum. Mol. Genet., 12 (2003), 2269-2276. |
| [40] | Y. F. Chang, J. S. Imam, M. F. Wilkinson, The nonsense-mediated decay RNA surveillance pathway, Annu. Rev. Biochem., 76 (2007), 51-74. |
| [41] |
M. M. Klein, A. G. Gittis, H. P. Su, M. O. Makobongo, J. M. Moore, S. Singh, et al., The cysteine-rich interdomain region from the highly variable plasmodium falciparum erythrocyte membrane protein-1 exhibits a conserved structure, PLoS Pathog., 4 (2008), e1000147. doi: 10.1371/journal.ppat.1000147
|
| [42] | J. A. Corcoran, R. Syvitski, D. Top, R. M. Epand, R. F. Epand, D. Jakeman, et al., Myristoylation, a protruding loop, and structural plasticity are essential features of a nonenveloped virus fusion peptide motif, J. Biol. Chem., 279 (2004), 51386-51394. |
| [43] | K. A. Morrow, C. D. Ochoa, R. Balczon, C. Zhou, L. Cauthen, M. Alexeyev, et al., Pseudomonas aeruginosa exoenzymes U and Y induce a transmissible endothelial proteinopathy, Am. J. Physiol. Lung Cell Mol. Physiol., 310 (2016), L337-353. |
| [44] | A. L. Woerman, S. A. Kazmi, S. Patel, A. Aoyagi, A. Oehler, K. Widjaja, D. A. Mordes, et al., Familial Parkinson's point mutation abolishes multiple system atrophy prion replication, Proc. Natl. Acad. Sci. U S A, 115 (2018), 409-414. |
| [45] | J. D. Beaudoin, J. P. Perreault, Exploring mRNA 3'-UTR G-quadruplexes: evidence of roles in both alternative polyadenylation and mRNA shortening, Nucleic Acids Res., 41 (2013), 5898-5911. |
| [46] | J. D. Beaudoin, R. Jodoin, J. P. Perreault, New scoring system to identify RNA G-quadruplex folding, Nucleic Acids Res., 42 (2014), 1209-1223. |
| [47] | R. Shahid, A. Bugaut, S. Balasubramanian, The BCL-2 5' untranslated region contains an RNA G-quadruplex-forming motif that modulates protein expression, Biochemistry, 49 (2010), 8300-8306. |
| [48] | S. Saxena, D. Miyoshi, N. Sugimoto, Sole and stable RNA duplexes of G-rich sequences located in the 5'-untranslated region of protooncogenes, Biochemistry, 49 (2010), 7190-7201. |
| [49] | J. D. Beaudoin, J. P. Perreault, 5'-UTR G-quadruplex structures acting as translational repressors, Nucleic Acids Res., 38 (2010), 7022-7036. |
| [50] | R. Jodoin, L. Bauer, J. M. Garant, A. Mahdi Laaref, F. Phaneuf, J. P. Perreault, The folding of 5'-UTR human G-quadruplexes possessing a long central loop, RNA, 20 (2014), 1129-1141. |
| [51] | D. Bhattacharyya, P. Diamond, S. Basu, An Independently folding RNA G-quadruplex domain directly recruits the 40S ribosomal subunit, Biochemistry, 54 (2015), 1879-1885. |
| [52] | A. Cammas, A. Dubrac, B. Morel, A. Lamaa, C. Touriol, M. P. Teulade-Fichou, et al., Stabilization of the G-quadruplex at the VEGF IRES represses cap-independent translation, RNA Biol., 12 (2015), 320-329. |
| [53] | M. J. Morris, Y. Negishi, C. Pazsint, J. D. Schonhoft, S. Basu, An RNA G-quadruplex is essential for cap-independent translation initiation in human VEGF IRES, J. Am. Chem. Soc., 132 (2010), 17831-17839. |
| [54] | J. Christiansen, M. Kofod, F. C. Nielsen, A guanosine quadruplex and two stable hairpins flank a major cleavage site in insulin-like growth factor II mRNA, Nucleic Acids Res., 22 (1994), 5709-5716. |
| [55] | S. Lammich, F. Kamp, J. Wagner, B. Nuscher, S. Zilow, A. K. Ludwig, et al., Translational repression of the disintegrin and metalloprotease ADAM10 by a stable G-quadruplex secondary structure in its 5'-untranslated region, J. Biol. Chem., 286 (2011), 45063-45072. |
| [56] | P. Agarwala, S. Pandey, S. Maiti, Role of G-quadruplex located at 5' end of mRNAs, Biochim. Biophys. Acta, 1840 (2014), 3503-3510. |
| [57] | M. Subramanian, F. Rage, R. Tabet, E. Flatter, J. L. Mandel, H. Moine, G-quadruplex RNA structure as a signal for neurite mRNA targeting, EMBO Rep., 12 (2011), 697-704. |
| [58] | A. von Hacht, O. Seifert, M. Menger, T. Schutze, A. Arora, Z. Konthur, et al., Identification and characterization of RNA guanine-quadruplex binding proteins, Nucleic Acids Res., 42 (2014), 6630-6644. |
| [59] | S. Balaratnam, S. Basu, Divalent cation-aided identification of physico-chemical properties of metal ions that stabilize RNA G-quadruplexes, Biopolymers, 103 (2015), 376-386. |
| [60] | S. Stefanovic, G. J. Bassell, M. R. Mihailescu, G quadruplex RNA structures in PSD-95 mRNA: potential regulators of miR-125a seed binding site accessibility, RNA, 21 (2015), 48-60. |
| [61] | J. C. Grigg, N. Shumayrikh, D. Sen, G-quadruplex structures formed by expanded hexanucleotide repeat RNA and DNA from the neurodegenerative disease-linked C9orf72 gene efficiently sequester and activate heme, PLoS One, 9 (2014), e106449. |
| [62] | P. Fratta, S. Mizielinska, A. J. Nicoll, M. Zloh, E. M. Fisher, G. Parkinson, et al., C9orf72 hexanucleotide repeat associated with amyotrophic lateral sclerosis and frontotemporal dementia forms RNA G-quadruplexes, Sci. Rep., 2 (2012), 1016. |
| [63] | H. Y. Weng, H. L. Huang, P. P. Zhao, H. Zhou, L. H. Qu, Translational repression of cyclin D3 by a stable G-quadruplex in its 5' UTR: implications for cell cycle regulation, RNA Biol., 9 (2012), 1099-1109. |
| [64] | H. H. Woo, T. Baker, C. Laszlo, S. K. Chambers, Nucleolin mediates microRNA-directed CSF-1 mRNA deadenylation but increases translation of CSF-1 mRNA, Mol. Cell Proteomics, 12 (2013), 1661-1677. |
| [65] | H. Dhayan, A. R. Baydoun, A. Kukol, G-quadruplex formation of FXYD1 pre-mRNA indicates the possibility of regulating expression of its protein product, Arch. Biochem. Biophys., 560 (2014), 52-58. |
| [66] | S. G. Rouleau, J. D. Beaudoin, M. Bisaillon, J. P. Perreault, Small antisense oligonucleotides against G-quadruplexes: specific mRNA translational switches, Nucleic Acids Res., 43 (2015), 595-606. |
| [67] | J. Kralovicova, A. Lages, A. Patel, A. Dhir, E. Buratti, M. Searle, et al., Optimal antisense target reducing INS intron 1 retention is adjacent to a parallel G quadruplex, Nucleic Acids Res., 42 (2014), 8161-8173. |
| [68] | M. Faudale, S. Cogoi, L. E. Xodo, Photoactivated cationic alkyl-substituted porphyrin binding to g4-RNA in the 5'-UTR of KRAS oncogene represses translation, Chem. Commun. (Camb.), 48 (2012), 874-876. |
| [69] | M. M. Ribeiro, G. S. Teixeira, L. Martins, M. R. Marques, A. P. de Souza, S. R. Line, G-quadruplex formation enhances splicing efficiency of PAX9 intron 1, Hum. Genet., 134 (2015), 37-44. |
| [70] | E. P. Booy, R. Howard, O. Marushchak, E. O. Ariyo, M. Meier, S. K. Novakowski, et al., The RNA helicase RHAU (DHX36) suppresses expression of the transcription factor PITX1, Nucleic Acids Res., 42 (2014), 3346-3361. |
| [71] | Y. Zhang, C. M. Gaetano, K. R. Williams, G. J. Bassell, M. R. Mihailescu, FMRP interacts with G-quadruplex structures in the 3'-UTR of its dendritic target Shank1 mRNA, RNA Biol., 11 (2014), 1364-1374. |
| [72] | H. Martadinata, A. T. Phan, Formation of a stacked dimeric G-quadruplex containing bulges by the 5'-terminal region of human telomerase RNA (hTERC), Biochemistry, 53 (2014), 1595-1600. |
| [73] | K. Hirashima, H. Seimiya, Telomeric repeat-containing RNA/G-quadruplex-forming sequences cause genome-wide alteration of gene expression in human cancer cells in vivo, Nucleic Acids Res., 43 (2015), 2022-2032. |
| [74] | Y. Katsuda, S. Sato, L. Asano, Y. Morimura, T. Furuta, H. Sugiyama, et al., A Small Molecule That Represses Translation of G-Quadruplex-Containing mRNA, J. Am. Chem. Soc., 138 (2016), 9037-9040. |
| [75] | C. K. Kwok, A. B. Sahakyan, S. Balasubramanian, Structural Analysis using SHALiPE to Reveal RNA G-Quadruplex Formation in Human Precursor MicroRNA, Angew. Chem. Int. Ed. Engl., 55 (2016), 8958-8961. |
| [76] | P. Agarwala, S. Kumar, S. Pandey, S. Maiti, Human telomeric RNA G-quadruplex response to point mutation in the G-quartets, J. Phys. Chem. B, 119 (2015), 4617-4627. |
| [77] | V. Marcel, P. L. Tran, C. Sagne, G. Martel-Planche, L. Vaslin, M. P. Teulade-Fichou, et al., G-quadruplex structures in TP53 intron 3: role in alternative splicing and in production of p53 mRNA isoforms, Carcinogenesis, 32 (2011), 271-278. |
| [78] | A. Decorsiere, A. Cayrel, S. Vagner, S. Millevoi, Essential role for the interaction between hnRNP H/F and a G quadruplex in maintaining p53 pre-mRNA 3'-end processing and function during DNA damage, Genes Dev., 25 (2011), 220-225. |
| [79] | W. Huang, P. J. Smaldino, Q. Zhang, L. D. Miller, P. Cao, K. Stadelman, et al., Yin Yang 1 contains G-quadruplex structures in its promoter and 5'-UTR and its expression is modulated by G4 resolvase 1, Nucleic Acids Res., 40 (2012), 1033-1049. |
| [80] | A. Arora, M. Dutkiewicz, V. Scaria, M. Hariharan, S. Maiti, J. Kurreck, Inhibition of translation in living eukaryotic cells by an RNA G-quadruplex motif, RNA, 14 (2008), 1290-1296. |
Figures(4) / Tables(6)
Siddhartha Kundu. Mathematical model of a short translatable G-quadruplex and an assessment of its relevance to misfolding-induced proteostasis[J]. Mathematical Biosciences and Engineering, 2020, 17(3): 2470-2493. doi: 10.3934/mbe.2020135
| Stage | Template | ||
| enc1 | conv 3 × 3, stride = 1, D = 64, BN + ReLU | max_pooling 2 × 2, stride = 2 | ×2 |
| enc2 | conv 3 × 3, stride = 1, D = 128, BN + ReLU | max_pooling 2 × 2, stride = 2 | ×2 |
| enc3 | conv 3 × 3, stride = 1, D = 256, BN + ReLU | max_pooling 2 × 2, stride = 2 | ×2 |
| enc4 | conv 3 × 3, stride = 1, D = 512, BN + ReLU | max_pooling 2 × 2, stride = 2 | ×2 |
| enc5 | conv 3 × 3, stride = 1, D = 512, BN + ReLU | max_pooling 2 × 2, stride = 2 | ×2 |
DownLoad:
CSV
| Method | BSCA[49] | FT[50] | MIL[51] | RC[52] | SMD[53] | FCN[40] | UNet[41] | DN[54] | DHSNet[55] | DSS[56] | My |
| MAE↓ | 0.2462 | 0157. 7 | 0.1764 | 0.1228 | 0.1913 | 0.0604 | 0.0643 | 0.0481 | 0.0399 | 0.0371 | 0.0369 |
| Fβ | 0.3404 | 0.4462 | 0.4873 | 0.3719 | 0.4665 | 0.6788 | 0.6783 | 0.7272 | 0.8447 | 0.8051 | 0.8485 |
DownLoad:
CSV
| Method | MAE | Fβ |
| CMN−+ lBCE | 0.0511 | 0.6977 |
| CMN++lBCE | 0.0490 | 0.7025 |
| CMN++L | 0.0369 | 0.8485 |
DownLoad:
CSV
| Stage | Template | ||
| enc1 | conv 3 × 3, stride = 1, D = 64, BN + ReLU | max_pooling 2 × 2, stride = 2 | ×2 |
| enc2 | conv 3 × 3, stride = 1, D = 128, BN + ReLU | max_pooling 2 × 2, stride = 2 | ×2 |
| enc3 | conv 3 × 3, stride = 1, D = 256, BN + ReLU | max_pooling 2 × 2, stride = 2 | ×2 |
| enc4 | conv 3 × 3, stride = 1, D = 512, BN + ReLU | max_pooling 2 × 2, stride = 2 | ×2 |
| enc5 | conv 3 × 3, stride = 1, D = 512, BN + ReLU | max_pooling 2 × 2, stride = 2 | ×2 |
| Method | BSCA[49] | FT[50] | MIL[51] | RC[52] | SMD[53] | FCN[40] | UNet[41] | DN[54] | DHSNet[55] | DSS[56] | My |
| MAE↓ | 0.2462 | 0157. 7 | 0.1764 | 0.1228 | 0.1913 | 0.0604 | 0.0643 | 0.0481 | 0.0399 | 0.0371 | 0.0369 |
| Fβ | 0.3404 | 0.4462 | 0.4873 | 0.3719 | 0.4665 | 0.6788 | 0.6783 | 0.7272 | 0.8447 | 0.8051 | 0.8485 |
| Method | MAE | Fβ |
| CMN−+ lBCE | 0.0511 | 0.6977 |
| CMN++lBCE | 0.0490 | 0.7025 |
| CMN++L | 0.0369 | 0.8485 |
